add doc explaining locations #1050
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jrnt30
reviewed
Nov 10, 2018
docs/locations.md
Outdated
| ark backup create full-cluster-backup | ||
| ``` | ||
|
|
||
| ## Limitations / Caveats |
There was a problem hiding this comment.
I think this is a helpful item and would be good to throw before the examples to provide context.
An additional limitation that is somewhat implied here but may be useful to call out explicitly is that there are no cross-provider snapshots supported either. When there are multiple VSLs for a given provider that will only be used for the volumes which are compatible.
|
I rearranged based on @jrnt30's feedback, and added some more color to the overview. Still need to fill out some details in the additional use cases section. PTAL if anyone has time! |
skriss
changed the title
wwitzel3
approved these changes
Nov 13, 2018
docs/locations.md
Outdated
|
|
||
| ## Motivations | ||
|
|
||
| In Ark prior to v0.10, the configuration for where to store backups & volume snapshots was stored in a `Config` custom resource. The `backupStorageProvider` section captured the place where all Ark backups should be stored. This was defined by a **provider** (e.g. `aws`, `azure`, `gcp`, `minio`, etc.), a **bucket**, and possibly some additional provider-specific settings (e.g. `region`). Similarly, the `persistentVolumeProvider` section captured the place where all persistent volume snapshots taken as part of Ark backups should be stored, and was defined by a **provider** and additional provider-specific settings (e.g. `region`). |
There was a problem hiding this comment.
I wonder about changing the tense here? Don't have a strong opinion, the paragraph just reads a little weird to me, prior versions still exist and store things that way. The past tense maybe right here, but something about how it read felt off to me.
In Ark versions prior to v0.10, the configuration for backup and volume snapshot locations is stored in a `Config` custom resource.
There was a problem hiding this comment.
I agree with @wwitzel3 on the tense. Because that version is not deprecated, it would sound more natural to talk about it in the present term because it is still in use.
docs/locations.md
Outdated
| - Have some Ark backups go to a bucket in an eastern USA region, and others go to a bucket in a western USA region | ||
| - For volume providers that support it (e.g. Portworx), have some snapshots be stored locally on the cluster and have others be stored in the cloud | ||
|
|
||
| Additionally, as we looked ahead to a major feature on our roadmap, backup replication, we knew that we'd need Ark to be able to support multiple possible storage locations. |
There was a problem hiding this comment.
Maybe reword the very last part? Something like ...
Additionally, as we look ahead to the backup replication feature, we know that we want to support having multiple storage locations.
docs/locations.md
Outdated
|
|
||
| ## Limitations / Caveats | ||
|
|
||
| - For volume snapshots, you are still limited by where your provider allows you to create snapshots. For example, AWS and Azure do not allow you to create a volume snapshot in a different region than where the volume is. If you try to take an Ark backup using a volume snapshot location with a different region than where your cluster's volumes are, the backup will fail. |
There was a problem hiding this comment.
Suggestion
Volume snapshots are still limited by where the provider allows you to create snapshots.
docs/locations.md
Outdated
|
|
||
| - Cross-provider snapshots are not supported. If you have a cluster with more than one type of volume (e.g. EBS and Portworx), but you only have a `VolumeSnapshotLocation` configured for EBS, then Ark will **only** snapshot the EBS volumes. | ||
|
|
||
| - If you're using Ark's restic integration, restic data is now stored under a prefix/subdirectory of the main Ark bucket, and will go into the bucket corresponding to the `BackupStorageLocation` selected by the user at backup creation time. |
There was a problem hiding this comment.
I don't think this needs to be qualified.
Restic data is now stored under a prefix/subdirectory of the main Ark bucket, and will go into the bucket corresponding to the `BackupStorageLocation` selected by the user at backup creation time.
wwitzel3
reviewed
Nov 13, 2018
jrnt30
reviewed
Nov 14, 2018
docs/locations.md
Outdated
|
|
||
| - For volume snapshots, you are still limited by where your provider allows you to create snapshots. For example, AWS and Azure do not allow you to create a volume snapshot in a different region than where the volume is. If you try to take an Ark backup using a volume snapshot location with a different region than where your cluster's volumes are, the backup will fail. | ||
|
|
||
| - Each Ark backup has one `BackupStorageLocation`, and one `VolumeSnapshotLocation` per volume provider. It is not possible (yet) to send a single Ark backup to multiple backup storage locations simultaneously, and similarly for volume snapshots. However, you can always set up multiple scheduled backups that differ only in the storage locations used if redundancy of backups across locations is important. |
There was a problem hiding this comment.
If I understand the current implication, it may be worth expanding on the and similarly for volume snapshots. to indicate that it is simply not possible to have multiple VSLs for the same provider.
Suggested change
| - Each Ark backup has one `BackupStorageLocation`, and one `VolumeSnapshotLocation` per volume provider. It is not possible (yet) to send a single Ark backup to multiple backup storage locations simultaneously, and similarly for volume snapshots. However, you can always set up multiple scheduled backups that differ only in the storage locations used if redundancy of backups across locations is important. | |
| - Each Ark backup has one `BackupStorageLocation`, and one `VolumeSnapshotLocation` per volume provider. It is not possible (yet) to send a single Ark backup to multiple backup storage locations simultaneously, and for volume snapshots you are limited to a single volume snapshot per Provider/Location. You can set up multiple scheduled backups that differ only in the backup storage location used if redundancy of backups across locations is important. |
jrnt30
reviewed
Nov 14, 2018
|
|
||
| - Each Ark backup has one `BackupStorageLocation`, and one `VolumeSnapshotLocation` per volume provider. It is not possible (yet) to send a single Ark backup to multiple backup storage locations simultaneously, and similarly for volume snapshots. However, you can always set up multiple scheduled backups that differ only in the storage locations used if redundancy of backups across locations is important. | ||
|
|
||
| - Cross-provider snapshots are not supported. If you have a cluster with more than one type of volume (e.g. EBS and Portworx), but you only have a `VolumeSnapshotLocation` configured for EBS, then Ark will **only** snapshot the EBS volumes. |
There was a problem hiding this comment.
Trying to clear up a bit to my current understanding (which may be off) which is that you can have a backup span multiple VSLs, they are just constrained by the volume's underlying provider.
Suggested change
| - Cross-provider snapshots are not supported. If you have a cluster with more than one type of volume (e.g. EBS and Portworx), but you only have a `VolumeSnapshotLocation` configured for EBS, then Ark will **only** snapshot the EBS volumes. | |
| - Cross-provider snapshots are not supported. If you have a cluster with more than one type of volume then Ark will **only** snapshot the volumes for their underlying provider. As an example, if we had a cluster with Portworx and EBS volumes, and a `VolumeStorageLocation` defined for each provider, the EBS volumes would only be snapshot by the EBS `VolumeStorageLocation` and the Portworx would only be snapshot by the Portworx `VolumeStorageLocation`. |
jrnt30
reviewed
Nov 14, 2018
| During backup creation: | ||
|
|
||
| ```shell | ||
| # Note that since in this example we have two possible volume snapshot locations for the Portworx |
There was a problem hiding this comment.
Is it worth calling out anywhere that the server also has some additional flags now to indicate which of these VSL (provider/location tuple) should be the default?
skriss
force-pushed
the
explain-locations
branch
from
d9af389 to
5a12c18
Compare
|
more updates based on feedback. |
carlisia
approved these changes
Nov 14, 2018
docs/locations.md
Outdated
|
|
||
| ## Motivations | ||
|
|
||
| In Ark versions prior to v0.10, the configuration for where to store backups & volume snapshots is stored in a `Config` custom resource. The `backupStorageProvider` section captures the place where all Ark backups should be stored. This is defined by a **provider** (e.g. `aws`, `azure`, `gcp`, `minio`, etc.), a **bucket**, and possibly some additional provider-specific settings (e.g. `region`). Similarly, the `persistentVolumeProvider` section captures the place where all persistent volume snapshots taken as part of Ark backups should be stored, and is defined by a **provider** and additional provider-specific settings (e.g. `region`). |
There was a problem hiding this comment.
Suggested change
| In Ark versions prior to v0.10, the configuration for where to store backups & volume snapshots is stored in a `Config` custom resource. The `backupStorageProvider` section captures the place where all Ark backups should be stored. This is defined by a **provider** (e.g. `aws`, `azure`, `gcp`, `minio`, etc.), a **bucket**, and possibly some additional provider-specific settings (e.g. `region`). Similarly, the `persistentVolumeProvider` section captures the place where all persistent volume snapshots taken as part of Ark backups should be stored, and is defined by a **provider** and additional provider-specific settings (e.g. `region`). | |
| In Ark versions prior to v0.10, the configuration for where to store backups & volume snapshots is specified in a `Config` custom resource. The `backupStorageProvider` section captures the place where all Ark backups should be stored. This is defined by a **provider** (e.g. `aws`, `azure`, `gcp`, `minio`, etc.), a **bucket**, and possibly some additional provider-specific settings (e.g. `region`). Similarly, the `persistentVolumeProvider` section captures the place where all persistent volume snapshots taken as part of Ark backups should be stored, and is defined by a **provider** and additional provider-specific settings (e.g. `region`). |
There was a problem hiding this comment.
Super minor change just to avoid repeating the concept of "storage" and perhaps avoid confusion.
Signed-off-by: Steve Kriss <[email protected]>
skriss
force-pushed
the
explain-locations
branch
from
5a12c18 to
2295907
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Steve Kriss [email protected]
Still a WIP but dumped as much as I could for today. Is this a useful document? What's missing or unclear?
If we all like this then I'd like to link to it prominently, in the release notes, probably the blog post, etc., so that folks can read it and get the background before diving into upgrading.
cc @Bradamant3 @nrb @wwitzel3 @carlisia @jrnt30