Skip to content

Commit

Permalink
refactor
Browse files Browse the repository at this point in the history 
Signed-off-by: SK Ali Arman <[email protected]>
  • Loading branch information
@sheikh-arman
sheikh-arman committed Dec 2, 2024
1 parent bc9944d commit 3cee601
Show file tree
Hide file tree
Showing 3 changed files with 71 additions and 30 deletions.
60 changes: 39 additions & 21 deletions docs/guides/mysql/pitr/restic/archiver.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ spec:
last: 2
```
```bash
$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/yamls/retention-policy.yaml
$ kubectl apply -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/restic/yamls/retention-policy.yaml
retentionpolicy.storage.kubestash.com/mysql-retention-policy created
```

Expand Down Expand Up @@ -164,9 +164,9 @@ stringData:
```
```bash
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pirt/yamls/mysqlarchiver.yaml
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/restic/yamls/mysqlarchiver.yaml
mysqlarchiver.archiver.kubedb.com/mysqlarchiver-sample created
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pirt/yamls/encryptionSecret.yaml
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/restic/yamls/encryptionSecret.yaml
```

# Deploy MySQL
Expand Down Expand Up @@ -199,6 +199,11 @@ spec:
deletionPolicy: WipeOut
```
```bash
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/restic/yamls/mysql.yaml
mysql.kubedb.com/mysql created
```

```bash
$ kubectl get pod -n demo
NAME READY STATUS RESTARTS AGE
Expand All @@ -214,9 +219,14 @@ retention-policy-mysql-archiver-manifest-backup-1733120326l79mb 0/1 Comple

`mysql-sidekick` is responsible for uploading binlog files

The pod mysql-archiver-full-backup-1733120326-4n8nd is responsible for creating the base backup of MySQL.
`mysql-archiver-full-backup-1733120326-4n8nd` is responsible for creating the base backup of MySQL.

`mysql-archiver-manifest-backup-1733120326-9gw4f` is the pod of the manifest backup related to MySQL object.

`retention-policy-mysql-archiver-full-backup-1733120326-7rx9t` will automatically clean up previous full-backup snapshots according to the rules defined in the `mysql-retention-policy` custom resource (CR).

`retention-policy-mysql-archiver-manifest-backup-1733120326l79mb` will automatically clean up previous manifest-backup snapshots according to the rules specified in the `mysql-retention-policy` custom resource (CR).

### Validate BackupConfiguration and BackupSession

```bash
Expand All @@ -230,11 +240,10 @@ $ kubectl get backupsession -n demo
NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE
mysql-archiver-full-backup-1733120326 BackupConfiguration mysql-archiver Succeeded 50s 14m
mysql-archiver-manifest-backup-1733120326 BackupConfiguration mysql-archiver Succeeded 25s 14m
1Gi longhorn-snapshot-vsc snapcontent-735e97ad-1dfa-4b70-b416-33f7270d792c 2m5s 2m5s
```

## data insert and switch wal
After each and every wal switch the wal files will be uploaded to backup storage
## Data Insert and Switch Binlog File
After each and every binlog switch the binlog files will be uploaded to backup storage

```bash
$ kubectl exec -it -n demo mysql-0 -- bash
Expand Down Expand Up @@ -286,7 +295,7 @@ mysql> select count(*) from demo_table;
## Point-in-time Recovery
Point-In-Time Recovery allows you to restore a MySQL database to a specific point in time using the archived transaction logs. This is particularly useful in scenarios where you need to recover to a state just before a specific error or data corruption occurred.
Let's say accidentally our dba drops the the table tab_1 and we want to restore.
Let's say accidentally our db drops the the table `demo_table` and we want to restore that.
```bash
$ kubectl exec -it -n demo mysql-0 -- bash
Expand All @@ -296,10 +305,9 @@ mysql> drop table demo_table;
mysql> flush logs;
```
We can't restore from a full backup since at this point no full backup was perform. so we can choose a specific time in which time we want to restore.We can get the specfice time from the wal that archived in the backup storage . Go to the binlog file and find where to store. You can parse binlog-files using `mysqlbinlog`.
We can't restore from a full backup since at this point no full backup was perform. so we can choose a specific time in which time we want to restore.We can get the specific time from the binlog that archived in the backup storage . Go to the binlog file and find where to store. You can parse binlog-files using `mysqlbinlog`.
For the demo I will use the previous time we get form `select now()`
For the demo I will use the previous time we get from `select now()`
```bash
mysql> select now();
Expand All @@ -311,6 +319,8 @@ mysql> select now();
```
### Restore MySQL
```yaml
apiVersion: kubedb.com/v1
kind: MySQL
Expand Down Expand Up @@ -343,30 +353,35 @@ spec:

```
```bash
$ kubectl apply -f restore.yaml
mysql.kubedb.com/restore-mysql created
```
```bash
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/restic/yamls/mysql-restore.yaml
mysql.kubedb.com/restore-mysql created
```
**check for Restored MySQL**
**Check for Restored MySQL**
```bash
$ kubectl get pod -n demo
data-restore-mysql-0-pvc-restorer-5vtj7 0/1 Completed 0 7m40s
restore-mysql-0 2/2 Running 0 6m40s
restore-mysql-1 2/2 Running 0 5m37s
restore-mysql-2 2/2 Running 0 5m21s
restore-mysql-binlog-restorer-0 0/2 Completed 0 5m58s
restore-mysql-manifest-restorer-pzx5z 0/1 Completed 0 6m54s
```
The pod `data-restore-mysql-0-pvc-restorer-5vtj7` is responsible for restoring the base backup.
The pod `restore-mysql-binlog-restorer-0` is responsible for restoring the binlog file.
```bash
$ kubectl get mysql -n demo
NAME VERSION STATUS AGE
mysql.kubedb.com/mysql 8.2.0 Ready 32m
mysql.kubedb.com/restore-mysql 8.2.0 Ready 2m53s
```
**Validating data on Restored MySQL**
**Validating Data on Restored MySQL**
```bash
$ kubectl exec -it -n demo restore-mysql-0 -- bash
Expand Down Expand Up @@ -404,16 +419,19 @@ The base backup and binlog files are restored exclusively on pod-0. Other replic
***fscopy***
The base backup and binlog files are restored on pod-0. The data is then copied from pod-0's data directory to the data directories of other replicas using file system copy. Once the data transfer is complete, the group replication process begins.
The base backup and binlog files are restored on pod-0. The data is then copied from pod-0's data directory to the data directories of other replicas using file system copy. Once the data transfer is complete, the group replication process begins. Please note that `fscopy` does not support cross-zone operations.
Choose the replication strategy that best fits your restoration and replication requirements. On this demonstration, we have used the sync replication strategy.
## Cleaning up
Choose the replication strategy that best fits your restoration and replication requirements.
To cleanup the Kubernetes resources created by this tutorial, run:
```bash
$ kubectl delete -n demo mysql/mysql
$ kubectl delete -n demo mysql/restore-mysql
$ kubectl delete -n demo backupstorage
$ kubectl delete -n demo mysqlarchiver
$ kubectl delete -n demo backupstorage.storage.kubestash.com/storage
$ kubectl delete -n demo mysqlarchiver/mysqlarchiver-sample
$ kubectl delete ns demo
```
Expand Down
0 ...ql/pitr/restic/yamls/retentionPolicy.yaml → ...l/pitr/restic/yamls/retention-policy.yaml
View file
File renamed without changes.
41 changes: 32 additions & 9 deletions docs/guides/mysql/pitr/volumesnapshot/archiver.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ section_menu_id: guides

> New to KubeDB? Please start [here](/docs/README.md).
# KubeDB MySQL - Continuous Archiving and Point-in-time Recovery
# KubeDB MySQL - Continuous Archiving and Point-in-time Recovery using VolumeSnapShot

Here, will show you how to use KubeDB to provision a MySQL to Archive continuously and Restore point-in-time.

Expand All @@ -24,8 +24,6 @@ Now,install `KubeDB` operator in your cluster following the steps [here](/docs/s

To install `KubeStash` operator in your cluster following the steps [here](https://github.com/kubestash/installer/tree/master/charts/kubestash).

To install `SideKick` in your cluster following the steps [here](https://github.com/kubeops/installer/tree/master/charts/sidekick).

To install `External-snapshotter` in your cluster following the steps [here](https://github.com/kubernetes-csi/external-snapshotter/tree/release-5.0).

To keep things isolated, this tutorial uses a separate namespace called `demo` throughout this tutorial.
Expand All @@ -34,9 +32,9 @@ To keep things isolated, this tutorial uses a separate namespace called `demo` t
$ kubectl create ns demo
namespace/demo created
```
> Note: The yaml files used in this tutorial are stored in [docs/guides/mysql/remote-replica/yamls](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/guides/mysql/remote-replica/yamls) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs).
> Note: The yaml files used in this tutorial are stored in [docs/guides/mysql/pitr/volumesnapshot/yamls](https://github.com/kubedb/docs/tree/{{< param "info.version" >}}/docs/guides/mysql/pitr/volumesnapshot/yamls) folder in GitHub repository [kubedb/docs](https://github.com/kubedb/docs).
## continuous archiving
## Continuous Archiving
Continuous archiving involves making regular copies (or "archives") of the MySQL transaction log files.To ensure continuous archiving to a remote location we need prepare `BackupStorage`,`RetentionPolicy`,`MySQLArchiver` for the KubeDB Managed MySQL Databases.

### BackupStorage
Expand Down Expand Up @@ -172,9 +170,9 @@ stringData:
```
```bash
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pirt/yamls/mysqlarchiver.yaml
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/volumesnapshot/yamls/mysqlarchiver.yaml
mysqlarchiver.archiver.kubedb.com/mysqlarchiver-sample created
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pirt/yamls/encryptionSecret.yaml
$ kubectl create -f https://github.com/kubedb/docs/raw/{{< param "info.version" >}}/docs/guides/mysql/pitr/volumesnapshot/yamls/encryptionSecret.yaml
```

## Ensure volumeSnapshotClass
Expand Down Expand Up @@ -205,6 +203,8 @@ $ kubectl apply -f volumesnapshotclass.yaml
volumesnapshotclass.snapshot.storage.k8s.io/longhorn-snapshot-vsc unchanged
```

Note: Ensure that the VolumeSnapshotClass is provisioned with the same storage class driver used for provisioning your MySQL database. In our case, we are using the longhorn storage class as our database provisioner, with the driver set to `driver.longhorn.io`.

# Deploy MySQL
So far we are ready with setup for continuously archive MySQL, We deploy a mysqlql referring the MySQL archiver object

Expand Down Expand Up @@ -256,7 +256,7 @@ mysql-sidekick 1/1 Running 0

`mysql-backup-config-manifest-1703680982-62x97` are the pod of the manifest backup related to MySQL object

### validate BackupConfiguration and VolumeSnapshots
### Validate BackupConfiguration and VolumeSnapshots

```bash

Expand All @@ -275,7 +275,7 @@ NAME READYTOUSE SOURCEPVC SOURCESNA
mysql-1702388096 true data-mysql-1 1Gi longhorn-snapshot-vsc snapcontent-735e97ad-1dfa-4b70-b416-33f7270d792c 2m5s 2m5s
```

## data insert and switch wal
## Data Insert and Switch Binlog File
After each and every wal switch the wal files will be uploaded to backup storage

```bash
Expand Down Expand Up @@ -432,6 +432,29 @@ mysql> select count(*) from demo_table;
**so we are able to successfully recover from a disaster**
**ReplicationStrategy**
The ReplicationStrategy determines how MySQL restores are managed when using the VolumeSnapshot. We support three strategies: `none`, `sync`, and `fscopy`, with `none` being the default.
To configure the desired strategy, set the `spec.init.archiver.replicationStrategy` field in your configuration. These strategies are applicable only when restoring a MySQL database in group replication mode.
**Strategies Overview:**
***none***
Each MySQL replica independently restores the base backup volumesnapshot and binlog files. After completing the restore process, the replicas individually join the replication group.
***sync***
The base backup volumesnapshot and binlog files are restored exclusively on pod-0. Other replicas then synchronize their data by leveraging the MySQL clone plugin to replicate from pod-0.
***fscopy***
The base backup and binlog files are restored on pod-0. The data is then copied from pod-0's data directory to the data directories of other replicas using file system copy. Once the data transfer is complete, the group replication process begins. Please note that `fscopy` does not support cross-zone operations.
Choose the replication strategy that best fits your restoration and replication requirements. On this demonstration, we have used the sync replication strategy.
## Cleaning up
To cleanup the Kubernetes resources created by this tutorial, run:
Expand Down

0 comments on commit 3cee601

Please sign in to comment.