Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add volumeClaimTemplates to raft storage api #5

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Add volumeClaimTemplates to raft storage api #5

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f0b0315
Add persistent volume claim to raft database
adibrastegarnia Apr 23, 2020
5b6ed64
remove debug statements
adibrastegarnia Apr 24, 2020
d0ad7e0
Update raft storage controller
adibrastegarnia Apr 24, 2020
7a60b9a
Update raft storage controller manifest
adibrastegarnia Apr 26, 2020
efaaa2c
Update examples
adibrastegarnia Apr 26, 2020
9ad46d3
Change namespace to kube-system
adibrastegarnia Apr 26, 2020
7c97b02
New design of api
adibrastegarnia Apr 27, 2020
f6203b2
Update manifest
adibrastegarnia Apr 27, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 23 additions & 16 deletions deploy/examples/raft-storage.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,26 +1,33 @@
apiVersion: storage.cloud.atomix.io/v1beta1
kind: RaftStorageClass
metadata:
name: raft
labels:
app: raft
name: raft-database
labels:
app: raft-database
spec:
image: atomix/raft-replica:latest
image: atomix/raft-storage-node:v0.1.0
imagePullPolicy: IfNotPresent
replicas: 1
partitionsPerCluster: 3
volumeClaimTemplates:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we keep this here as an example?

- metadata:
name: data
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "standard"
resources:
requests:
storage: 1Gi
---
apiVersion: cloud.atomix.io/v1beta2
apiVersion: cloud.atomix.io/v1beta3
kind: Database
metadata:
name: raft
name: raft-database
spec:
clusters: 3
template:
spec:
partitions: 10
storage:
group: storage.cloud.atomix.io
version: v1beta1
kind: RaftStorageClass
name: raft
namespace: kube-system
partitions: 1
storageClass:
group: storage.cloud.atomix.io
version: v1beta1
kind: RaftStorageClass
name: raft-database
namespace: default
103 changes: 87 additions & 16 deletions deploy/raft-storage-controller.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,23 +29,94 @@ spec:
type: string
replicas:
type: integer
min: 1
max: 9
default: 1
partitionsPerCluster:
type: integer
min: 1
max: 1024
default: 1
volumeClaimTemplates:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just convert this to YAML just so it's clear and so we're sure we can use it in the Helm chart as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will figure that out. It looks like it is working in helm chart as well :)

type: array
"items": {
"type": object,
"required": [
"metadata",
"spec"
],
"properties": {
"metadata": {
"$id": "#/items/properties/metadata",
"type": "object",
"default": {},
"required": [
"name"
],
"properties": {
"name": {
"$id": "#/items/properties/metadata/properties/name",
"type": "string",
"default": "",

}
}
},
"spec": {
"$id": "#/items/properties/spec",
"type": "object",
"default": {},
"required": [
"accessModes",
"storageClassName",
"resources"
],
"properties": {
"accessModes": {
"$id": "#/items/properties/spec/properties/accessModes",
"type": "array",
"default": [],

"items": {
"$id": "#/items/properties/spec/properties/accessModes/items",
"type": "string",
"default": "",

}
},
"storageClassName": {
"$id": "#/items/properties/spec/properties/storageClassName",
"type": "string",
"default": "",

},
"resources": {
"$id": "#/items/properties/spec/properties/resources",
"type": "object",
"default": {},
"required": [
"requests"
],
"properties": {
"requests": {
"$id": "#/items/properties/spec/properties/resources/properties/requests",
"type": "object",
"default": {},
"required": [
"storage"
],
"properties": {
"storage": {
"$id": "#/items/properties/spec/properties/resources/properties/requests/properties/storage",
"type": "string",
"default": "",

}
}
}
}
}
}
}
}
}

---
apiVersion: cloud.atomix.io/v1beta2
kind: StorageController
metadata:
name: raft
spec:
group: storage.cloud.atomix.io
version: v1beta1
kind: RaftStorageClass
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
Expand Down Expand Up @@ -97,7 +168,7 @@ metadata:
subjects:
- kind: ServiceAccount
name: raft-storage-controller
namespace: kube-system
namespace: default
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a cluster scoped controller it should be deployed in the kube-system namespace as is convention for all controllers

roleRef:
kind: ClusterRole
name: raft-storage-controller
Expand All @@ -107,13 +178,13 @@ apiVersion: v1
kind: ServiceAccount
metadata:
name: raft-storage-controller
namespace: kube-system
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: raft-storage-controller
namespace: kube-system
namespace: default
spec:
replicas: 1
selector:
Expand Down
3 changes: 3 additions & 0 deletions pkg/apis/storage/v1beta1/storageclass.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ type RaftStorageClassSpec struct {

// ImagePullPolicy is the pull policy to apply
ImagePullPolicy corev1.PullPolicy `json:"imagePullPolicy,omitempty"`

// VolumeClaim
VolumeClaimTemplates []corev1.PersistentVolumeClaim `json:"volumeClaimTemplates,omitempty"`
}

// +genclient
Expand Down
59 changes: 40 additions & 19 deletions pkg/controller/reconciler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,34 @@ func (r *Reconciler) reconcileStatefulSet(database *v1beta3.Database, storage *v
return err
}

func newDataVolume(database *v1beta3.Database, storage *v1beta1.RaftStorageClass, cluster int) corev1.Volume {
dataVolume := corev1.Volume{
Name: dataVolume,
VolumeSource: corev1.VolumeSource{
EmptyDir: &corev1.EmptyDirVolumeSource{},
},
}

return dataVolume

}

func newConfigVolume(database *v1beta3.Database, storage *v1beta1.RaftStorageClass, cluster int) corev1.Volume {
configVolume := corev1.Volume{

Name: configVolume,
VolumeSource: corev1.VolumeSource{
ConfigMap: &corev1.ConfigMapVolumeSource{
LocalObjectReference: corev1.LocalObjectReference{
Name: getClusterName(database, cluster),
},
},
},
}
return configVolume

}

func (r *Reconciler) addStatefulSet(database *v1beta3.Database, storage *v1beta1.RaftStorageClass, cluster int) error {
log.Info("Creating raft replicas", "Name", database.Name, "Namespace", database.Namespace)

Expand All @@ -274,6 +302,15 @@ func (r *Reconciler) addStatefulSet(database *v1beta3.Database, storage *v1beta1
pullPolicy = corev1.PullIfNotPresent
}

volumes := []corev1.Volume{
newConfigVolume(database, storage, cluster),
}

if storage.Spec.VolumeClaimTemplates == nil {

volumes = append(volumes, newDataVolume(database, storage, cluster))
}

set := &appsv1.StatefulSet{
ObjectMeta: metav1.ObjectMeta{
Name: getClusterName(database, cluster),
Expand All @@ -289,7 +326,8 @@ func (r *Reconciler) addStatefulSet(database *v1beta3.Database, storage *v1beta1
UpdateStrategy: appsv1.StatefulSetUpdateStrategy{
Type: appsv1.RollingUpdateStatefulSetStrategyType,
},
PodManagementPolicy: appsv1.ParallelPodManagement,
PodManagementPolicy: appsv1.ParallelPodManagement,
VolumeClaimTemplates: storage.Spec.VolumeClaimTemplates,
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: newClusterLabels(database, cluster),
Expand Down Expand Up @@ -356,24 +394,7 @@ func (r *Reconciler) addStatefulSet(database *v1beta3.Database, storage *v1beta1
},
},
},
Volumes: []corev1.Volume{
{
Name: configVolume,
VolumeSource: corev1.VolumeSource{
ConfigMap: &corev1.ConfigMapVolumeSource{
LocalObjectReference: corev1.LocalObjectReference{
Name: getClusterName(database, cluster),
},
},
},
},
{
Name: dataVolume,
VolumeSource: corev1.VolumeSource{
EmptyDir: &corev1.EmptyDirVolumeSource{},
},
},
},
Volumes: volumes,
},
},
},
Expand Down