This repository has been archived by the owner on Jan 21, 2020. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 262
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: YujiOshima <[email protected]>
- Loading branch information
1 parent
fd983ac
commit 289516b
Showing
1,168 changed files
with
473,206 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
InfraKit Flavor Plugin - Swarm | ||
============================== | ||
|
||
A [reference](/README.md#reference-implementations) implementation of a Flavor Plugin that creates a [Kubernetes](https://kubernetes.io/) cluster. | ||
|
||
## Schema & Templates | ||
|
||
This plugin has a schema that looks like this: | ||
```json | ||
{ | ||
"InitScriptTemplateURL": "file:///home/ubuntu/go/src/github.com/docker/infrakit/examples/flavor/kubernetes/manager-init.sh", | ||
"KubeJoinIP": "192.168.2.200", | ||
"KubeBindPort": 6443, | ||
"KubeNWAddOn": "flannel" | ||
} | ||
``` | ||
Note that the Kubernetes connection information, as well as what IP in the Kubernetes managers and workers should use | ||
to advertise and join. | ||
|
||
This plugin makes heavy use of Golang template to enable customization of instance behavior on startup. For example, | ||
the `InitScriptTemplateURL` field above is a URL where a init script template is served. The plugin will fetch this | ||
template from the URL and processe the template to render the final init script for the instance. | ||
|
||
The plugin exposes a set of template functions that can be used, along with primitives already in [Golang template] | ||
(https://golang.org/pkg/text/template/) and functions from [Sprig](https://github.com/Masterminds/sprig#functions). | ||
This makes it possible to have complex templates for generating the user data / init script of the instances. | ||
|
||
For example, this is a template for the init script of a manager node: | ||
|
||
``` | ||
#!/bin/sh | ||
set -o errexit | ||
set -o nounset | ||
set -o xtrace | ||
{{/* Install Docker */}} | ||
{{ include "install-docker.sh" }} | ||
{{/* Install Kubeadm */}} | ||
{{ include "install_kubeadam.sh" }} | ||
kubeadm init --token {{ KUBEADM_JOIN_TOKEN }} | ||
export KUBECONFIG=/etc/kubernetes/admin.conf | ||
{{ if NETWORK_ADDON }} | ||
kubectl apply -f {{ NETWORK_ADDON }} | ||
{{ else }} | ||
{{ end }} | ||
``` | ||
|
||
There are tags such as `{{ KUBEADM_JOIN_TOKEN }}` or `{{ INSTANCE_LOGICAL_ID }}`: these are made available by the | ||
plugin and they are evaluated / interpolated during the `Prepare` phase of the plugin. The plugin will substitute | ||
these 'placeholders' with actual values. The templating engine also supports inclusion of other templates / files, as | ||
seen in the `{{ include "install-docker.sh" }}` tag above. This makes it easy to embed actual shell scripts, and other | ||
texts, without painful and complicated escapes to meet the JSON syntax requirements. For example, the 'include' tag | ||
above will embed the `install-docker.sh` template/file: | ||
|
||
``` | ||
# Tested on Ubuntu/trusty | ||
apt-get update -y | ||
wget -qO- https://get.docker.com/ | sh | ||
``` | ||
|
||
### A Word on Security | ||
|
||
Since Kubeadm use Token to authorize nodes, initializing | ||
the Kubernetes requires: | ||
|
||
Docken socke API server exposes the remote API, but it is protected by TLS. Infrakit intends to make access to kubernetes manager from the side, but we can not send commands such as `get nodes` yet. | ||
For installation, we use [kubeadm](https://kubernetes.io/docs/admin/kubeadm/) and build a secure cluster. | ||
|
||
|
||
### Building & Running -- An Example | ||
|
||
There are scripts in this directory to illustrate how to start up the InfraKit plugin ensemble and examples for creating | ||
a Docker swarm via vagrant. | ||
|
||
Building the binaries - do this from the top level project directory: | ||
```shell | ||
make binaries | ||
``` | ||
|
||
Start required plugins. We use the `infrakit plugin start` utility and a `plugins.json` to start up all the plugins, | ||
along with the InfraKit manager: | ||
|
||
```shell | ||
infrakit-group-default | ||
infrakit-instance-vagrant | ||
infrakit-flavor-kubernetes | ||
``` | ||
|
||
Now start up the cluster comprised of a manager and a worker group. In this case, see `groups-master.json` where we will create a manager group of one node and in `group-worker.json` create a worker group of 3 nodes. The topology in this is a single ensemble of infrakit running on your local machine that manages 4 vagrant vms running Kubernetes. | ||
At Kubernetes flavor, you should run manager group first. | ||
Worker group will try to connect to manager before start. | ||
And as this flavor based on kubeadm, currently it support only one manager node. | ||
|
||
```shell | ||
infrakit group commit groups-manager.json | ||
``` | ||
Wait for manager comes up. | ||
As it need to install docker and kubeadm, it take a little time... | ||
|
||
```shell | ||
infrakit group commit groups-worker.json | ||
``` | ||
|
||
Now cluster will come up. | ||
Now check the kubernetes: | ||
You should log in to manager node. | ||
Then | ||
|
||
```shell | ||
export KUBECONFIG=/etc/kubernetes/admin.conf | ||
kubectl get nodes | ||
NAME STATUS AGE VERSION | ||
ip-192.168.2.200 Ready 4m v1.6.3 | ||
ip-192.168.2.2 Ready 2m v1.6.3 | ||
ip-192.168.2.3 Ready 2m v1.6.3 | ||
ip-192.168.2.4 Ready 2m v1.6.3 | ||
``` |
Oops, something went wrong.