Documentation updates

Author: chrislovecnm

CHANGELOG.md

@@ -0,0 +1 @@
+Our change log is available [here](docs/releases/).

HISTORY.md

@@ -39,6 +39,7 @@ launching a Kubernetes cluster hosted on AWS.
 * Ability to generate configuration files for AWS [CloudFormation](https://aws.amazon.com/cloudformation/) and Terraform [Terraform configuration](/docs/terraform.md)
 * Supports custom Kubernetes [add-ons](/docs/addons.md)
 * Command line [autocompletion](/docs/cli/kops_completion.md)
+* Manifest Based API [Configuration](/docs/manifests_and_customizing_via_api.md)
 * Community supported!
 
 
@@ -58,6 +59,9 @@ Documentation is in the `/docs` directory, [and the index is here.](docs/README.
 $ brew update && brew install kops
 
 ```
+
+The `kops` binary is also available via our [releases](https://github.com/kubernetes/kops/releases/latest).
+
 ### Linux
 
 Download the [latest release](https://github.com/kubernetes/kops/releases/latest), then:
@@ -67,9 +71,6 @@ $ chmod +x kops-linux-amd64                 # Add execution permissions
 $ mv kops-linux-amd64 /usr/local/bin/kops   # Move the kops to /usr/local/bin
 ```
 
-
-
-
 ## History
 
 See the [releases](https://github.com/kubernetes/kops/releases) for more

README.md

@@ -1,15 +1,13 @@
-Documentation Index
-===================
+# Documentation Index
 
-Quick start
------------
+## Quick start
 * [Getting started on AWS](aws.md)
 * [CLI reference](cli/kops.md)
 
 
-Overview
---------
+## Overview
 
+* [Getting started on AWS](aws.md)
 * [Command-line interface](#commandline-interface)
 * [Inspection](#inspection)
 * [`kops` design documents](#kops-design-documents)
@@ -19,8 +17,7 @@ Overview
 * [Workflows](#workflows)
 
 
-
-### Command-line interface ###
+## Command-line interface
 
 * [CLI argument explanations](arguments.md)
 * [CLI reference](cli/kops.md)
@@ -32,52 +29,53 @@ Overview
     * how to point kubectl to your `kops` cluster
 
 
-### Inspection ###
-
-* [Download `kops` configuration](download_config.md)
-    * methods to download the current generated `kops` configuration
-* [Get AWS subdomain NS records](ns.md)
-
-
-### `kops` design documents ###
-
-* [`kops` cluster boot sequence](boot-sequence.md)
-* [`kops` cluster spec](cluster_spec.md)
-* [`kops` instance groups](instance_groups.md)
-* [`kops` philosophy](philosophy.md)
-* [`kops` state store](state.md)
-
-
-### Networking ###
-
-* [Networking overview](networking.md)
-* [Run `kops` in an existing VPC](run_in_existing_vpc.md)
-* [Supported network topologies](topology.md)
-* [Subdomain setup](creating_subdomain.md)
-
-
-### Operations ###
+## Operations
 
+* API / Manifest Custom Values
+    * [godocs Cluster specific documentation - `spec`](https://godoc.org/k8s.io/kops/pkg/apis/kops#ClusterSpec).
+    * [godocs Instance Group specific documentation - `spec`] (https://godoc.org/k8s.io/kops/pkg/apis/kops#InstanceGroupSpec).
 * [Cluster addon manager](addon_manager.md)
 * [Cluster addons](addons.md)
 * [Cluster configuration management](changing_configuration.md)
+* [`kops` cluster API definitions](cluster_spec.md)
+    * overview of some of the API value to customize a `kops` cluster
 * [Cluster upgrades and migrations](cluster_upgrades_and_migrations.md)
 * [`etcd` volume encryption setup](etcd_volume_encryption.md)
 * [`etcd` backup setup](etcd_backup.md)
 * [GPU setup](gpu.md)
 * [High Availability](high_availability.md)
+* [`kops` instance groups API](instance_groups.md)
+    * overview of some of the API value to customize a `kops` groups of ks8 nodes
 * [InstanceGroup images](images.md)
     * how to use other image for cluster nodes, and information on available/tested images
+* [Using Manifests and Customization via the API](manifests_and_customizing_via_api.md)
+    * how to use YAML to manage clusters.
+    * how to customize cluster via the `kops` API.
 * [`k8s` upgrading](upgrade.md)
 * [`kops` updating](update_kops.md)
 * [`kube-up` to `kops` upgrade](upgrade_from_kubeup.md)
 * [Label management](labels.md)
     * for cluster nodes
 * [Secret management](secrets.md)
-* [Single-master to multi-master update](single-to-multi-master.md)
+* [Moving from a Single Master to Multiple HA Masters](single-to-multi-master.md)
 
 
-### Security ###
+## Networking
+
+* [Networking Overview including CNI](networking.md)
+* [Run `kops` in an existing VPC](run_in_existing_vpc.md)
+* [Supported network topologies](topology.md)
+* [Subdomain setup](creating_subdomain.md)
+
+
+## `kops` design documents
+
+* [`kops` cluster boot sequence](boot-sequence.md)
+* [`kops` philosophy](philosophy.md)
+* [`kops` state store](state.md)
+
+
+## Security
 
 * [Bastion setup](bastion.md)
 * [IAM roles](iam_roles.md)
@@ -87,7 +85,27 @@ Overview
     * overview of secret storage, SSH credentials etc.
 
 
-### Workflows ###
+## Inspection
+
+* [Download `kops` configuration](download_config.md)
+    * methods to download the current generated `kops` configuration
+* [Get AWS subdomain NS records](ns.md)
+
+
+## Development
+
+* [Developing using Docker](development/Docker.md)
+* [Development with vSphere](development/vsphere-dev.md)
+* [Documentation Guidelines](development/documentation.md)
+* [E2E testing with `kops` clusters](development/testing.md)
+* [Example on how to add a feature](development/adding_a_feature.md)
+* [Hack Directory](development/hack.md)
+* [How to update `kops` API](development/api_updates.md)
+* [Low level description on how kops works](how_it_works.md)
+* [Notes on Gossip design](development/gossip.md)
+* [Notes on master instance sizing](development/instancesizes.md)
+* [Our release process](development/release.md)
+* [Releasing with Homebrew](development/homebrew.md)
+* [Rolling Update Diagrams](development/rolling_update.md)
+* [Updating Go Dependencies](development/dependencies.md)
 
-* [E2E testing with `kops` clusters](testing.md)
-* [Getting started on AWS](aws.md)

docs/README.md

@@ -1,4 +1,6 @@
-# Detailed description of arguments
+# Detailed description of a few selected arguments
+
+This list is not the full list, but a few arguments that where choosen.
 
 ## admin-access
 
@@ -24,7 +26,7 @@ See the docs in [cluster_spec.md#adminaccess](cluster_spec.md#adminaccess)
 `dns-zone` controls the Route53 hosted zone in which DNS records will be created.  It can either by the name
 of the hosted zone (`example.com`), or it can be the ID of the hosted zone (`Z1GABCD1ABC2DEF`)
 
-Suppose you're creating a cluster named "dev.kubernetes.example.com`:
+Suppose you are creating a cluster named "dev.kubernetes.example.com`:
 
 * You can specify a `--dns-zone=example.com` (you can have subdomains in a hosted zone)
 * You could also use `--dns-zone=kubernetes.example.com`
@@ -63,9 +65,9 @@ Values:
 
 `out` determines the directory into which kubectl will write the target output.  It defaults to `out/terraform`
 
-# YAML-only Arguments
+# API only Arguments
 
-Certain arguments can only be passed via YAML, eg, `kops edit cluster`. The following documents some of the more interesting or lesser-known options.
+Certain arguments can only be passed via the API, eg, `kops edit cluster`. The following documents some of the more interesting or lesser-known options.
 
 ## kubeletPreferredAddressTypes
 
@@ -79,3 +81,5 @@ kubeAPIServer:
 	- InternalIP
 	- ExternalIP
 ```
+
+More information about using YAML is available [here](manifests_and_customizing_via_api.md).

docs/arguments.md

@@ -1,10 +1,9 @@
 # Description of Keys in `config` and `cluster.spec`
 
-This list is not complete, but aims to document any keys that are less than self-explanatory.
+This list is not complete but aims to document any keys that are less than self-explanatory. Our [godoc](https://godoc.org/k8s.io/kops/pkg/apis/kops) reference provides a more detailed list of API values. [ClusterSpec](https://godoc.org/k8s.io/kops/pkg/apis/kops#ClusterSpec), defined as `kind: Cluster` in YAML, and [InstanceGroup](https://godoc.org/k8s.io/kops/pkg/apis/kops#InstanceGroup), defined as `kind: InstanceGroup` in YAML, are the two top-level API values used to describe a cluster.
 
 ## spec
 
-
 ### api
 
 This object configures how we expose the API:
@@ -159,9 +158,39 @@ Will result in the flag `--feature-gates=ExperimentalCriticalPodAnnotation=true,
 
 ### networkID
 
-On AWS, this is the id of the VPC the cluster is created in. If creating a cluster from scratch, this field doesn't need to be specified at create time; `kops` will create a `VPC` for you.
+On AWS, this is the id of the VPC the cluster is created in. If creating a cluster from scratch, this field does not need to be specified at create time; `kops` will create a `VPC` for you.
 
 ```yaml
 spec:
   networkID: vpc-abcdefg1
 ```
+
+More information about running in an existing VPC is [here](run_in_existing_vpc.md).
+
+### hooks
+
+Hooks allow the execution of a container before the installation of Kubneretes on every node in a cluster.  For intance you can install nvidia drivers for using GPUs.
+
+```
+spec:
+  # many sections removed
+  hooks:
+  - execContainer:
+      image: kopeio/nvidia-bootstrap:1.6
+```
+
+Install Ceph
+
+```
+spec:
+  # many sections removed
+  hooks:
+  - execContainer:
+      command:
+      - sh
+      - -c
+      - chroot /rootfs apt-get update && chroot /rootfs apt-get install -y ceph-common
+      image: busybox
+```
+
+

docs/cluster_spec.md

@@ -1,51 +1,62 @@
 # Download kops config spec file
+
 KOPS operates off of a config spec file that is generated during the create phase.  It is uploaded to the amazon s3 bucket that is passed in during create.
 
 If you download the config spec file on a running cluster that is configured the way you like it, you can just pass that config spec file in to the create command and have kops create the cluster for you , `kops create -f spec_file` in a completely unattended manor.
 
-## How to download the config spec file. ( 2 possible ways )
-
-Let's say you create your cluster with the following configuration options:
+Let us say you create your cluster with the following configuration options:
 
 ```
-declare -x KOPS_STATE_STORE=s3://k8s-us-west
-declare -x CLOUD=aws
-declare -x ZONE="us-west-1a"
-declare -x MASTER_ZONES="us-west-1a"
-declare -x NAME=westtest.c.foo.com
-declare -x K8S_VERSION=1.4.6
-declare -x NETWORKCIDER="10.240.0.0/16"
-declare -x MASTER_SIZE="t2.medium"
-declare -x WORKER_SIZE="t2.large"
+export KOPS_STATE_STORE=s3://k8s-us-west
+export CLOUD=aws
+export ZONE="us-west-1a"
+export MASTER_ZONES="us-west-1a"
+export NAME=k8s.example.com
+export K8S_VERSION=1.6.4
+export NETWORKCIDER="10.240.0.0/16"
+export MASTER_SIZE="m3.large"
+export WORKER_SIZE="m4.large"
 ```
 Next you call the kops command to create the cluster in your terminal:
 
 ```
-  kops create cluster $NAME       \
-    --cloud=$CLOUD                      \
-    --zones="$ZONE"                     \
-    --kubernetes-version=$K8S_VERSION   \
-    --master-zones="$MASTER_ZONES"       \
-    --node-count=3                      \
-    --node-size="$WORKER_SIZE"          \
-    --master-size="$MASTER_SIZE"        \
-    --network-cidr=${NETWORKCIDER}      \
-    --dns-zone=ZVO7KL181S5AP \
-    --ssh-public-key=/Users/foo/.ssh/lab_no_password.pub
+kops create cluster $NAME              \
+   --cloud=$CLOUD                      \
+   --zones="$ZONE"                     \
+   --kubernetes-version=$K8S_VERSION   \
+   --master-zones="$MASTER_ZONES"      \
+   --node-count=3                      \
+   --node-size="$WORKER_SIZE"          \
+   --master-size="$MASTER_SIZE"        \
+   --network-cidr=${NETWORKCIDER}      \
+   --dns-zone=ZVO7KL181S5AP            \
+   --ssh-public-key=$HOME/.ssh/lab_no_password.pub
 ```
 
-### Method One - Kops command
+## kops command
 
-You can simply use the kops command `kops get cluster -o yaml > a_fun_name_you_will_remember.yml`
+You can simply use the kops command `kops get --name $NAME -o yaml > a_fun_name_you_will_remember.yml`
 
-Note: for the above command to work the cluster NAME and the KOPS_STATE_STORE will have to be exported in your environment.
+Note: for the above command to work the cluster NAME and the KOPS_STATE_STORE will have to be exported in your environment.  
 
-### Method Two - Manual method directly from s3
+For more information on how to use and modify the configurations see (here)[manifests_and_customizing_via_api.md].
 
-Your spec file will be located in your s3 bucket, in the location `s3://k8s-us-west/$NAME/config`.  Using the above as an example the config file is located at `s3://k8s-us-west/westtest.c.foo.com/config`
+## Managing instance groups
 
-To download this file via the aws cli you can use the aws s3 copy command.  Using the above cluster as an example the command would be
-
-`aws s3 cp $KOPS_STATE_STORE/$NAME/config ~/a_fun_name_you_will_remember.yml`
+You can also manage instance groups in seperate YAML files as well.  The command `kops get --name $NAME -o yaml > $NAME.yml` exports the entire clsuter.  An option is to have a YAML file for the cluster, and individual YAML files for the instance groups.  This allows you to do stuff like:
 
+```shell
+if ! kops get cluster --name "$NAME"; then
+    kops create -f "kops/$CLUSTER/$REGION.yaml"
+else
+ kops replace -f "kops/$CLUSTER/$REGION.yaml"
+fi
 
+for ig in kops/$CLUSTER/instancegroup/*; do
+  if ! kops get ig --name "$NAME" "$(basename "$ig")"; then
+    kops create -f "$ig"
+  else
+    kops replace -f "$ig"
+  fi
+done
+```