Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul kpt concepts #3098

Merged
merged 2 commits into from
May 6, 2022
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 42 additions & 49 deletions site/book/02-concepts/00.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,44 @@
What is kpt?

> kpt is a Git-native, schema-aware, extensible client-side tool for packaging, customizing,
> validating, and applying Kubernetes resources.

That's a pretty terse description, so let's break it down:

- **Git-native**: kpt uses Git to share, consume, and update configuration. As a result,
configuration changes may be reviewed, approved, audited and rolled back using existing Git
workflows.

- **Schema-aware**: Configuration used by kpt must conform to the _Kubernetes Resource Model
(KRM)_ which have a defined schema and semantics. KRM resources — whether core Kubernetes
resources such as `Deployment` or custom resources defined using a `CRD` — have an OpenAPI
specification which is used for schema-aware operations in kpt. For example, when rebasing a
locally modified package to a new version, the merge operation is more than simply performing a
text-based file merge, it takes the schema into account.

In contrast, many configuration tools today interleave data and code, for example by embedding a
templating language in YAML. As configuration becomes complex, it becomes hard to read and
understand. Furthermore, this prevents external tooling from easily consuming such configuration
leading to a closed ecosystem.

- **Extensible**: kpt provides a small core machinery and powerful extension mechanisms. There are
two main extension mechanisms in kpt: a) kpt can gain semantic understanding of arbitrary KRM
resources by consuming OpenAPI documents. kpt core itself does not hardcode any type-specific
information. b) kpt can perform arbitrary operations on resources using kpt functions.

- **Packaging**: A package is a bundle of KRM resources representing a useful unit of functionality.

- **Customizing**: Off-the-shelf packages are rarely deployed without any customization. We know
from experience working with Kubernetes configuration and with internal systems at Google that
there is no one-size-fit-all customization technique. Sometimes the most appropriate customization
is to edit a YAML file directly. Sometimes, you want to automate a one-time customization,
sometimes you want to repeatedly perform a customization. Sometimes you want to use
parameterization, sometimes perform bulk search & replace, sometimes you want to use a
general-purpose programming language. kpt enables all these techniques. In contrast, tools that
provide just one customization technique (e.g. parameterization) cannot address all these
different use cases well and inevitably lead to anti-patterns (e.g. over-parameterization).

- **Validating**: In the era of fast-paced DevOps, we need to reduce the risk of misconfiguration
leading to an outage, security vulnerabilities, or non-compliance. Validation goes hand-in-hand
with customization and kpt functions can be used to automate both mutation and validation of
resources. Example of validator functions include schema-validation, linting, policy enforcement,
and security auditing.

- **Applying**: When it comes to deploying a package to a Kubernetes cluster, kpt complements
`kubectl`. By keeping an inventory of deployed resources, kpt enables resource pruning, aggregated
status and observability, and an improved dry-run experience.

The two fundamental concepts in kpt are packages and functions. Let's define them.
> A package-centric toolchain that enables a WYSIWYG configuration authoring, automation, and delivery experience and simplifies managing Kubernetes platforms and KRM-driven infrastructure at scale by manipulating declarative Configuration as Data, separated from the code that transforms it.

kpt supports management of [Configuration as Data](https://github.com/GoogleContainerTools/kpt/blob/main/docs/design-docs/06-config-as-data.md), which enables WYSIWYG editing and interoperable automation on a declarative data model for Kubernetes and any infrastructure represented in the [Kubernetes Resource Model (KRM)](https://github.com/kubernetes/design-proposals-archive/blob/main/architecture/resource-management.md).

kpt manages KRM resources in bundles called **packages**.

Off-the-shelf packages are rarely deployed without any customization. Like [kustomize](https://kustomize.io), kpt applies transformation **functions**, using the same [KRM function specification](https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md), but optimizes for in-place configuration transformation rather than out-of-place transformation.

Validation goes hand-in-hand with customization and kpt functions can be used to automate both mutation and validation of resources, similar to [Kubernetes admission control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/).

The kpt toolchain includes the following components:

- [**kpt CLI**](https://kpt.dev/reference/cli/): The kpt CLI supports package and function operations, and also
deployment, via either direct apply or GitOps. By keeping an inventory of deployed resources, kpt enables resource pruning,
aggregated status and observability, and an improved preview experience.

- **Function SDKs**: Any general-purpose or domain-specific language can be used to create functions to transform and/or validate
the YAML KRM input/output format, but we provide SDKs to simplify the function authoring process, in
[Go](https://kpt.dev/book/05-developing-functions/02-developing-in-Go),
[Typescript](https://kpt.dev/book/05-developing-functions/03-developing-in-Typescript), and
[Starlark](https://catalog.kpt.dev/starlark/v0.2/), a Python-like embedded language.

- [**Function catalog**](https://catalog.kpt.dev/): A catalog of off-the-shelf, tested functions. kpt makes configuration
easy to create and transform, via reusable functions. Because they are expected to be used for in-place transformation,
the functions need to be idempotent.

- [**Package orchestrator**](https://github.com/GoogleContainerTools/kpt/blob/main/docs/design-docs/07-package-orchestration.md):
The package orchestrator enables the magic behind the unique WYSIWYG experience. It provides a control plane for creating,
modifying, updating, and deleting packages, and evaluating functions on package data. This enables operations on packaged resources
similar to operations directly on the live state through the Kubernetes API.

- [**Config Sync**](https://cloud.google.com/anthos-config-management/docs/config-sync-overview): While the package orchestrator
can be used with any GitOps tool, Config Sync provides a reference GitOps implementation to complete the WYSIWYG management
experience and enable end-to-end development of new features, such as
[OCI-based packages](https://github.com/GoogleContainerTools/kpt/issues/2300). Config Sync is also helping to drive improvements
in upstream Kubernetes. For instance, Config Sync is built on top of [git-sync](https://github.com/kubernetes/git-sync) and
leverages [Kustomize](https://kustomize.io) to automatically render manifests on the fly when needed. It uses the same apply
logic as the kpt CLI.

- **Backage UI plugin**: We've created a proof-of-concept UI to demonstrate the WYSIWYG experience that's possible on top of the
package orchestrator. More scenarios can be supported by implementing form-based editors for additional Kubernetes resource types.