Main

Add structs for ResourceGroup objects (kptdev#2619)

* feat: Add structs for ResourceGroup objects

This commit introduces the required types to move inventory information
out from thee Kptfile and into a separate resourcegroup.yaml resource
file.

Since the ResourceGroup CRD was hard coded previously, and ResourceGroup
objects were always handled as unstructured objects, these type
definitions did not exist prior to this commit.

* refactor: Rename default meta struct for ResourceGroup

propose ownership change to facilitate PR reviews and improving kpt.dev docs (kptdev#2679)

docs: update kpt completion docs (kptdev#2673)

This change makes two key updates to the completion docs:
- Provide instructions to remove an artifact of previous installations
which breaks kpt completion functionality. This issue was reported by a
user of kpt.
- Direct the user to the help command for per-shell instructions to
enable kpt completion. Since the completion functionality is provided by
a third party library (cobra), this ensures the user is provided with
accurate and up to date instructions.

Ensure release license file exists (kptdev#2682)

* ci: ensure existence of lib.zip for releases

ensure existence of the lib.zip file for downstream releases. This is
done by creating a base zip file containing a README and updating it
with mozilla_repos if mozilla_repos is not empty.

* ci: use array for mozilla_repos

Update mozilla_repos to use an array rather than a string with space
delimited elements. This is intended to provide a more straightforward
type for storing a list of elements as well as provide more explicit
word splitting in line with SC2086.

fix: report NotFound status for deleted KCC resources (kptdev#2689)

* fix: report NotFound status for deleted KCC resources

The current custom StatusReader for Config Connector resources will
report Unknown status when a Config Connector resource is not found (aka
deleted). This causes the `kpt live destroy` reconcile loop to run
forever since it expects a NotFound status to end. This commit ensures
that deleted resources report a NotFound status instead.

* refactor: Fix linting issue for unkeyed fields in composite literal

Fix missing colon in design doc (kptdev#2693)

This helps people that copy and paste from the examples!

Main

Save inventory information into resourcegroup.yaml (kptdev#2615)

Fix site sidebar to also show annotation references (kptdev#2734)

effective customizations chapter (kptdev#2659)

merging this as an MVP and will iterate based on additional requests and customer feedback.

docs: Describe how reconcile status is computed for Config Connector resources (kptdev#2739)

Mods: Merge main into porch

Main

feat: enable migrating from Kptfile & CM to resourcegroup inventories (kptdev#2705)

This commit enables `kpt live migrate` to migrate inventory information
from either a ConfigMap or Kptfile to a separate ResourceGroup file.

This functionality is currently behind a feature gate and is not exposed
to the user via any CLI flags. Enabling of this feature to users will be
done later.

make kpt binary optional in test harness (kptdev#2758)

feat: enable STDIN apply and destroy using RG inventory (kptdev#2709)

This commit enables actuation from STDIN using inventory information
that is stored in a ResourceGroup file.

This feature is currently behind a feature gate and will be exposed to
users as a CLI flag in a future commit/PR.

updated the version to 1.0.0-beta.13 (kptdev#2806)

Merge main into porch

Correct Apache License Text (kptdev#2675)

LICENSE file is supposed to be an exact copy of
https://www.apache.org/licenses/LICENSE-2.0.txt.

CODEOWNERS

@@ -1,5 +1,5 @@
 # Docs need to have a single approver to maintain consistency.
-/site @droot @frankfarzan @phanimarupaka @mengqiy
+/site @droot @frankfarzan @phanimarupaka @mengqiy @yuwenma @natasha41575
 # These packages are referenced in docs.
 /package-examples/nginx/ @droot @mortent @phanimarupaka @mengqiy
 /package-examples/wordpress/ @droot @mortent @phanimarupaka @mengqiy

Formula/kpt.rb

@@ -15,8 +15,8 @@
 class Kpt < Formula
   desc "Toolkit to manage,and apply Kubernetes Resource config data files"
   homepage "https://googlecontainertools.github.io/kpt"
-  url "https://github.com/GoogleContainerTools/kpt/archive/v1.0.0-beta.7.tar.gz"
-  sha256 "e31e7ee63006150f730ed8a08e9063a2fb2faf9e617353e3649ef893bdf7c156"
+  url "https://github.com/GoogleContainerTools/kpt/archive/v1.0.0-beta.13.tar.gz"
+  sha256 "d98a95f1de38fc8c7dee861ec249874ea3ddf03f5a28ae7fc4a0364d3578667b"
 
   depends_on "go" => :build
 

LICENSE

@@ -187,7 +187,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright 2019 Google LLC
+   Copyright [yyyy] [name of copyright owner]
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
@@ -199,4 +199,4 @@
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
-   limitations under the License.
+   limitations under the License.

MAINTAINERS

@@ -2,4 +2,6 @@ Mortent Torkildsen <[email protected]>
 Sunil Arora <[email protected]>
 Frank Farzan <[email protected]>
 Phani Teja Marupaka <[email protected]>
-Mengqi Yu <[email protected]>
+Mengqi Yu <[email protected]>
+Yuwen Ma <[email protected]>
+Natasha Sarkar <[email protected]>

docs/design-docs/01-live-invenstory-to-rg.md

@@ -72,7 +72,7 @@ resoucegroup.yaml
 ```yaml
 apiVersion: kpt.dev/v1alpha1
 kind: ResourceGroup
-metadata  
+metadata:
   name: <INVENTORY_NAME>
   namespace: <INVENTORY_NAMESPACE>
   labels:

internal/cmdapply/cmdapply.go

@@ -108,6 +108,7 @@ type Runner struct {
 	pruneTimeout                 time.Duration
 	inventoryPolicyString        string
 	dryRun                       bool
+	rgFile                       string
 	printStatusEvents            bool
 
 	inventoryPolicy inventory.InventoryPolicy
@@ -166,7 +167,7 @@ func (r *Runner) runE(c *cobra.Command, args []string) error {
 		}
 	}
 
-	objs, inv, err := live.Load(r.factory, path, c.InOrStdin())
+	objs, inv, err := live.Load(r.factory, path, r.rgFile, c.InOrStdin())
 	if err != nil {
 		return err
 	}

internal/cmddestroy/cmddestroy.go

@@ -83,6 +83,7 @@ type Runner struct {
 	output                string
 	inventoryPolicyString string
 	dryRun                bool
+	rgFile                string
 	printStatusEvents     bool
 
 	inventoryPolicy inventory.InventoryPolicy
@@ -128,7 +129,7 @@ func (r *Runner) runE(c *cobra.Command, args []string) error {
 		}
 	}
 
-	_, inv, err := live.Load(r.factory, path, c.InOrStdin())
+	_, inv, err := live.Load(r.factory, path, r.rgFile, c.InOrStdin())
 	if err != nil {
 		return err
 	}

internal/cmdliveinit/cmdliveinit.go

@@ -6,8 +6,11 @@ package cmdliveinit
 import (
 	"context"
 	"crypto/sha1"
+	goerrors "errors"
 	"fmt"
+	"io/ioutil"
 	"os"
+	"path/filepath"
 	"strconv"
 	"strings"
 	"time"
@@ -16,16 +19,19 @@ import (
 	"github.com/GoogleContainerTools/kpt/internal/errors"
 	"github.com/GoogleContainerTools/kpt/internal/pkg"
 	"github.com/GoogleContainerTools/kpt/internal/printer"
+	"github.com/GoogleContainerTools/kpt/internal/types"
 	"github.com/GoogleContainerTools/kpt/internal/util/attribution"
 	"github.com/GoogleContainerTools/kpt/internal/util/pathutil"
 	kptfilev1 "github.com/GoogleContainerTools/kpt/pkg/api/kptfile/v1"
+	rgfilev1alpha1 "github.com/GoogleContainerTools/kpt/pkg/api/resourcegroup/v1alpha1"
 	"github.com/GoogleContainerTools/kpt/pkg/kptfile/kptfileutil"
 	"github.com/spf13/cobra"
 	"k8s.io/cli-runtime/pkg/genericclioptions"
 	k8scmdutil "k8s.io/kubectl/pkg/cmd/util"
 	"sigs.k8s.io/cli-utils/pkg/common"
 	"sigs.k8s.io/cli-utils/pkg/config"
 	"sigs.k8s.io/kustomize/kyaml/filesys"
+	"sigs.k8s.io/kustomize/kyaml/yaml"
 )
 
 const defaultInventoryName = "inventory"
@@ -38,6 +44,24 @@ func (i *InvExistsError) Error() string {
 	return "inventory information already set for package"
 }
 
+// InvInRGExistsError defines new error when the inventory
+// values have already been set on the ResourceGroup file and we will warn
+// the user to migrate rather than init. This is part of kpt live STDIN work.
+type InvInRGExistsError struct{}
+
+func (i *InvInRGExistsError) Error() string {
+	return "inventory information already set for package"
+}
+
+// InvInKfExistsError defines new error when the inventory
+// values have already been set on the Kptfile and we will warn
+// the user to migrate rather than init. This is part of kpt live STDIN work.
+type InvInKfExistsError struct{}
+
+func (i *InvInKfExistsError) Error() string {
+	return "inventory information already set within Kptfile for package"
+}
+
 func NewRunner(ctx context.Context, factory k8scmdutil.Factory,
 	ioStreams genericclioptions.IOStreams) *Runner {
 	r := &Runner{
@@ -76,6 +100,7 @@ type Runner struct {
 	Force       bool   // Set inventory values even if already set in Kptfile
 	Name        string // Inventory object name
 	namespace   string // Inventory object namespace
+	RGFile      string // resourcegroup object filepath
 	InventoryID string // Inventory object unique identifier label
 	Quiet       bool   // Output message during initialization
 }
@@ -112,6 +137,7 @@ func (r *Runner) runE(_ *cobra.Command, args []string) error {
 		Quiet:       r.Quiet,
 		Name:        r.Name,
 		InventoryID: r.InventoryID,
+		RGFileName:  r.RGFile,
 		Force:       r.Force,
 	}).Run(r.ctx)
 	if err != nil {
@@ -129,12 +155,22 @@ type ConfigureInventoryInfo struct {
 
 	Name        string
 	InventoryID string
+	RGFileName  string
 
 	Force bool
 }
 
 // Run updates the inventory info in the package given by the Path.
 func (c *ConfigureInventoryInfo) Run(ctx context.Context) error {
+	// Use ResourceGroup file for inventory logic if the resourcegroup file
+	// is set directly. For this feature gate, the resourcegroup must be directly set
+	// through our tests since we are not exposing this through the command surface as a
+	// flag, currently. When we promote this, the resourcegroup filename can be empty and
+	// the default filename value will be inferred/used.
+	if c.RGFileName != "" {
+		return c.runLiveInitWithRGFile(ctx)
+	}
+
 	const op errors.Op = "cmdliveinit.Run"
 	pr := printer.FromContextOrDie(ctx)
 
@@ -189,6 +225,119 @@ func (c *ConfigureInventoryInfo) Run(ctx context.Context) error {
 	return nil
 }
 
+// func runLiveInitWithRGFile is a modified version of ConfigureInventoryInfo.Run that stores the
+// package inventory information in a separate resourcegroup file. The logic for this is branched into
+// a separate function to enable feature gating.
+func (c *ConfigureInventoryInfo) runLiveInitWithRGFile(ctx context.Context) error {
+	const op errors.Op = "cmdliveinit.runLiveInitWithRGFile"
+	pr := printer.FromContextOrDie(ctx)
+
+	namespace, err := config.FindNamespace(c.Factory.ToRawKubeConfigLoader(), c.Pkg.UniquePath.String())
+	if err != nil {
+		return errors.E(op, c.Pkg.UniquePath, err)
+	}
+	namespace = strings.TrimSpace(namespace)
+	if !c.Quiet {
+		pr.Printf("initializing ResourceGroup inventory info (namespace: %s)...", namespace)
+	}
+
+	// Autogenerate the name if it is not provided through the flag.
+	if c.Name == "" {
+		randomSuffix := common.RandomStr()
+		c.Name = fmt.Sprintf("%s-%s", defaultInventoryName, randomSuffix)
+	}
+
+	// Finally, create a ResourceGroup containing the inventory information.
+	err = createRGFile(c.Pkg, &kptfilev1.Inventory{
+		Namespace:   namespace,
+		Name:        c.Name,
+		InventoryID: c.InventoryID,
+	}, c.RGFileName, c.Force)
+	if !c.Quiet {
+		if err == nil {
+			pr.Printf("success\n")
+		} else {
+			pr.Printf("failed\n")
+		}
+	}
+	if err != nil {
+		return errors.E(op, c.Pkg.UniquePath, err)
+	}
+	// add metrics annotation to package resources to track the usage as the resources
+	// will be applied using kpt live group
+	at := attribution.Attributor{PackagePaths: []string{c.Pkg.UniquePath.String()}, CmdGroup: "live"}
+	at.Process()
+	return nil
+}
+
+// createRGFile fills in the inventory object values into the resourcegroup object and writes to file storage.
+func createRGFile(p *pkg.Pkg, inv *kptfilev1.Inventory, filename string, force bool) error {
+	const op errors.Op = "cmdliveinit.createRGFile"
+	// Read the resourcegroup object io io.dir
+	rg, err := p.ReadRGFile(filename)
+	if err != nil && !goerrors.Is(err, os.ErrNotExist) {
+		return errors.E(op, p.UniquePath, err)
+	}
+
+	// Read the Kptfile to ensure that inventory information is not in Kptfile either.
+	kf, err := p.Kptfile()
+	if err != nil {
+		return errors.E(op, p.UniquePath, err)
+	}
+	// Validate the inventory values don't exist in Kptfile.
+	isEmpty := kptfileInventoryEmpty(kf.Inventory)
+	if !isEmpty && !force {
+		return errors.E(op, p.UniquePath, &InvInKfExistsError{})
+	}
+	// Set the Kptfile inventory to be nil if we force write to resourcegroup instead.
+	kf.Inventory = nil
+
+	// Validate the inventory values don't already exist in Resourcegroup.
+	if rg != nil && !force {
+		return errors.E(op, p.UniquePath, &InvExistsError{})
+	}
+	// Initialize new resourcegroup object, as rg should have been nil.
+	rg = &rgfilev1alpha1.ResourceGroup{ResourceMeta: rgfilev1alpha1.DefaultMeta}
+	// // Finally, set the inventory parameters in the ResourceGroup object and write it.
+	rg.Name = inv.Name
+	rg.Namespace = inv.Namespace
+	if inv.InventoryID != "" {
+		rg.Labels = map[string]string{rgfilev1alpha1.RGInventoryIDLabel: inv.InventoryID}
+	}
+	if err := writeRGFile(p.UniquePath.String(), rg, filename); err != nil {
+		return errors.E(op, p.UniquePath, err)
+	}
+
+	// Rewrite Kptfile without inventory existing Kptfile contains inventory info. This
+	// is required when a user appends the force flag.
+	if !isEmpty {
+		if err := kptfileutil.WriteFile(p.UniquePath.String(), kf); err != nil {
+			return errors.E(op, p.UniquePath, err)
+		}
+	}
+
+	return nil
+}
+
+// writeRGFile writes a ResourceGroup inventory to local disk.
+func writeRGFile(dir string, rg *rgfilev1alpha1.ResourceGroup, filename string) error {
+	const op errors.Op = "cmdliveinit.writeRGFile"
+	b, err := yaml.MarshalWithOptions(rg, &yaml.EncoderOptions{SeqIndent: yaml.WideSequenceStyle})
+	if err != nil {
+		return err
+	}
+	if _, err := os.Stat(filepath.Join(dir, filename)); err != nil && !goerrors.Is(err, os.ErrNotExist) {
+		return errors.E(op, errors.IO, types.UniquePath(dir), err)
+	}
+
+	// fyi: perm is ignored if the file already exists
+	err = ioutil.WriteFile(filepath.Join(dir, filename), b, 0600)
+	if err != nil {
+		return errors.E(op, errors.IO, types.UniquePath(dir), err)
+	}
+	return nil
+}
+
 // Run fills in the inventory object values into the Kptfile.
 func updateKptfile(p *pkg.Pkg, inv *kptfilev1.Inventory, force bool) error {
 	const op errors.Op = "cmdliveinit.updateKptfile"