Generate apis types from existed crd definition yaml

[ArenaSu]

What do you want to happen?

How to generate apis types from existed crd definition yaml ? For example i have a crd definition yaml, and i want to generate golang apis types, is there a way to do it ?

The Scenario is below:
My project (generated by kubebuilder) import openfunction v0.7.0 which import keda(github.com/kedacore/keda/v2) v2.4.0 .Meanwhile my project import another inner project which import keda(github.com/kedacore/keda/v2) v2.8.1. Then it conflicts and error occurred when exec go mod tidy. I want to generate keda v2.4.0 api types from v2.4.0 crds and generate keda v2.8.1 api types from v2.8.1 crds. Then move the api types to different directories and import them inner.

And i know kubernetes-client Java have a action named "CRD Java Model Generate", it can generate apis types from crd definition yaml.

Extra Labels

No response

[camilamacedo86]

Hi @ArenaSu,

It should be possible. Mainly with the tool, you will need to call create API and generate only the controller. However, we need to do some manual changes, for now, see that:

• we have an open issue to better address that: Handle external API types explicitly in create api #1999
• you can also check that we have an old doc: https://github.com/kubernetes-sigs/kubebuilder/blob/master/docs/using_an_external_type.md
• Also, as guidance would be recommended you check that the API is added to the cluster in the main.go. Note that you will need to add the schema for the API that is not owned by the project, see: https://developers.redhat.com/blog/2020/02/04/how-to-use-third-party-apis-in-operator-sdk-projects

Therefore, would be great to do (IMHO):

Create a doc in the reference (https://book.kubebuilder.io/reference/reference.html) such as Working with external/third-party apis

• Add on this section: the recommendations to check out if the API is applied in the main.go and how to add the schema: https://developers.redhat.com/blog/2020/02/04/how-to-use-third-party-apis-in-operator-sdk-projects
• Add the info from the old doc https://github.com/kubernetes-sigs/kubebuilder/blob/master/docs/using_an_external_type.md (updated)
• Ensure that we doc all steps to do it with the current scaffold and create a note pointing out the open issue Handle external API types explicitly in create api #1999

Also, help to address the issue #1999 is welcome

Would you like to contribute with?

[jmrodri]

/label help-wanted

[k8s-ci-robot]

@jmrodri: The label(s) /label help-wanted cannot be applied. These labels are supported: api-review, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, team/katacoda, refactor. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

In response to this:

/label help-wanted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jmrodri]

/cc @varshaprasad96 @everettraven

[jmrodri]

/cc @jmrodri

[varshaprasad96]

Adding few pointers here, in case it helps:
Just to understand right, the scenario looks like this:

• Module A imports [email protected]
• Module B imports [email protected]
• You require both Module A and Module B in your project.
  Doing so, go mod tidy errors, since I assume both are incompatible and breaking. This seems to be a Golang issue.

This dependency cycle in a project is highly not suggested since go cannot differentiate between the two. In order to use 2 versions of the same package in a project, you would have to use 2 different major versions, (ie) you can have something/pkg and something/v2/pkg. More details on the same are also here. The other fragile workaround (I would not suggest this) is to fork both the versions locally, and use replace directives in go.mod.

Meanwhile, coming to the best practices in Kubernetes world - it would be helpful to understand your use case. Having multiple GVKs in a same project can definitely be possible, though only one version is eventually stored in etcd server on cluster. In your case, if v2.8.1 and v2.4.0 are incompatible (I assume), which means both of them belong to different versions of custom resources (say CR/v1 or maybe CR/v2). This indirectly also means that though you may need to generate _types individually for both, they in turn can be translated to have different major versions. Unfortunately, here we have a thin line of conversion between semantic and alphanumeric versioning which needs to be sorted out based on the use case and nature of APIs.

[camilamacedo86]

Oh thank you @varshaprasad96,
No I got that I misunderstood the question.

@ArenaSu maybe nit trying to create controllers for Kinds that are created by other projects.
actually is looking to import the definitions from another project and then re-created the Kind/Api.

I'd suggest you do not do that. And instead to do the steps described on my previous comment: #3082 (comment)

[ArenaSu]

Adding few pointers here, in case it helps: Just to understand right, the scenario looks like this:

• Module A imports [email protected]
• Module B imports [email protected]
• You require both Module A and Module B in your project.
  Doing so, go mod tidy errors, since I assume both are incompatible and breaking. This seems to be a Golang issue.

This dependency cycle in a project is highly not suggested since go cannot differentiate between the two. In order to use 2 versions of the same package in a project, you would have to use 2 different major versions, (ie) you can have something/pkg and something/v2/pkg. More details on the same are also here. The other fragile workaround (I would not suggest this) is to fork both the versions locally, and use replace directives in go.mod.

Meanwhile, coming to the best practices in Kubernetes world - it would be helpful to understand your use case. Having multiple GVKs in a same project can definitely be possible, though only one version is eventually stored in etcd server on cluster. In your case, if v2.8.1 and v2.4.0 are incompatible (I assume), which means both of them belong to different versions of custom resources (say CR/v1 or maybe CR/v2). This indirectly also means that though you may need to generate _types individually for both, they in turn can be translated to have different major versions. Unfortunately, here we have a thin line of conversion between semantic and alphanumeric versioning which needs to be sorted out based on the use case and nature of APIs.

The Scenario is below:

1. k8s cluster A installed [email protected]
2. k8s cluster B installed [email protected]
3. two apps (crd controller and apiserver in cluster C only used to manage multi cluster resources) outside of the two clusters need to create and get two version keda crd resources

@camilamacedo86 @varshaprasad96 It's a very common scenario i think. What i want to know is how to generate apis types from crd definition yamls and i think it can solve the problem indeed (as CRD Java Model Generate action do). And i'm sorry that i do not find the method from your comments. Do i miss the point ?

[varshaprasad96]

@ArenaSu I'm not personally aware of any tool that generates go code from CRDs. The usual option is generally unmarshalling the yaml and generating the struct out of it (https://pkg.go.dev/sigs.k8s.io/[email protected]#Unmarshal) to handle the objects further.

[everettraven]

@ArenaSu since you are using CRDs from a Go package that already exists with type definitions. While you may not be able to import both v2.4.0 and v2.8.1, you can create you own type definitions by copying their implementation and then renaming them as you see fit for use in your project. Personally I think this would be easier than trying to find a tool to do that from a CRD YAML file (which I am not aware of one that exists for Go).

I am still curious if there is a better way to handle a scenario like this rather than having to re-implement the type definitions. I would assume that there aren't incompatible changes between the CRDs since they are both the same major version, but there may be new fields added to them from v2.4.0 --> v2.8.1 (This is also assuming they are versioning their CRDs in this way. You would likely need to verify that there are no incompatible API changes).

If your operator can already tell which cluster requires which version of the CRD to use you could have certain logic to populate the fields that only exist in the corresponding version of the CRD. This way you would only need to import and use the definitions from keda v2.8.1 and use as is for cluster B but when you need to do something on cluster A you only use the fields that exist in a keda v2.4.0 CRD (you may end up having to use an unstructured.Unstructured object to represent creation of a v2.4.0 CR but I'm not entirely sure here).

[ArenaSu]

@everettraven @camilamacedo86 I get it, thanks a lot. I do use runtime.RawExtension and unstructured.Unstructured to make multi-version crds compatible and handle them for different clusters respectively over not only keda but also many other crds. And i will copy multi-version crd definition code to my project to solve the problem. Um it's a "strenuous" work.

[ArenaSu]

@everettraven @camilamacedo86 Hi, i am trying to develop a tool to generate api types code from CRD yaml.But i have no experience before.And i am reading the controller-tools source code to find the way how to create a standard generator. I would be very grateful if you give me some ideas or suggestions to generate code from apiextensions CustomResourceDefinition as you are experts in this area. Maybe some useful libs or the specialized way to do it?

[varshaprasad96]

@ArenaSu I've had the experience of developing custom generators using controller-tool as a library and hence got the urge to answer the question on that :) So adding some inputs here, in case it helps!

Note:
If you're looking to parse yaml and generate an api type, I doubt controller-tools is the right tool. I'm still going to go ahead and explain the utilities of controller-tools in case you are looking to decode the library and would like to use any helpers for your use case.

The major utilities which c-t provides us with is the ability to define markers in the format we want, parse them with any custom logic and scaffold out relevant bits of code (the output maybe a yaml, a go file or anything).

Step 1:
Declaring markers and parsing them:

• The first step is to declare the format of markers and create a "definition" out of them. The commonly used helper for this is marker.MakeDefinition (https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/markers#MakeDefinition).
• Markers can be defined to be on a type, on a package or on a field. (like kubebuilder object/validation/webhook markers). These can be specified through Describes<*> field which is added when you define a marker (https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/markers#DescribesPackage)
• The marker format can accept arguments. The fields can be defined as a struct and passed on to the definition, similar to https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/crd/markers#PrintColumn.
  An example of this is here: https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/crd/markers#CRDMarkers.

Step 2:
Parse the marker:
This is done using gengo for which we have wrappers in controller-tools. An example for the same is here: https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/crd#Generator.Generate. We have several helpers around the parsing logic: https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/markers#MarkerValues.Get

Step 3:
Generate output:
The output can be of any form. An example of writing yaml is here: https://pkg.go.dev/sigs.k8s.io/controller-tools/pkg/genall#GenerationContext.WriteYAML, and one for writing go files is: https://github.com/kubernetes-sigs/controller-tools/blob/v0.11.1/pkg/deepcopy/gen.go#L290. The GenerationContext wraps around all these functionalities. It just boils down to having bytes of data and utilizing other go tool to write the output in the format we want.

Also not to complicate more, but just suggesting on one of the methods on how to parse CRD/yaml effectively (and dynamically - when we don't know the field names) - is to convert the yaml into AST (https://pkg.go.dev/go/ast), decode the same and create api types. This is more involved process.

[ArenaSu]

@ArenaSu I've had the experience of developing custom generators using controller-tool as a library and hence got the urge to answer the question on that :) So adding some inputs here, in case it helps!
...

Thank you very much.It do help a lot.I will dig into the method what you list and try to develop the tool.