[Question] How to decode a RawExtension ?

[eddycharly]

I have a hard time understanding how decoding a RawExtension into an Object.

Given the comment below:

apimachinery/pkg/runtime/types.go

Lines 49 to 102 in 829ed19

	// RawExtension is used to hold extensions in external versions.
	//
	// To use this, make a field which has RawExtension as its type in your external, versioned
	// struct, and Object in your internal struct. You also need to register your
	// various plugin types.
	//
	// // Internal package:
	// type MyAPIObject struct {
	// runtime.TypeMeta `json:",inline"`
	// MyPlugin runtime.Object `json:"myPlugin"`
	// }
	// type PluginA struct {
	// AOption string `json:"aOption"`
	// }
	//
	// // External package:
	// type MyAPIObject struct {
	// runtime.TypeMeta `json:",inline"`
	// MyPlugin runtime.RawExtension `json:"myPlugin"`
	// }
	// type PluginA struct {
	// AOption string `json:"aOption"`
	// }
	//
	// // On the wire, the JSON will look something like this:
	// {
	// "kind":"MyAPIObject",
	// "apiVersion":"v1",
	// "myPlugin": {
	// "kind":"PluginA",
	// "aOption":"foo",
	// },
	// }
	//
	// So what happens? Decode first uses json or yaml to unmarshal the serialized data into
	// your external MyAPIObject. That causes the raw JSON to be stored, but not unpacked.
	// The next step is to copy (using pkg/conversion) into the internal struct. The runtime
	// package's DefaultScheme has conversion functions installed which will unpack the
	// JSON stored in RawExtension, turning it into the correct object type, and storing it
	// in the Object. (TODO: In the case where the object is of an unknown type, a
	// runtime.Unknown object will be created and stored.)
	//
	// +k8s:deepcopy-gen=true
	// +protobuf=true
	// +k8s:openapi-gen=true
	type RawExtension struct {
	// Raw is the underlying serialization of this object.
	//
	// TODO: Determine how to detect ContentType and ContentEncoding of 'Raw' data.
	Raw []byte `json:"-" protobuf:"bytes,1,opt,name=raw"`
	// Object can hold a representation of this extension - useful for working with versioned
	// structs.
	Object Object `json:"-"`
	}

Especially this part:

// The next step is to copy (using pkg/conversion) into the internal struct. The runtime
// package's DefaultScheme has conversion functions installed which will unpack the
// JSON stored in RawExtension, turning it into the correct object type, and storing it
// in the Object.

I fail to find anything useful in pkg/conversion that could help me in converting the RawExtension into an Object. All i can think of is something like this:

codecs := serializer.NewCodecFactory(scheme)
deserializer := codecs.UniversalDeserializer()
obj, gvk, err := deserializer.Decode(rawExtension.Raw, nil, nil)

Has someone any advice on the correct approach to use here ?

[dionysius]

I've stumbled on this issue as well and tried to look a bit around. I found out that this project is using RawExtension with the help of this issue (while the issue itself is not directly related to RawExtension itself, but was helpful to dig trough the source code)

• Review the definition of the type using RawExtension: in go, in proto
• Use of +kubebuilder:pruning:PreserveUnknownFields is important, respectively your CRD needs the option x-kubernetes-preserve-unknown-fields: true if you use CRD v1 (also for v1alpha1 if you enabled pruning of additional fields). This was my main issue not getting it to work
• They write the raw json directly to *RawExtension.Raw: https://github.com/searchlight/grafana-operator/blob/7e1d1eecd3c583ff9800f5726b31da64f0641a5d/test/e2e/dashboard_test.go#L122.
• That means you are free to use the .Raw field for reading and writing. As long as it is a json encoded blob as []byte in the end (! - yaml didn't work for me), you can do whatever you want outside of it
• Also neat: kubectl is still able to display that field in yaml with the correct indention and that resource is also editable in that form.

I just can't say if this is the right official way, but I hope I could help you.

[hicaistar]

I've found that kube-scheduler use RawExtension for PluginConfig: https://github.com/kubernetes/kube-scheduler/blob/master/config/v1beta1/types.go#L225, and it use DecodeNestedObjects interface which I'm not familiar with. When kube-scheduler start, it will read a json file, code is here: https://github.com/kubernetes/kubernetes/blob/master/cmd/kube-scheduler/app/options/configfile.go#L40 ， it also use
serializer.NewCodecFactory(scheme).

[vorishirne]

Kubernetes admission review also uses the type { Raw []byte 'json:"-",Object Object json:"-"` }

Which A.T. golang official doc, will just ignore the Raw and Object fields during unmarshal (beacause of a hyphon in the tag), inspite of populating them. Cant understand why during unmarshal, "Raw" field is populated.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

@sTb1337: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.