Sealer Application definition #1658

justadogistaken · 2022-08-20T08:12:03Z

Design

Overview

Implementation

KubefileParser

kubefileParser has the three major attributions appRootPathFunc, appConfigRootPathFunc, imageEngine.
appRootPathFunc and appConfigRootPathFunc will return the path relative to the sealer rootfs. Currently they will be $ROOTFS/application/apps/[helm|kube]/$appname/ and $ROOTFS/application/configs/[helm|kube]/$appname/ respectively.

As for imageEngine, we need this to pull the FROM image(name it fromImage). Because there are some definitions of application stored in the fromImage, we need to merge the definitions of application from fromImage and current definitions of application (the merge strategy is overwrite the existing application, and reserve the other applications).

We implement the Kubefile based on Dockerfile.
There are some special operations:

Remote files: APP https://xxx AS [helm|kube]:xxx. It will be ADD https://xxx application/apps/[helm|kube]/$appname/
Local files: APP xxx AS [helm|kube]:xxx. It will be COPY xxx application/apps/[helm|kube]/$appname/
LAUNCH will not generate any instructions in Dockerfile.

APP could specify multiple sources, so if there are two types of sources for APP, it will generate two instructions(COPY and ADD)

TODOs:

Extract the from image for getting the image extension.
Integrate this feature to support apply.
Image render at building.
Env/Config.

Note

I put configurations of a app into a application/configs/[kube|helm]/[appname] and other app artifact into application/apps[kube|helm]/[appname].

I didn't complete the env or config for applications in the work. Currently the helm -f env.yaml will be rendered. I will have a detailed design for env/config. In that work, it will distinguish app-env, scripts-env and so on...

Additional

should we provide a application cmd, to support checking info of application.

Example

There is a Kubefile:

FROM busybox as base
APP mysql https://mysql.yaml
# There are two ways to start APP. `LAUNCH` and `CMDS`. Only one of them could be specified in Kubefile.
# LAUNCH mysql
# CMDS ["kubectl apply -f application/apps/mysql/"]

kubefileParser will download https://mysql.yaml into .sealer-xxx directory within build context.
kubefileParser will generate dockerfile and application&launch cmds
dockerfile:

FROM busybox as base
copy .sealer-xxx/ application/apps/mysql/

application: {configurations, name, type}
launch cmds: [
kubectl apply -f mysql/,
]

dockerfile will be the input for imageEngine.Build. application&launch cmds will be stored into annotation of image.

codecov-commenter · 2022-08-22T12:03:35Z

Codecov Report

Base: 18.37% // Head: 19.89% // Increases project coverage by +1.52% 🎉

Coverage data is based on head (51913ef) compared to base (7b1bb6b).
Patch coverage: 27.88% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1658      +/-   ##
==========================================
+ Coverage   18.37%   19.89%   +1.52%     
==========================================
  Files          66       69       +3     
  Lines        5372     6478    +1106     
==========================================
+ Hits          987     1289     +302     
- Misses       4258     5012     +754     
- Partials      127      177      +50

Impacted Files	Coverage Δ
build/buildimage/utils.go	`0.00% <0.00%> (ø)`
build/kubefile/parser/file_fetcher.go	`0.00% <0.00%> (ø)`
build/layerutils/charts/charts.go	`0.00% <ø> (ø)`
cmd/sealer/cmd/utils/cluster.go	`0.00% <0.00%> (ø)`
pkg/cluster-runtime/installer.go	`0.00% <0.00%> (ø)`
pkg/cluster-runtime/utils.go	`0.00% <0.00%> (ø)`
pkg/clusterfile/util.go	`0.00% <ø> (ø)`
pkg/config/config.go	`29.21% <0.00%> (-1.74%)`	⬇️
pkg/runtime/utils.go	`12.50% <0.00%> (ø)`
utils/decode.go	`0.00% <ø> (ø)`
... and 37 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

allencloud · 2022-08-31T16:24:24Z

build/kubefile/parser/kubefile.go

+func (kp *KubefileParser) ParseKubefile(rwc io.Reader) (*KubefileResult, error) {
+	result, err := parse(rwc)
+	if err != nil {
+		return nil, fmt.Errorf("failed to parse dockerfile: %v", err)


s/dockerfile/Kubefile

allencloud · 2022-08-31T16:30:50Z

I still wish to see more illustration in the pull request description. Actually this is a quite large pr, more design info could help review a lot. @justadogistaken

justadogistaken · 2022-09-05T06:26:36Z

build/buildimage/differ.go

@@ -84,6 +86,112 @@ func NewRegistryDiffer(platform v1.Platform) Differ {
 	}
 }

+func parseApplicationImages(srcPath string) ([]string, error) {
+	// TODO this should be global
+	applicationPath := filepath.Join(srcPath, "application/apps")


Actually, we need to have a rootfs pkg or manager to provide the application path relative to rootfs.(Make it TODO?)

In runtime and it sub module, It provide by private func such as: getRootfs or something. It duplicate with other module in runtime. I think have a public pkg to manage path is a good suggestion!

allencloud · 2022-09-08T06:14:24Z

Thanks for your providing more details in pull request description. While, I think we should pay more energy on APPLICATION related primitives definition. And we already see the detailed implementation of your way. Design comes first, and then the implementation.

Back to your design in the picture, I feel a little bit more complicated than Dockerfile's primitives. Taking APP mysql.yml http://sealer/mysql/config.yml AS kube:mysql as an example, I think APP mysql http://sealer/mysql/config.yml is sufficient. And the APP definition grammar is:

format APP APP_NAME [scheme]LOCATION;
APP_NAME is the user-defined application name;
scheme has a enum type of [file://, http://, https://], and scheme's default value is file:// which means the file location of building context.

Wish to get more feedback. @justadogistaken

justadogistaken · 2022-09-08T06:43:25Z

Here are my considerations for current design:

Why declare the app type: first, I'm both ok with the order APP APP_NAME [scheme]LOCATION or APP [scheme]LOCATION APP_NAME. But I think we'd better keep the type of app, otherwise we should check the structure of source directory(values.yaml && chart && template to take it as "helm"). So I'rather make the format like APP APP_TYPE:APP_NAME [scheme]LOCATION.
And for the configurations, I provide the options. APP --files=[schema]LOCATION --set=key=value APP_TYPE:APP_NAME [scheme]LOCATION

allencloud · 2022-09-09T02:44:57Z

I'rather make the format like APP APP_TYPE:APP_NAME [scheme]LOCATION

I could understand your concern. While I insist on simplify as much as possible for the user-facing interface. With my principle, I think it is a mind-consuming way to add flag thing in APP command. @justadogistaken

justadogistaken · 2022-09-09T02:57:02Z

@allencloud I'm just afraid that this will need us to do application type detection, which may have potential problems. Currently our cases are not complicated. But I can't think of any issus for now.

We can remove the flag in the first version. When the use cases become more complicated, we may need to propose some limitations.

justadogistaken · 2022-09-28T15:39:08Z

@sealerio/sealer-maintainers
I add an option for building image.
sealer build --type kube-installer | app-installer.

kube-installer:

It contains the the cluster runtime, which means it is able to install k8s and apps.

app-installer:

It does not contain the cluster runtime, it can install apps only. (Maybe it contains the cluster runtime, but it will not install k8s)

The type attribute will be inherited.
If image A is typeC,
Then image B is built from A,
Eventually, image B is typeC too.

kakaZhou719 · 2022-09-29T02:15:44Z

@sealerio/sealer-maintainers I add an option for building image. sealer build --type kube-installer | app-installer.

kube-installer:

It contains the the cluster runtime, which means it is able to install k8s and apps.

app-installer:

It does not contain the cluster runtime, it can install apps only. (Maybe it contains the cluster runtime, but it will not install k8s)

The type attribute will be inherited. If image A is typeC, Then image B is built from A, Eventually, image B is typeC too.

nice，It's clear to distinguish the different image types, i have a little confusion about this:

how about sealer build --image-type app since we take kubeinstaller as default?
if this attribute is inheritable, what will happens if from app to build kubeinstaller and from app to build app?

justadogistaken · 2022-09-29T03:30:49Z

@kakaZhou719
Your comments are valuable.
For the second question, how about let's forget about "inheritable" this time? Taking the user input as image type?

allencloud · 2022-09-29T06:29:13Z

build/kubefile/command/command.go

+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package command


I am afraid that in the future we could replace command with instruction. WDYT?

allencloud · 2022-09-29T06:54:34Z

build/kubefile/parser/utils.go

+)
+
+const (
+	schemaLocal = "local://"


scheme, rather than schema.
https://datacadamia.com/web/resource/scheme

allencloud · 2022-09-29T07:03:09Z

build/kubefile/parser/utils.go

+	}
+
+	chartInTargetsRoot := 0
+	oneOfChatsArtifact := func(str string) {


s/Chats/Charts/ ?

allencloud · 2022-09-29T08:47:18Z

build/kubefile/parser/flags.go

+	if len(strs) < 2 {
+		return listFlag{}, errors.New("flags should be like --flag=[value] or --flag=value")
+	}
+	key, values := strs[0], strs[1]


When setting flags, I think there will be some possibility that a char = exists in the item value. If that, there will be strs[2] and so on. Then the code here seems to be incorrect.

How about adding an unit test for this function?

I added the ut.
And I think there is no need to cover all cases.
If there is = exists in value, let it be. It will be error in the next process.

justadogistaken · 2022-10-04T06:46:02Z

@sealerio/sealer-maintainers PTAL.

allencloud · 2022-10-04T12:44:44Z

docs/design/kubefile_alpha.md

+There can be only one `LAUNCH` or `CMDS` instruction in a `Kubefile`.
+
+### CMDS


How about just naming as CMD instead of CMDS? Here is my reasons:

Telling users that Kubefile only allows users to input one command. One command could be quite strict for users to understand;

Command format seems to be shell scripts. If end users wish to let command play several roles, users could use && to combine commands;

It is explicit that what is the running/exit status of command. If there are serveral commands, I think it becomes harder to explain what is the status of commands running.

In addition, is the CMD only allowed to execute towards k8s/k3s/k0s APIServer? Or CMD is allowed to run towards Linux Kernel, which seems weird? After all we are cluster application oriented.

I'm ok with CMD or CMDS.
I think it's fine to define several cmds in CMD like CMD ["kubectl ...", "kubectl ..."]. And of course user can do like CMD ["kubectl xxx && kubectl xxxxx"].

CMD will exec over master0 machine, so it can be executed towards apiserver and Linux kernel both. IMO, CMD is just an reserved instruction, which is not recommended.

starnop · 2022-10-09T02:53:49Z

build/kubefile/parser/file_fetcher.go

+	if err != nil {
+		return "", errors.Wrapf(err, "error creating file to target %s for %s", target, src)
+	}
+	defer func() {


defer f.Close()

Our ci check will give a warning here, which make it fail to pass ci.

Signed-off-by: david.bao <[email protected]>

Signed-off-by: David Bao <[email protected]>

starnop · 2022-10-09T06:41:07Z

LGTM

github-actions bot added area/doc ImageBuilding related to all staff with image building test labels Aug 20, 2022

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch 2 times, most recently from 64afa51 to c3d0026 Compare August 22, 2022 11:57

justadogistaken changed the title ~~[wip]application-def and kubefile translator.~~ [wip]sealer Aug 23, 2022

justadogistaken changed the title ~~[wip]sealer~~ [wip]Sealer Aug 23, 2022

justadogistaken changed the title ~~[wip]Sealer~~ [wip]Sealer Application definition Aug 23, 2022

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch from b8ba2e3 to 578eb2b Compare August 31, 2022 09:43

allencloud reviewed Aug 31, 2022

View reviewed changes

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch from 578eb2b to 3a63902 Compare September 1, 2022 07:25

justadogistaken changed the title ~~[wip]Sealer Application definition~~ Sealer Application definition Sep 1, 2022

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch 4 times, most recently from ce0d0a0 to d74a47b Compare September 5, 2022 06:23

justadogistaken commented Sep 5, 2022

View reviewed changes

justadogistaken requested review from kakaZhou719, bxy4543, starComingup, Stevent-fei and VinceCui September 5, 2022 07:01

justadogistaken mentioned this pull request Sep 8, 2022

Optimize: Dynamic variable render #1693

Open

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch from d74a47b to 1ac4567 Compare September 13, 2022 03:07

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch 2 times, most recently from b7b07c3 to 761aa01 Compare September 28, 2022 15:33

allencloud reviewed Sep 29, 2022

View reviewed changes

build/kubefile/parser/utils.go Outdated

}

chartInTargetsRoot := 0

oneOfChatsArtifact := func(str string) {

Copy link

Member

allencloud Sep 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Chats/Charts/ ?

allencloud reviewed Sep 29, 2022

View reviewed changes

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch 4 times, most recently from 7f07cfa to 35cf7bd Compare September 30, 2022 04:46

justadogistaken requested review from allencloud, jsparter and kakaZhou719 and removed request for jsparter October 2, 2022 06:34

allencloud reviewed Oct 4, 2022

View reviewed changes

starnop reviewed Oct 9, 2022

View reviewed changes

justadogistaken added 2 commits October 9, 2022 11:16

add kubefile translator & application definition

c4db178

Signed-off-by: david.bao <[email protected]>

taking user input as image type

b4d33de

Signed-off-by: David Bao <[email protected]>

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch from 35cf7bd to e58e643 Compare October 9, 2022 03:30

github-actions bot removed the plugin label Oct 9, 2022

fix conflicts

51913ef

justadogistaken force-pushed the feature/kubefile-2-dockerfile branch from e58e643 to 51913ef Compare October 9, 2022 03:34

starnop self-requested a review October 9, 2022 06:40

starnop merged commit a20019a into sealerio:main Oct 9, 2022

jsparter mentioned this pull request Oct 20, 2022

[Feature] Add chartmuseum to base kubernetes image #1783

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sealer Application definition #1658

Sealer Application definition #1658

justadogistaken commented Aug 20, 2022 •

edited

Loading

codecov-commenter commented Aug 22, 2022 •

edited

Loading

allencloud Aug 31, 2022

allencloud commented Aug 31, 2022

justadogistaken Sep 5, 2022

starComingup Sep 7, 2022

allencloud commented Sep 8, 2022

justadogistaken commented Sep 8, 2022

allencloud commented Sep 9, 2022

justadogistaken commented Sep 9, 2022

justadogistaken commented Sep 28, 2022 •

edited

Loading

kakaZhou719 commented Sep 29, 2022

kube-installer:

app-installer:

justadogistaken commented Sep 29, 2022

allencloud Sep 29, 2022

allencloud Sep 29, 2022

allencloud Sep 29, 2022

allencloud Sep 29, 2022 •

edited

Loading

justadogistaken Sep 30, 2022

justadogistaken commented Oct 4, 2022

allencloud Oct 4, 2022

justadogistaken Oct 5, 2022

starnop Oct 9, 2022

justadogistaken Oct 9, 2022

starnop commented Oct 9, 2022

		There can be only one `LAUNCH` or `CMDS` instruction in a `Kubefile`.

		### CMDS

Sealer Application definition #1658

Sealer Application definition #1658

Conversation

justadogistaken commented Aug 20, 2022 • edited Loading

Design

Overview

Implementation

KubefileParser

Note

Additional

Example

codecov-commenter commented Aug 22, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

allencloud commented Aug 31, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

allencloud commented Sep 8, 2022

justadogistaken commented Sep 8, 2022

allencloud commented Sep 9, 2022

justadogistaken commented Sep 9, 2022

justadogistaken commented Sep 28, 2022 • edited Loading

kube-installer:

app-installer:

kakaZhou719 commented Sep 29, 2022

kube-installer:

app-installer:

justadogistaken commented Sep 29, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

allencloud Sep 29, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justadogistaken commented Oct 4, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

starnop commented Oct 9, 2022

justadogistaken commented Aug 20, 2022 •

edited

Loading

codecov-commenter commented Aug 22, 2022 •

edited

Loading

justadogistaken commented Sep 28, 2022 •

edited

Loading

allencloud Sep 29, 2022 •

edited

Loading