Template Composition

[jithine]

What happened:

There is no option to share template configuration among different templates. Users are forced to duplicate template configuration for related templates but differs only in few builds steps.

What you expected to happen:

Screwdriver should provide a mechanism which allows templates to inherit configuration from another template, which allows

1. add more configurations like steps
2. override existing configurations like steps/image

How to reproduce it:

N/A

[tkyi]

Proposal

1. Support using a template in a template (just as template usage works in a normal screwdriver.yaml today)
2. Add new key order as an array for template owners to pick and choose steps (and specify order) from both the external template config and its own config; templates will only run steps specified in the order field in that order; all other fields will be merged as usual, with current template taking precedence over the external one

Notes:

• The current config will take precedence over the external one.
• images field will also be merged
• Question: should wrap(pre/post) and user teardown (teardown-) steps be supported for template composition?

template/external@1:

name: template/external
version: 1.2.3
description: this is the external template
maintainer: [email protected]
images:
    latest-image: node:12
    stable-image: node:10
config:
  image: stable-image
  steps:
    - install: npm install
    - test: npm test
  environment:
    FOO: bar
  secrets:
     - EXTERNAL_SECRET

current template:

name: template/current
version: 2.0.1
description: this is the current template
maintainer: [email protected]
images:
    test-image: node:18
config:
  template: template_namespace/external@1
  image: stable-image
  order: [init, install, test, end]
  steps:
    - init: echo Starting
    - install: ./run_script.sh
    - end: echo Finished
  environment:
    BAR: baz
  secrets:
     - CURRENT_SECRET

result:

name: template/current
version: 2.0.1
description: this is the current template
maintainer: [email protected]
images:
    test-image: node:18
    latest-image: node:12
    stable-image: node:10
config:
  image: node:10
  steps:
    - init: echo Starting
    - install: ./run_script.sh
    - test: npm test
    - end: echo Finished
  environment:
    BAR: baz
    FOO: bar
  secrets:
     - CURRENT_SECRET
     - EXTERNAL_SECRET

[klu909]

Should we keep record of parent/child template relationship? Also should we provide lock option to prevent overwriting certain config/step?

[jithine]

Should we keep record of parent/child template relationship? Also should we provide lock option to prevent overwriting certain config/step?

Nice catch. Yes we should. Will enable UI to show the relation.
Also we should look at how version of parent is locked into.
Regarding locking yes that's in the pipeline, whether we do it as part of Template composition or not, I will leave it up to you @klu909 & @tkyi to define.

We have thoughts on expanding step representation as part of #1645
perhaps lock can be an attribute for step which is applicable in a template context.

[tkyi]

Should we keep record of parent/child template relationship?

We will store templateId being used, just as we do for jobs in a screwdriver.yaml now.

Also should we provide lock option to prevent overwriting certain config/step?

Lock option can be done separately.

[tkyi]

Example:

[yoshwata]

#1292
It's a similar issue.