Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement When Expressions #3117

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions docs/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ weight: 3
- [Using the `from` parameter](#using-the-from-parameter)
- [Using the `runAfter` parameter](#using-the-runafter-parameter)
- [Using the `retries` parameter](#using-the-retries-parameter)
- [Guard `Task` execution using `When Expressions`](#guard-task-execution-using-whenexpressions)
- [Guard `Task` execution using `Conditions`](#guard-task-execution-using-conditions)
- [Configuring the failure timeout](#configuring-the-failure-timeout)
- [Using `Results`](#using-results)
Expand Down Expand Up @@ -316,8 +317,60 @@ tasks:
name: build-push
```

### Guard `Task` execution using `WhenExpressions`

To run a `Task` only when certain conditions are met, it is possible to _guard_ task execution using
the `when` field. The `when` field allows you to list a series of references to `WhenExpressions`.
jerop marked this conversation as resolved.
Show resolved Hide resolved

There are a lot of scenarios where `WhenExpressions` can be really useful. Some of these are:
- Checking if the name of a git branch matches
- Checking if the `Result` of a previous `Task` is as expected
- Checking if a git file has changed in the previous commits
- Checking if an image exists in the registry
- Checking if the name of a CI job matches

The components of `WhenExpressions` are `Input`, `Operator` and `Values`:

- `Input` is the input for the `WhenExpression` which can be static inputs or variables (`Parameters` or `Results`).
- `Operator` represents an `Input`'s relationship to a set of `Values`. `Operators` we will use in `WhenExpressions` are `in` and `notin`.
- `Values` is an array of string values. The `Values` array must be non-empty. It can contain static values or variables (`Parameters` or `Results`.

The declared `WhenExpressions` are evaluated before the `Task` is run. If all the `WhenExpressions`
evaluate to `True`, the `Task` is run. If any of the `WhenExpressions` evaluate to `False`, the `Task` is
not run and the `TaskRun` status field `ConditionSucceeded` is set to `False` with the reason set to
`WhenExpressionsEvaluatedToFalse`.

In these examples, `create-readme-file` task will only be executed if the `path` parameter is `README.md` and `echo-file-exists` task will only be executed if the `status` result from `check-file` task is `exists`.

```yaml
tasks:
- name: first-create-file
when:
- input: "$(params.path)"
operator: in
values: ["README.md"]
taskRef:
name: create-readme-file
---
tasks:
- name: echo-file-exists
when:
- input: "$(tasks.check-file.results.status)"
operator: in
values: ["exists"]
taskRef:
name: echo-file-exists
```

For an end-to-end example, see [PipelineRun with WhenExpressions](../examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml).

When `WhenExpressions` are specified in a `Task`, [`Conditions`](#guard-task-execution-using-conditions) should not be speficied in the same `Task`.
The `Pipeline` will be rejected as invalid if both `WhenExpressions` and `Conditions` are included.

### Guard `Task` execution using `Conditions`
jerop marked this conversation as resolved.
Show resolved Hide resolved

**Note:** `Conditions` are deprecated, use [`WhenExpressions`](#guard-task-execution-using-whenexpressions) instead.

To run a `Task` only when certain conditions are met, it is possible to _guard_ task execution using
the `conditions` field. The `conditions` field allows you to list a series of references to
[`Condition`](./conditions.md) resources. The declared `Conditions` are run before the `Task` is run.
Expand Down
239 changes: 239 additions & 0 deletions examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
# Copied from https://github.com/tektoncd/catalog/blob/v1beta1/git/git-clone.yaml :(
# This can be deleted after we add support to refer to the remote Task in a registry (Issue #1839) or
# add support for referencing task in git directly (issue #2298)
jerop marked this conversation as resolved.
Show resolved Hide resolved
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-clone-from-catalog
spec:
workspaces:
- name: output
description: The git repo will be cloned onto the volume backing this workspace
params:
- name: url
description: git url to clone
type: string
- name: revision
description: git revision to checkout (branch, tag, sha, ref…)
type: string
default: master
- name: refspec
description: (optional) git refspec to fetch before checking out revision
default: ""
- name: submodules
description: defines if the resource should initialize and fetch the submodules
type: string
default: "true"
- name: depth
description: performs a shallow clone where only the most recent commit(s) will be fetched
type: string
default: "1"
- name: sslVerify
description: defines if http.sslVerify should be set to true or false in the global git config
type: string
default: "true"
- name: subdirectory
description: subdirectory inside the "output" workspace to clone the git repo into
type: string
default: ""
- name: deleteExisting
description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there
type: string
default: "false"
- name: httpProxy
description: git HTTP proxy server for non-SSL requests
type: string
default: ""
- name: httpsProxy
description: git HTTPS proxy server for SSL requests
type: string
default: ""
- name: noProxy
description: git no proxy - opt out of proxying HTTP/HTTPS requests
type: string
default: ""
results:
- name: commit
description: The precise commit SHA that was fetched by this Task
steps:
- name: clone
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.12.1
script: |
CHECKOUT_DIR="$(workspaces.output.path)/$(params.subdirectory)"

cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/"
# or the root of a mounted volume.
if [[ -d "$CHECKOUT_DIR" ]] ; then
# Delete non-hidden files and directories
rm -rf "$CHECKOUT_DIR"/*
# Delete files and directories starting with . but excluding ..
rm -rf "$CHECKOUT_DIR"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "$CHECKOUT_DIR"/..?*
fi
}

if [[ "$(params.deleteExisting)" == "true" ]] ; then
cleandir
fi

test -z "$(params.httpProxy)" || export HTTP_PROXY=$(params.httpProxy)
test -z "$(params.httpsProxy)" || export HTTPS_PROXY=$(params.httpsProxy)
test -z "$(params.noProxy)" || export NO_PROXY=$(params.noProxy)

/ko-app/git-init \
-url "$(params.url)" \
-revision "$(params.revision)" \
-refspec "$(params.refspec)" \
-path "$CHECKOUT_DIR" \
-sslVerify="$(params.sslVerify)" \
-submodules="$(params.submodules)" \
-depth "$(params.depth)"
cd "$CHECKOUT_DIR"
RESULT_SHA="$(git rev-parse HEAD | tr -d '\n')"
EXIT_CODE="$?"
if [ "$EXIT_CODE" != 0 ]
then
exit $EXIT_CODE
fi
# Make sure we don't add a trailing newline to the result!
echo -n "$RESULT_SHA" > $(results.commit.path)
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: check-file
spec:
params:
- name: path
workspaces:
- name: source
description: The workspace to check for the file.
results:
- name: status
description: indicating whether the file exists
steps:
- name: check-file
image: alpine
script: |
if test -f $(workspaces.source.path)/$(params.path); then
printf exists | tee /tekton/results/status
else
printf missing | tee /tekton/results/status
fi
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: guarded-pipeline
spec:
params:
- name: path
type: string
description: The path of the file to be created.
default: "README.md"
- name: repo-url
type: string
description: The git repository URL to clone from.
- name: branch-name
type: string
description: The git branch to clone.
workspaces:
- name: source-repo
description: |
This workspace will receive the cloned git repo and be passed
to the next Task to create a file.
tasks:
- name: fetch-repo
taskRef:
name: git-clone-from-catalog
params:
- name: url
value: $(params.repo-url)
- name: revision
value: $(params.branch-name)
workspaces:
- name: output
workspace: source-repo
- name: create-file
when:
- input: "$(params.path)"
operator: in
values: ["README.md"]
workspaces:
- name: source
workspace: source-repo
runAfter:
- fetch-repo
taskSpec:
workspaces:
- name: source
description: The workspace to create the readme file in.
steps:
- name: write-new-stuff
image: ubuntu
script: 'touch $(workspaces.source.path)/README.md'
- name: check-file
when:
- input: "foo"
operator: in
values: ["foo", "bar"]
params:
- name: path
value: "$(params.path)"
workspaces:
- name: source
workspace: source-repo
taskRef:
name: check-file
runAfter:
- create-file
- name: echo-file-exists
when:
- input: "$(tasks.check-file.results.status)"
operator: in
values: ["exists"]
taskSpec:
steps:
- name: echo
image: ubuntu
script: 'echo file exists'
- name: task-should-be-skipped
when:
- input: "foo"
operator: notin
values: ["foo"]
- input: "foo"
operator: in
values: ["bar"]
taskSpec:
steps:
- name: echo
image: ubuntu
script: exit 1
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: guarded-pr
spec:
serviceAccountName: 'default'
pipelineRef:
name: guarded-pipeline
params:
- name: repo-url
value: https://github.com/tektoncd/pipeline.git
- name: branch-name
value: master
workspaces:
- name: source-repo
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
15 changes: 15 additions & 0 deletions internal/builder/v1beta1/pipeline.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
resource "github.com/tektoncd/pipeline/pkg/apis/resource/v1alpha1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/selection"
"knative.dev/pkg/apis"
)

Expand Down Expand Up @@ -54,6 +55,9 @@ type PipelineRunStatusOp func(*v1beta1.PipelineRunStatus)
// PipelineTaskConditionOp is an operation which modifies a PipelineTaskCondition
type PipelineTaskConditionOp func(condition *v1beta1.PipelineTaskCondition)

// PipelineTaskWhenExpressionOp is an operation which modifies a WhenExpression.
type PipelineTaskWhenExpressionOp func(*v1beta1.WhenExpression)

// Pipeline creates a Pipeline with default values.
// Any number of Pipeline modifier can be passed to transform it.
func Pipeline(name string, ops ...PipelineOp) *v1beta1.Pipeline {
Expand Down Expand Up @@ -332,6 +336,17 @@ func PipelineTaskConditionResource(name, resource string, from ...string) Pipeli
}
}

// PipelineTaskWhenExpression adds a WhenExpression with the specified input, operator and values.
func PipelineTaskWhenExpression(input string, operator selection.Operator, values []string) PipelineTaskOp {
return func(pt *v1beta1.PipelineTask) {
pt.WhenExpressions = append(pt.WhenExpressions, v1beta1.WhenExpression{
Input: input,
Operator: operator,
Values: values,
})
}
}

// PipelineTaskWorkspaceBinding adds a workspace with the specified name, workspace and subpath on a PipelineTask.
func PipelineTaskWorkspaceBinding(name, workspace, subPath string) PipelineTaskOp {
return func(pt *v1beta1.PipelineTask) {
Expand Down
11 changes: 7 additions & 4 deletions internal/builder/v1beta1/pipeline_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import (
"github.com/google/go-cmp/cmp"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/selection"
"knative.dev/pkg/apis"
duckv1beta1 "knative.dev/pkg/apis/duck/v1beta1"

Expand Down Expand Up @@ -54,6 +55,7 @@ func TestPipeline(t *testing.T) {
tb.PipelineTaskOutputResource("some-image", "my-only-image-resource"),
),
tb.PipelineTask("never-gonna", "give-you-up",
tb.PipelineTaskWhenExpression("foo", selection.In, []string{"foo", "bar"}),
tb.RunAfter("foo"),
tb.PipelineTaskTimeout(5*time.Second),
),
Expand Down Expand Up @@ -133,10 +135,11 @@ func TestPipeline(t *testing.T) {
}},
},
}, {
Name: "never-gonna",
TaskRef: &v1beta1.TaskRef{Name: "give-you-up"},
RunAfter: []string{"foo"},
Timeout: &metav1.Duration{Duration: 5 * time.Second},
Name: "never-gonna",
TaskRef: &v1beta1.TaskRef{Name: "give-you-up"},
WhenExpressions: []v1beta1.WhenExpression{{Input: "foo", Operator: selection.In, Values: []string{"foo", "bar"}}},
RunAfter: []string{"foo"},
Timeout: &metav1.Duration{Duration: 5 * time.Second},
}, {
Name: "foo",
TaskSpec: &v1beta1.EmbeddedTask{
Expand Down
Loading