Storing and updating PipelineSpec

[jerop]

"I was having trouble seeing unresolved when expressions in status.pipelineSpec.tasks[] and was trying to troubleshoot and discovered we actually store the pipelineSpec as is even before resolving params and it's never updated for auditing purpose. But wouldn't it be clearer if its updated once after resolving all params (and pipeline resources)?" ~ @pritidesai

Originally posted by @pritidesai in #3135 (comment)

[bobcatfish]

Some context:

• feature request: snapshot pipeline data in pipelineRun for historical and repeatability purposes #279 was the original issue about this
• in PipelineRun Status bloat and excessive updates #3140 we're getting feedback that we might be storing too much
• the functionality was added in Dereference the TaskSpec into TaskRun.Status. #2444 and Dereference the PipelineSpec into PipelineRun.Status #2489

I think the motivation for storing this at all is to be able to tell exactly what spec the pipeline was run with, since pipelines can be modified later

But wouldn't it be clearer if its updated once after resolving all params (and pipeline resources)?

probably depends on why we're storing this information - i think it's less about being easy to see what happened and more about a historical record. if we mutated it before storing it, then the spec in this section would differ between runs

[pritidesai]

Yup agree, it depends on how it's being consumed. The spec is populated right after reading it during the first reconciler cycle and not updated based on the informer's copy. But if there is no objection, the spec can be populated after resolving all params and pipeline resources during the first reconciler cycle so that it does not differ between runs. This also means that spec will not be available or populated if it fails any of the validation check (invalid DAG, invalid syntax, invalid resource bindings, invalid params, etc) unlike today.

[jerop]

Duplicate in: #3140
/close

[tekton-robot]

@jerop: Closing this issue.

In response to this:

Duplicate in: #3140
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.