Commit provisioning transaction before starting pipeline

[lovromazgon]

Description

When provisioning a pipeline through a pipeline configuration file, the pipeline is started inside of the same db transaction in which it's created. We should rather commit the transaction after creating the pipeline and starting it outside of that transaction. The main reason is that the context passed to Start should not have a transaction attached, especially not one that will be committed after the pipeline is started, as that context might be used to retrieve or write data to the database. If it contains a committed transaction, those operations will fail.

This was found when trying to create a schema in a standalone processor using the built-in schema registry:

2024-08-07T09:09:07+00:00 ERR failed executing service method error="failed to get schema by subject and version: failed to get subject schema from store: failed to get subject schema with subject:version \"test:employees-source:payload\":'\\x01': badger: could not get key \"schemaregistry:subjectschema:test:employees-source:payload:1\": This transaction has been discarded. Create a new one" component=standalone.wasmProcessor method=get_schema processor_id=test:access-schema stack=[{"file":"/Users/mahahajja/repos/conduit/pkg/plugin/processor/procutils/schema.go","func":"github.com/conduitio/conduit/pkg/plugin/processor/procutils.(*SchemaService).GetSchema","line":76}]

Quick checks

• I have followed the Code Guidelines.
• There is no other pull request for the same update/change.
• I have written unit tests.
• I have made sure that the PR is of reasonable size and can be easily reviewed.

[hariso]

Nice find!