vdk-heartbeat: cover requirements.txt automatic installs #2393
Conversation
Upon `vdk deploy` the Control Service is responsible for installing all external dependecies defined in requirements.txt of the data job so they can be used during a cloud run. Recent outage in one of our customers which caused data jobs have not had their requirements installed. It was trace to be due to a release of [data job base image](https://hub.docker.com/layers/versatiledatakit/data-job-base-python-3.11) This was not caught by CICD of the open source VDK. This change aim to address this gap and to cover this improtnat functionality for correct installation of data job's python requirements.
antoniivanov
force-pushed
the
person/aivanov/vdk-heartbeat
branch
from
deff8c1 to
423d004
Compare
|
Is this PR just a test or a fix for the issue too? |
It is only a test. The fix(es) would be separate PR(s). But it fails to reproduce the issue. So I am trying to figure out why ... It's still valuable coverage. So likely I will keep it. |
The fix is in #2429 |
projects/vdk-heartbeat/src/vdk/internal/heartbeat/vdk_heartbeat_data_job/trino/requirements.txt
Show resolved
Hide resolved
Wait, I'm confused now. 😄 If you're just testing something, please mark it as draft. |
No, I mean , this is not fixing the issue , The fix is in separate PR (#2429). |
Upon
vdk deploythe Control Service is responsible for installing all external dependecies defined in requirements.txt of the data job so they can be used during a cloud run.A recent outage in one of our customers caused data jobs to ignore their requirements.txt and not install any external libraries. It was trace to be due to a release of data job base
image
CICD of the open source VDK did not catch this.
I am extending VDK heartbeat to test for the issue so we can catch it in the future.
See also #2391