-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Source File: Update Dockerfile to latest Airbyte template #11969
Conversation
/test connector=connectors/source-file
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would leave comments in the Dockerfile explaining why we have this unique setup there and not in other connectors (may end up being a link to the docs you eventually write)
the alternative here is to leave the dockerfile as is, do what file secure is currently doing, but that limits us to only one such intra repo dependency right? I think the explicit approach is better fwiw
@sherifnada good shouts
I'll do that in the files-secure Dockerfile since that will be the one with the extra dependency lines. This one is now just matching our general format.
exactly, which is fine and functional in this case as is, but I'm altering these to create an example for the new method (allowing multi) which I can link to in the docs. |
/publish connector=connectors/source-file
|
RUN apt-get update | ||
WORKDIR /airbyte/integration_code | ||
COPY setup.py ./ | ||
RUN pip install --prefix=/install . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we just install this in /usr/local instead of doing the whole multi stage thing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've emulated our current generator template.
afaia, the benefit to multi-staging like this is that any unneeded build dependencies, artifacts aren't in the final Docker image, potentially reducing image size (although I haven't tested how much difference it's actually making in the Airbyte case)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh ok
* update to our newer Dockerfile format * fixes to source-file spec * auto-bump connector version Co-authored-by: Octavia Squidington III <[email protected]>
What
As a POC part of #6455, I'm going to have source-file-secure use the new intra-repo dependency method to inherit from source-file.
This PR simply updates source-file's Dockerfile to our current layout so that this will actually work. This is because the intra-repo dependency method (at least the template example) relies on py libs & module code files from the dependency module being in a particular directory in its docker image.
note: this isn't going to be a necessity, the paths can be changed to suit the dependency docker image locations, however I want this as a look-at-me example in the docs so best to use standard layout.
also, updated the spec.json so it passes acceptance tests