Improve release process #484
Conversation
Commit Dockerfile, docker-compose and needed make target to allow running the release process from within a Docker container. The container tries to use the username/uid as the local user and bind-mounts the needed files for gpg, ssh-agent and github release token. Running `make it` is still a task left outside of the container primarily due to execution time on macOS.
As we intend to run the release script inside a container, there's no need to rely on a virtualenv, hence use `--user` to allow Python3 install the needed pip's with restricted privileges.
|
Even if this looks ok, let's hold off merging it until we feel confident it works as it should, as I had limited capability of testing everything. |
| ARG NEW_USER | ||
|
|
||
| RUN apt-get -y update && \ | ||
| apt-get install -y vim openssh-client python-tox && \ |
There was a problem hiding this comment.
out of curiosity: Why do we need vim?
There was a problem hiding this comment.
This was just in case something goes weird inside the container and we need to edit a file, while exec'ed inside the container. Since we are binding everything locally this is generally not needed, but I thought maybe there's this edge case where we need to quickly editing something (had to do it myself :) ).
|
Thanks for the review; yes I am afraid we can only properly test this on a real release; there are a few things though to try esp. on a macOS system before hand esp. if the the ssh keys get passed correctly through the agent and gpg. +1: for rebase here. |
|
Closing this, as passing ssh keys inside a container when keys are cached on macOS using keychain is not trivial. May revisit this in the future. |
`it` tests and `tox` relying on a number of Python 3 versions via pyenv are a good candidate for running through Docker, as setting up pyenv and the needed Python versions (as well as required JAVA_HOME/RUNTIME_JAVA_HOME) can be fiddly. This is an easier task than elastic#484 as it doesn't rely on passing ssh credentials via the ssh-agent. Only issue is that on operating systems where Docker runs as a VM (macOS, Windows) the default configured memory will not to be increased in order to run the entirety of the integration tests, as Rally uses the 4GB car for Elasticsearch (on top of additional tasks). However, this target is especially suitable for running integration tests inside a powerful server/workstation Linux environment without having to worry about configuring all the dependencies.
`it` tests and `tox` relying on a number of Python 3 versions via pyenv are a good candidate for running through Docker, as setting up pyenv and the needed Python versions (as well as required JAVA_HOME/RUNTIME_JAVA_HOME) can be fiddly. This is an easier task than #484 as it doesn't rely on passing ssh credentials via the ssh-agent. Only issue is that on operating systems where Docker runs as a VM (macOS, Windows) the default configured memory will not to be increased in order to run the entirety of the integration tests, as Rally uses the 4GB car for Elasticsearch (on top of additional tasks). However, this target is especially suitable for running integration tests inside a powerful server/workstation Linux environment without having to worry about configuring all the dependencies. Relates #487
Commit
Dockerfile,docker-composeand newrelease-in-dockermake targetto allow running the release process from within a Docker container.
The container tries to use the same username/uid as the local user and
bind-mounts the needed files for gpg, ssh-agent and github release
token.
The execution of
make itis still a task left outside of the containerprimarily due to execution time on macOS.
Relates #483