Execute integration tests in DigitalOcean #144
Conversation
|
|
||
| ## Communication | ||
|
|
||
| ### GitHub repositories | ||
|
|
||
| Much of the issues, goals and ideas are tracked in the respective projects in GitHub. Please use this channel to report bugs and post ideas. | ||
|
|
||
| ### Trello |
There was a problem hiding this comment.
I removed Trello as it looks dead, I hope thats fine
|
If changes in |
|
it looks like supermarket has some issues... |
There was a problem hiding this comment.
Hi @artem-sidorenko Great work, Digital Ocean config looks great. Is there a specific reason why you removed the dokken and vagrant kitchen support? This does not allow me to run integration tests locally anymore.
| @@ -1,44 +0,0 @@ | |||
| --- | |||
| driver: | |||
There was a problem hiding this comment.
I would prefer to have:
.kitchen.yml- Docker-based.kitchen.vagrant.yml- vagrant based.kitchen.digitalocean.yml- in digital ocean
This would allow all users to run the tests locally as well.
There was a problem hiding this comment.
well, I did not remove it, I tried to have only one .kitchen.yml:
- DRY
- currently kitchen-dokken does not work here
So the idea was to have only one kitchen.yml. Currently you will see locally the vagrant setup and it works without any export KITCHEN_YAML steps. In the CI you have the according DO env vars configured, so it works automatically with DO.
In the end its one .kitchen.yml and not two or three, where some of this files does not get updates usually.
I'm not sure yet how this should/could be implemented for chef-ssh-hardening, as somebody might use kitchen-dokken locally (what leads for my system to problems because of systemd) or some might use kitchen-vagrant locally. But this is a bit another discussion
Can you tell me where exactly do you see a problem with this approach? Do you have any suggestions how we can resolve that but in the same time keep one file?
There was a problem hiding this comment.
I agree with your aim to merge the kitchen.yml into a more maintainable structure. We could try to use a base configuration and capture the provider in a local kitchen config (KITCHEN_LOCAL_YAML). This would allow us to:
- kitchen.yml with vagant
- kitchen.do.local.yml for DigitalOcean
The kitchen local approach does not work well with kitchen-dokken, since nearly everything needs to be overwritten. We could switch to kitchen-docker, but this works by installing ssh into the container while kitchen-dokken is not.
The initial idea of switching to kitchen-dokken was to do all integration tests in travis. Therefore kitchen-dokken was the best solution for all chef projects (except for os-hardening). It works within the travis without external dependencies and is for free. I agree that we need to have another solution for os-hardening since we are testing kernel parameters, but I am not sure we should switch all other projects.
My main goal is to have a self-contained project that can be fully tested locally. I like the digital ocean idea, since it increases test coverage. I do not think we should restrict testing to digital ocean.
@artem-sidorenko Thoughts?
There was a problem hiding this comment.
@chris-rock I have the same view more or less. I will have a look how to rework it with kitchen local approach, its a good idea, thanks!
I agree that we need to have another solution for os-hardening since we are testing kernel parameters, but I am not sure we should switch all other projects.
I'm not sure as well, but lets have this discussion outside of this PR
There was a problem hiding this comment.
@chris-rock I switched from erb approach to the kitchen local approach. Can you please have a look?
There was a problem hiding this comment.
@chris-rock ping
.travis.yml
Outdated
| script: | ||
| - bundle exec rake prepare_do_env kitchen | ||
|
|
||
| matrix: |
There was a problem hiding this comment.
maybe we should use a syntax like:
matrix:
include:
- rvm: 2.3.3
bundler_args: "--without guard tools"
script: bundle exec rake $SUITE
env: SUITE=test:integration OS=default-ubuntu-1204 DOCKER=true
reference: https://github.com/chef/inspec/blob/master/.travis.yml#L15-L31
There was a problem hiding this comment.
@chris-rock could you maybe explain it? Maybe I miss something: via your suggestion it gets 3x-4x more configuration lines with exactly the same result. The "job definition" is a bit easier to understand, however .travis.yml is always a bit ugly/complicated in my eyes if I compare it for instance with .gitlab-ci.yml job definitions...
I used the approach of chef-client cookbook here: https://github.com/chef-cookbooks/chef-client/blob/master/.travis.yml
There was a problem hiding this comment.
Agreed. Lets use your approach.
artem-sidorenko
force-pushed
the
do-travis
branch
from
264c066 to
9cfe819
Compare
There was a problem hiding this comment.
This is awesome work @artem-sidorenko I am seeing some errors in our integration tests, therefore it is good to see this executed during every merge! Thank you @artem-sidorenko
Focus of this PR is only to get the travis-kitchen-DO setup running. I'll fix the failures in the next PRs