Define db and ssh test markers and run db tests in ci pipeline

[jenhagg]

Purpose

Allow selecting db tests specifically, or filtering them out to run other integration tests, e.g. pytest -m 'not db'. Run db tests in github action.

What it does

Define the db mark in pytest.ini and apply it to relevant tests, in addition to the integration mark. Also defined the ssh mark to deselect those from github test runs, while selecting just those locally. New step was added to the ci .yml to start the db container, enabling tests to run against it.

Time to review

10 min

[danielolsen]

If we are marking these tests as db, should we remove the integration mark? They need to connect to the local db, not the server.

[rouille]

If we are marking these tests as db, should we remove the integration mark? They need to connect to the local db, not the server.

I agree with @danielolsen. Also, perhaps the marker name integration is not well chosen or could be more explicit. What do you think of ssh, server or ssh connection?

[jenhagg]

I suppose we could name the marks anything as long as we can select subsets of tests for our purposes. My thinking was that integration means tests that have any external dependency - server connection, db, 3rd party api, etc. So I think that's a useful thing to keep, but adding the db mark lets us select those separately, and we can still run the combinations we currently want. Open to other ideas though.

[danielolsen]

I suppose we could name the marks anything as long as we can select subsets of tests for our purposes. My thinking was that integration means tests that have any external dependency - server connection, db, 3rd party api, etc. So I think that's a useful thing to keep, but adding the db mark lets us select those separately, and we can still run the combinations we currently want. Open to other ideas though.

I definitely see the value in having a 'no external dependencies' flag to run with pytest. Right now the description for integration talks about sshing to the server, so maybe we want three tags: integration, db, and ssh, where ssh is a subset of integration, and perhaps db is too (can we run the db tests on github?).

[jenhagg]

I will update the description of integration. We should be able to run db on github soon, so in that case we might want to select not ssh, but I think until then the ssh mark would be redundant?

edit: I actually have a branch that was able to run the db tests in github, so I can add the ssh mark and include that in this PR
edit: I integrated those changes, think it's good now

[jenhagg]

The loop waits for the container to start, otherwise tests can fail due to race condition. nc -z checks if the port is listening.

[danielolsen]

What's going on with these changes?

[jenhagg]

They kinda snuck in from the branch I was testing on, but kept it since the simpler approach seemed better. Github treats 3.x as the latest minor release which I think is currently 3.8.5, so should be compatible with 3.8.3 which we mainly use. I think the main reason to test older versions is if we were distributing this for installation in other environments and wanted the flexibility. So I didn't see it adding much value, but don't mind keeping it if there's a use case.

[danielolsen]

Sounds good

[danielolsen]

Thanks! This will definitely make local development easier.