Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests #370

Closed
subfuzion opened this issue Oct 24, 2016 · 5 comments
Closed

Integration tests #370

subfuzion opened this issue Oct 24, 2016 · 5 comments
Milestone

Comments

@subfuzion
Copy link
Contributor

We need to increase test coverage and extend integration tests to include networking, etc (see current integration tests here).

Work on removing any redundancy with current backend integration tests (ie, those that require starting amplifier), that can be defined by yaml test specs and added here.

@bquenin
Copy link
Contributor

bquenin commented Oct 24, 2016

Do you mind if I submit a PR for renaming the test files from:

00-invalid-sample-01.yml
00-invalid-sample-02.yml
01-create-service.yml
02-list-service.yml
03-curl-service.yml
04-remove-service.yml
05-create-stack.yml
06-list-stack.yml
07-stop-stack.yml
08-restart-stack.yml
09-remove-stack.yml
10-cpu-stats.yml
11-mem-stats.yml
12-io-stats.yml
13-task-stats.yml
14-container-stats.yml
15-net-stats.yml
16-node-stats.yml
17-service-stats.yml
18-stack-logs.yml
19-numbered-logs.yml
20-metadata-logs.yml
21-all-logs.yml

to

00-invalid-sample-01.yml
00-invalid-sample-02.yml
01-service-create.yml
02-service-list.yml
03-service-curl.yml
04-service-remove.yml
05-stack-create.yml
06-stack-list.yml
07-stack-stop.yml
08-stack-restart.yml
09-stack-remove.yml
10-stats-cpu.yml
11-stats-mem.yml
12-stats-io.yml
13-stats-task.yml
14-stats-container.yml
15-stats-net.yml
16-stats-node.yml
17-stats-service.yml
18-logs-stack.yml
19-logs-numbered.yml
20-logs-metadata.yml
21-logs-all.yml

?

@ndegory
Copy link
Contributor

ndegory commented Oct 24, 2016

@bquenin if you can wait a bit before renaming the yaml files, I have a PR (#375) in progress impacting those files, thanks.

@subfuzion
Copy link
Contributor Author

subfuzion commented Oct 26, 2016

I'd like them renamed, but there is no reason for them to be numbered sequentially, giving a false sense of them running in order (see discussion here: #374 (comment)). Just drop the numbers altogether.

Also, let's get rid of "invalid". That was copied over from the stack file parse tests where it made sense to indicate that the stack file format was invalid. In addition to having support for expected output (which can be an error message just as much as it can be anything else), we should also support an expected exit code. It's not a bad idea to have a key for each test case that does indicated it is a negative ("unhappy") case test just so we can keep track and report in test results.

cc/ @ndegory @bquenin @JosephGJ @neha-viswanathan @generalhenry

@ndegory
Copy link
Contributor

ndegory commented Nov 7, 2016

IMO current status of CLI tests justifies that we close this PR (in time for 0.3.0)

@subfuzion
Copy link
Contributor Author

We made good progress on testing. This is an ongoing effort, so I'm going to update the milestone.

@subfuzion subfuzion modified the milestones: 0.4.0, 0.3.0 Nov 7, 2016
@ndegory ndegory modified the milestones: 0.5.0, 0.4.0 Dec 20, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants