TST/CI: Unify CI test scripts (single and multi) and simplify the system #23924

datapythonista · 2018-11-26T11:33:03Z

refs CLN: Reestructure tools and scripts #23658
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

The goal of this PR is:

Avoid repetitive code by merging ci/script_single.sh and ci/script_multi.sh
Simplify calls to different tests (mainly slow vs not slow), by avoiding the previously two different syntax (SLOW=true and --only-slow) and using the pytest native syntax instead
Integrate upload_coverage.sh into the new unified script to avoid the extra complexity of having many scripts

pep8speaks · 2018-11-26T11:33:05Z

Hello @datapythonista! Thanks for submitting the PR.

There are no PEP8 issues in the file pandas/conftest.py !

WillAyd · 2018-11-26T11:44:30Z

ci/upload_coverage.sh

-fi
-
-
-echo "uploading coverage"


Any reason to remove this echo? Figure it doesn't hurt to keep around to give an indication that we've gotten to this point

makes sense, got lost when moving the code

codecov · 2018-11-26T12:56:09Z

Codecov Report

Merging #23924 into master will decrease coverage by 49.84%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master   #23924       +/-   ##
===========================================
- Coverage    92.3%   42.45%   -49.85%     
===========================================
  Files         161      161               
  Lines       51551    51549        -2     
===========================================
- Hits        47585    21886    -25699     
- Misses       3966    29663    +25697

Flag	Coverage Δ
#multiple	`?`
#single	`42.45% <ø> (+0.01%)`	⬆️

Impacted Files	Coverage Δ
pandas/io/formats/latex.py	`0% <0%> (-100%)`	⬇️
pandas/core/categorical.py	`0% <0%> (-100%)`	⬇️
pandas/io/sas/sas_constants.py	`0% <0%> (-100%)`	⬇️
pandas/tseries/plotting.py	`0% <0%> (-100%)`	⬇️
pandas/tseries/converter.py	`0% <0%> (-100%)`	⬇️
pandas/io/formats/html.py	`0% <0%> (-97.64%)`	⬇️
pandas/core/groupby/categorical.py	`0% <0%> (-95.46%)`	⬇️
pandas/io/sas/sas7bdat.py	`0% <0%> (-91.17%)`	⬇️
pandas/io/sas/sas_xport.py	`0% <0%> (-90.15%)`	⬇️
pandas/core/tools/numeric.py	`10.44% <0%> (-89.56%)`	⬇️
... and 119 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c3d9cb...8c9b557. Read the comment docs.

jreback · 2018-11-26T13:17:03Z

ci/run_tests.sh

+        NUM_JOBS=2
+    fi
+
+    pytest -m "$TYPE_PATTERN$PATTERN" -n $NUM_JOBS -s --strict --durations=10 --junitxml=test-data-$TYPE.xml $TEST_ARGS $COVERAGE pandas


you have TEST_ARGS here, though you removed it elsewhere

I wanted to get rid of it, but it's needed in the numpydev build for the -W error

any other way to do that. this makes it pretty clunky.

I want to take a look at pytest concurrency in a different PR.

I think it should be a simpler way to avoid running the simple tests in parallel, without having to call pytest twice. I think that would be cleaner, and we can simplify all this even more.

Also, I want to see how many processes we have, and not call always for 2 (which I assume it's the number of cores in Travis), but instead use all the available (I think that should be as easy as -n auto, but I want to take a look at how many cores are detected in azure, and how many makes sense to use).

the single tests must remain though
we got failures because these are using single system resources

jreback · 2018-11-26T13:17:22Z

pandas/conftest.py

@@ -28,28 +28,13 @@


 def pytest_addoption(parser):
-    parser.addoption("--skip-slow", action="store_true",


i wouldn't remove these, these are useful from the command line

Any reason why to use pytest --skip-slow instead of pytest -m "no slow"? Not a big deal to restore them, but I personally think that they are more complexity than value.

yes this is a pretty common idiom. I don't see value in removing them.

@datapythonista can you restore these

jreback · 2018-11-29T13:59:20Z

pandas/conftest.py

@@ -28,28 +28,13 @@


 def pytest_addoption(parser):
-    parser.addoption("--skip-slow", action="store_true",


@datapythonista can you restore these

datapythonista · 2018-11-29T21:46:43Z

@jreback

jreback · 2018-11-29T22:06:51Z

looks like the doc build failed: https://travis-ci.org/pandas-dev/pandas/builds/461388989?utm_source=github_status&utm_medium=notification

i restarted, let's see.

datapythonista · 2018-11-29T23:20:24Z

I didn't see the docs failed. I forgot to skip the tests in the doc build, and it was timing out doing both things.

It should be fixed now.

jreback · 2018-11-30T21:33:47Z

thanks @datapythonista

…tem (pandas-dev#23924) * WIP: unifying script_single and script_multi * Unified scripts that run the tests (single and multi) * pending fixes * logging when we upload the coverage * Restoring pytest --skip-slow... options * Not running tests on doc build

datapythonista added 3 commits November 26, 2018 10:46

WIP: unifying script_single and script_multi

252729c

Merging from master

be9fb9b

Unified scripts that run the tests (single and multi)

12bb059

datapythonista added Testing pandas testing functions or related to the test suite CI Continuous Integration Clean labels Nov 26, 2018

pending fixes

9fac328

WillAyd reviewed Nov 26, 2018

View reviewed changes

logging when we upload the coverage

93c41db

jreback requested changes Nov 26, 2018

View reviewed changes

datapythonista mentioned this pull request Nov 26, 2018

TST: Run tests using a database explicitly #23928

Closed

jreback requested changes Nov 29, 2018

View reviewed changes

datapythonista added 2 commits November 29, 2018 17:20

Merge remote-tracking branch 'upstream/master' into unify_test_scripts

18446b2

Restoring pytest --skip-slow... options

a17a9ce

jreback added this to the 0.24.0 milestone Nov 29, 2018

jreback approved these changes Nov 29, 2018

View reviewed changes

Not running tests on doc build

8c9b557

jreback merged commit 5411421 into pandas-dev:master Nov 30, 2018

datapythonista mentioned this pull request Nov 30, 2018

CI: Move CircleCI job to Azure pipelines #23821

Closed

datapythonista mentioned this pull request Dec 3, 2018

CI/TST: Making ci/run_tests.sh fail if one of the steps fail #24075

Merged

jbrockmendel mentioned this pull request Sep 23, 2020

CLN: Reestructure tools and scripts #23658

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TST/CI: Unify CI test scripts (single and multi) and simplify the system #23924

TST/CI: Unify CI test scripts (single and multi) and simplify the system #23924

datapythonista commented Nov 26, 2018 •

edited

Loading

pep8speaks commented Nov 26, 2018

WillAyd Nov 26, 2018

datapythonista Nov 26, 2018

codecov bot commented Nov 26, 2018 •

edited

Loading

jreback Nov 26, 2018

datapythonista Nov 26, 2018

jreback Nov 27, 2018

datapythonista Nov 29, 2018

jreback Nov 29, 2018

jreback Nov 26, 2018

datapythonista Nov 26, 2018

jreback Nov 27, 2018

jreback Nov 29, 2018

jreback Nov 29, 2018

datapythonista commented Nov 29, 2018

jreback commented Nov 29, 2018

datapythonista commented Nov 29, 2018

jreback commented Nov 30, 2018

		@@ -28,28 +28,13 @@


		def pytest_addoption(parser):
		parser.addoption("--skip-slow", action="store_true",

TST/CI: Unify CI test scripts (single and multi) and simplify the system #23924

TST/CI: Unify CI test scripts (single and multi) and simplify the system #23924

Conversation

datapythonista commented Nov 26, 2018 • edited Loading

pep8speaks commented Nov 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Nov 26, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

datapythonista commented Nov 29, 2018

jreback commented Nov 29, 2018

datapythonista commented Nov 29, 2018

jreback commented Nov 30, 2018

datapythonista commented Nov 26, 2018 •

edited

Loading

codecov bot commented Nov 26, 2018 •

edited

Loading