Skip to content

Commit 45a145e

Browse files
HyukjinKwondongjoon-hyun
authored andcommitted
[SPARK-37658][INFRA][PYTHON] Skip PIP packaging test in Jenkins
### What changes were proposed in this pull request? This PR proposes to skip PIP packaging test in Jenkins. We're already running it in GitHub Actions, and Jenkins is expected to be dropped soon this month. ### Why are the changes needed? ``` ... Writing pyspark-3.3.0.dev0/setup.cfg Creating tar archive removing 'pyspark-3.3.0.dev0' (and everything under it) Installing dist into virtual env Obtaining file:///home/jenkins/workspace/SparkPullRequestBuilder%402/python pyspark requires Python '>=3.7' but the running Python is 3.6.8 Cleaning up temporary directory - /tmp/tmp.CCragmNU1X [error] running /home/jenkins/workspace/SparkPullRequestBuilder2/dev/run-pip-tests ; received return code 1 Attempting to post to GitHub... ``` After we drop Python 3.6 at SPARK-37632, PIP packaging test fails intermittently https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/146255/console. Apparently, different Python versions are installed in some of Jenkins machines. We should skip the test in these cases. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Jenkins build in this PR would test it out. Closes #34917 from HyukjinKwon/SPARK-37658. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 816aba3 commit 45a145e

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

dev/run-tests.py

+4-3
Original file line numberDiff line numberDiff line change
@@ -500,9 +500,10 @@ def run_python_tests(test_modules, parallelism, with_coverage=False):
500500

501501

502502
def run_python_packaging_tests():
503-
set_title_and_block("Running PySpark packaging tests", "BLOCK_PYSPARK_PIP_TESTS")
504-
command = [os.path.join(SPARK_HOME, "dev", "run-pip-tests")]
505-
run_cmd(command)
503+
if not os.environ.get("AMPLAB_JENKINS"):
504+
set_title_and_block("Running PySpark packaging tests", "BLOCK_PYSPARK_PIP_TESTS")
505+
command = [os.path.join(SPARK_HOME, "dev", "run-pip-tests")]
506+
run_cmd(command)
506507

507508

508509
def run_build_tests():

0 commit comments

Comments
 (0)