Python drivers #5668
Merged
Python drivers #5668
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
prwolfe
added
the
AT: WIP
Because the PR testing will have it and fail if we don't
I had previously sorted the key set to avoid order mismatches between python 2/3. Fort whatever reason the tests showed AttributeError: 'KeysView' object has no attribute 'sort' in python 3 - likely a different point version.
prwolfe
removed
the
AT: WIP
|
Full testing was done on prwolfe#12 Note: after some thought I suspect this may suffer from a reverse bootstrapping issue. I.E. may fail now and would work after merging. I will monitor for this. |
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Using Repos:
Pull Request Author: prwolfe |
mhoemmen
previously approved these changes
Aug 6, 2019
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.8.4 # 4176 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_intel_17.0.1 # 3997 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.9.3_SERIAL # 2437 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_7.2.0 # 2259 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_cuda_9.2 # 1838 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_2 # 312 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_3 # 312 (click to expand) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Using Repos:
Pull Request Author: prwolfe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED Pull Request Auto Testing has PASSED (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
|
|
Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ mhoemmen ]! |
|
Status Flag 'Pull Request AutoTester' - AutoMerge IS ENABLED, but the Label AT: AUTOMERGE is not set. Either set Label AT: AUTOMERGE or manually merge the PR... |
prwolfe
added
the
AT: WIP
1) the cuda environment included a 'module swap' which was incorrectly set up and now works. 2) the shell driver script was not passing the arguments needed to the Test section. It was in fact never moved to use the python.
mhoemmen
previously approved these changes
Aug 7, 2019
Remove old Test.sh script Correct executable bit on new Test.py script
Correct path to subproject list Add cmake to python testing module list
Load python before the Merge/Test scripts - the default version is that old Correct path for cuda compilers so the nvcc wrapper can find them
mhoemmen
previously approved these changes
Aug 8, 2019
Turns out the entire dictionary is constructed every time.
The environmental changes map is built on python start up. The test was seeting the CC and FC values before that rather than at module load as the real runs will. This modifies the test and code to match what will actually occur.
It looks like the path is not getting upodated but instead replaced when modifying the environment. Since I know this works in testing and on all the other builds I am guessing that the old version of python that ride defaults to is the issue.
mhoemmen
previously approved these changes
Aug 14, 2019
This turned out to be the issue for the ride jobs - I was adding in path before loading modules and then the map update overwrote all the changes with the initial path. It did not show on other systems because they were not manually adding paths.
The previous shell scripts had unloaded the seme-python module then added path and pythonpath directly as the sierra python modules are not working for the trilinos entity. I was attempting to fix that as it would be simpler, but it looks like that is a no-go on the cloud nodes at this time.
And by checking that this comes through I should help avoid regressioooon to the issue that caused the CUDA failures.
prwolfe
removed
the
AT: WIP
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Using Repos:
Pull Request Author: prwolfe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.8.4 # 4249 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_intel_17.0.1 # 4070 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.9.3_SERIAL # 2510 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_7.2.0 # 2334 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_cuda_9.2 # 1911 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_2 # 377 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_3 # 377 (click to expand) |
mhoemmen
previously approved these changes
Aug 15, 2019
|
Hey guys, As expected this failed in a regular merge request since I am re-writing the main script. You can see the full run of testing at prwolfe#12 and results at https://testing.sandia.gov/cdash/index.php?project=Trilinos&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercombine=and&filtercount=2&showfilters=1&filtercombine=and&field1=buildname&compare1=63&value1=PR-12-test&field2=buildstarttime&compare2=84&value2=NOW (yes I ran it many times to clean up all the module loading issues.) I want more than just one review before I merge this one since I will have to force it, so please take a look. Paul |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Using Repos:
Pull Request Author: prwolfe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.8.4 # 4254 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_intel_17.0.1 # 4075 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.9.3_SERIAL # 2515 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_7.2.0 # 2339 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_cuda_9.2 # 1916 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_2 # 382 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_3 # 382 (click to expand) |
jwillenbring
previously approved these changes
Aug 22, 2019
There was a problem hiding this comment.
@prwolfe Please see a couple comments I made. I think this look ok to merge. Let's do so when you can keep an eye on the testing afterward.
|
Sounds good - I am working on the fixes for your comments now. |
|
I plan to merge this tomorrow so I can watch Fridays PR runs and I will add a few of my own just to be sure. Thanks! |
|
Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request. |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Using Repos:
Pull Request Author: prwolfe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.8.4 # 4331 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_intel_17.0.1 # 4153 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.9.3_SERIAL # 2592 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_7.2.0 # 2416 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_cuda_9.2 # 1993 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_2 # 448 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_3 # 448 (click to expand) |
|
Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects: Pull Request Auto Testing STARTING (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Using Repos:
Pull Request Author: prwolfe |
|
Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run. Pull Request Auto Testing has FAILED (click to expand)Build InformationTest Name: Trilinos_pullrequest_gcc_4.8.4
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_intel_17.0.1
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_4.9.3_SERIAL
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_gcc_7.2.0
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_cuda_9.2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_2
Jenkins Parameters
Build InformationTest Name: Trilinos_pullrequest_python_3
Jenkins Parameters
Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.8.4 # 4333 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_intel_17.0.1 # 4155 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_4.9.3_SERIAL # 2594 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_gcc_7.2.0 # 2418 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_cuda_9.2 # 1995 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_2 # 450 (click to expand)Console Output (last 100 lines) : Trilinos_pullrequest_python_3 # 450 (click to expand) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@trilinos/framework
Description
This creates python versions of our driver scripts and associated tests (fairly comprehensive, but not exaustive.) Those tests are added to the python testing in our PR system
Motivation and Context
The hope is that better testing will keep us from having as many issues when these scripts are modified. Until that point, caution is advised and this must be run by hand to resolve the bootstrapping issue that led to them being split in the first place.
How Has This Been Tested?
The unit tests pass in both python 2 and 3. The Merge script has been run directly, but the full set of 3 scripts needs tested on a full PR isolated from develop. That is next
Checklist