Update zos_mvs_raw.py - removed duplicate name entry from the examples. #279
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Removed the duplicate "name" entry from the examples. Defect fix for https://jsw.ibm.com/browse/NAZARE-6112.
ddimatos
approved these changes
May 19, 2021
ddimatos
added a commit
that referenced
this pull request
Jul 23, 2021
* Update to support job id up to 7 numeric positions (#259) Signed-off-by: ddimatos <[email protected]> * Bugfix/3687 (#251) * 3687: looking for 'JCL ERROR' in return code message, since that will not contain a "CC" value to return. Temporarily assigning that a CC=16 * Expanded changes for JCL error to 'fail fast' + added "JCL ERROR" to job_completion_messages list + corrected grammar in error message@615 (was 'slow to response') * rebuilt timing delay loops for zos_job_submit * fixed variable usage, added more meaningful error for JCL ERROR and ABEND detectors * simplified jcl error search string corrected logic in test (testing for failure) * added debug call/conv strings to return from job_submit, to understand blank returns * argh...typo in the tracking string * continuing to track non-response issue * corrected issue with temp_file.name (may need to add comment that it is a string, not an object) * eliminated duplicate temp file 2 removal * adding badjcl indicator, since the glitch moved * added another fail_json to job_submit * removed failed=true as an assertion * removed tracking statements. * minor corrections on comments... still need to discuss return code and no-wait action changes * changed job to return msg_code=None on JCL error updated documentation of return to note this * Changed documentation to help clarify the wait/wait_time_s issue Fixed another all-caps error message Added JobId info to long-running meessage note. * forgot to switch back to black editor...autopep8 moved the import statements argh! * Added discovered customer issue into description for the 'cmd' parame… (#265) * Added discovered customer issue into description for the 'cmd' parameter. * Update zos_operator.py Minor text edits * removed colon from 'for example', which tripped a sanity test in documentation Co-authored-by: RadhaV2020 <[email protected]> * Bugfix/5950 (#260) * bugfix 5950 (exception thrown when a job info didn't find a matching job) New behavior: job.py returns a single record populated with "NOTFOUND" output, query, and submit functions had a small change to document that a single notfound could be returned test feature added to look for job_name=NOTAJOB to see if the notfound entry is returned * minor change removing a print statement from job.py * added missing blank line between functions (pep8) * corrected two outdated comments, and verified 'not found' code is in place. * standardized on NOTFOUND, tweaked stepname. fixed the 'no jobs found' reaction. * correction on adding a 'not found' message...needed to pass blanks in one case. * corrected issue with job value assignment in 'jobs not found' corrected grammatical typo in error message in submit * changes to get documentation more consistent, and not-found response more system complient * corrected test_job_output to look for the new 'not found' text * Changed the not found response to give None responses on system/subsystem Documentation was general enough to work with this Changed unit test to look for the ret_code/msg value * job.py didn't save, so I had to re-save and re-push * typo in tester: typed "Jobs" instead of "jobs". argh! * Changed return code to JOB NOT FOUND on empty record, to match other response messages. Changed JCLERROR return code to mixed case, for readability. Corrected documentation entries that needed to match code changes. Changed output test to match code changes. * Update zos_mvs_raw.py (#279) Removed the duplicate "name" entry from the examples. Defect fix for https://jsw.ibm.com/browse/NAZARE-6112. * Enhancement/3723 (#256) * added '_parse_stepwise' to job_output .. this should add job["ret_code"]["stepwise"][<step_name>] = <last field of CC string> to the job entries .. assuming that the full response is available when the data is pulled * whoopsie: pep8 issue on a regex string * .. expanded documentation to show 'steps' return in the 3 impacted functions .. added/expanded a test on zos_job_output to make sure steps were being returned * Adjusting return definition for yaml compliance Correcting typo in 'sample' return, that wasn't encapsulated Added 1 more layer to output function test * added pretty print to uss_file_to_mvs_pds_member to see what is happening there * added more output around new sibmit result test * issue moved, so more output messages * added directory list to exception message...temporary * listdir not listdirs... argh * pep8 issue * correction because black formatter couldn't handle [0: * removing tmp literal * simplified vars() to print(result) * changed a dictionary (typo) back to a list, so I can append to it. * more test tweaking, on test zos_job_output * corrected test to look for steps in ret_code, not in main return object * corrected path import * changed a test in zos_data_set, which doesn't account for the response on a pre-existing file. * added tracking output to test_uss_encoding_conversion_mvs_ps_to_mvs_vsam (looks like a case where submit may not return a job list?) * copied 5950 changes to (job.py) into this branch. changed 'stepid' to 'step_name' and 'stepcc' to 'step_name' (snake?). updated documentation in submit and output commands. * turns out ansible sanity checks flake-8 as well as pep-8. Space before semicolon line 305 of encode. * Changed unit test to use new 'step_name' instead of 'stepid'. * Merged job.py 5950 into this ticket. Added 'JOB NOT FOUND' into doc message in job.py. Corrected description of 'abend' example in all 3 zos_job modules. (be instead of better). * Corrected documentation to match field names from job.py. Swapped "The" to replace "Actual" because of implications. * Copy6113 (#282) * Initial proposal to have copy create data set when copying uss file to non-existant data set. Code is there, documentation is there, but I haven't done sanity run yet. * Removal of model_ds from copy function and sub-functions Now through ansible-sanity. This should address Jira 6183 as well as 6113 Note: 2 minor changes to zos_fetch based on a sanity test issue there with 128/130 strings * Changed documentation to set force default to true, which is implemented in action * Changes based on demetri's testing: 1 - Case correction for backup_name member 2 - explicitly generating member name, instead of depending on dcp to generate a value * tried to remove extra blank line on 1905 * Changes based on pull request comments. * Pull request updates to doc Signed-off-by: ddimatos <[email protected]> * partial change moving options to suboptions - not stable * Local *3.8) sanity passing on changed indents and size changes. Need to do playbook testing. * corrected zos_copy to auto-populate destination_dataset if it is not provided * removed a find_test that looked for files it had not created. * corrected suboptions in dictionary definition * bugfixes from playbook: copy to pds was allowing invalid filenames (period in member, member>char, lower member) rebuild workbook tests to make sure these tested okay Co-authored-by: ddimatos <[email protected]> * Feature zos_mount module added (#280) * Initial work on zos_mount. This is NOT complete. Making sure push goes to the right place. * Initial commit of git_mount: needs unit tests * Basic mount/unmount testing passed manually * made interface more ansible-like cleaned up formatting saving before new test runs * passed PEP8 testing in ansible-sanity... still working on documentation * All p3p8 resolved. Down to 6 validte-module issues. * final series of ansible-test issues resolved locally moving to server-level testing * added mount to correct directory * removed zos_mount from wrong folder * GPL3 header correction * Update zos_mount.py Added second line between metadata and documentation, to see if that addresses the license issue * new parameters for mount, aliases for mount, updated ignore files * minor tweak to ignore file * corrected string search and use of tmp directory * corrected code for pep8 provisions * pulled dev back in for code updates. Touched ignore files. * Responded to Blake and Nehnam's recommendations, except: 1 - Still not sure if cp->zos_copy change will help 2 - Need to add unit tests 3 - Need to run my test playbooks again * responding to Blake and Behnam's ssuggestions * code changes per Blake and Behnam's suggestions Still looking at zos_copy Still need to build unit tests * simplified df to search with python for safe shell run... re-tested mount idempotence Need to test multiline comments for bpxprm modifier * corrections for multiline comments. All manually tested. first pass at function test entries. * removed zos_mount functional test until I can get it to pass locally. * forcing rebuild * corrected pep8 errors introduced during recommit * re-pulled dev branch, added basic mount tests component * added functional tests for mount, double mount, and permanent configs * pep8 corrections from ansible-sanity * return value edits * removed 'persistent' from returns because it was tripping up sanity * corrected documentation on comments I missed previously. * pep8 fixes...found vscode was adding spaces when moving lines around. * Reponse changes to December review. No new tests yet. * corrected bad merge. * remerge * Added newer licensing info at top. * updated zos_mount to use 2021 header minor test tweak merged current dev into mount * changing includes and renaming persistent_ds to handle sub-vars * working through pep 8 and reducing imports * Copied backupOper routine because of import issues. * Figured out how to return a sub-dictionary in RETURN doc section * added two detail lines in the routine that is closest to the JSON error * pep8 issues in the test module * changed zos_test to use the upper of the first component of sysname ...interesting that it returns differently when called manually * added to test import funnctionality to see if DataSet class impacts data_set implementation * Moving imports around to get data_set functions working * commenting out one of the routines firing data_set entry, to see if the error moves * change a few os calls to hosts.all.os removed to assertions that couldn't complete because of scope * increased mount space allocation slightly really, previous run was in a bad initial condition * Added missing spaces in zos_mount/unmount function Changed test cleanups from hosts.all.os to hosts.all.shell * argh: black formatter adds a space before a colon (zos_mount: 618) content_lines[startidx : endidx + 1] will trigger a sanity fault * in zos_mount, a discription in the doc was confusing in test_zos_mount_func, 2 values were not passed during an unmount that were mandatory * changed from shell/cp to zos_copy, so that pds members could be created * corrected over-indent in test * test routine was using 2 different filenames * Changed test to fix a bad removal step, and to comment out the tmp_file removal * changed implementation of populate_tmpfile to see if I can get it to stick around * changing tempfile to open 'w' instead of 'w+b' * hard-coding tmpfile creation to see if that lets the file exist * changed copy to shell/cp * adding trace/track messages (stonehenge style), and escaping a DS/member name's parentheses * resolving pep8 issues * zos_data_set call had extransous character in it * correcting print statement * changed testfile generator to be target-local * bad block comment * switching back to zos_copy, now that the tmp file is on the target * fixed block comment * changed comments in test to tabcomment, which is what it is defined as in zos_mount * added tracking on comment tester for last of test chain * space after paren in a print statement * corrected issue where backup_name was not being loaded from persistent data section * changed backup_name mechanism to use mt_backupOper convention * added remote_src=True for copy-back, so the file should get copied (250 of mount test) * changed zos_copy calls to include is_binary=True...trying to reconcile encoding * changing to using outbound temp file on ansible server, instead of target * adding cat of original test_tmp_file_filename to check encoding. * back up step to check copy origin * tryping to use copy_p2uss and uss2mvs instead of zos_copy for testing * back to cp commands: zos_copy adds a 'F', and the copy functions won't import to test * added missing spaces to the cp commands * trying hosts.all.copy to move the tmp file to the target systems * propagated 2-step copy to comment routine un-commented the file deleters * added more output in bcb * space after paren in new print statements * trying copy...content instead of copy...src * removed an extra blank line * changed the file's flag after it landed chtag -t -c ISO8859-1 (destfile) * it looks like chtag is making the file readable... trying for the third display * commented out the dataset assertion until I find the setting to make data_set work * re-introducing zos_copy to see if the sole issue was the file flag * propagated change back to zos_copy + chtag to the other cases. Retesting before moving to the data_set issue * added remote_src=True to the zos_copy * trying remote_src=true for copy in bcb, since it worked in bnn * Added data set removal at the finally stage of tests. Removed comment about zos_copy in the mount module. * In test, removed the -volumes argument * put volumes option back in * changed test aggregate name because of catalog glitch. * 2-10: removed redundant entry for zos_mount 2-11 & 2-12: added an entry for zos_mount * Changed hard-coded "IMSCN1" to "222222", which should usually be a temp volume. * Reponse to latest code reviews. Corrected documentation, variable naming. Added remount test. * added second blank link @141 for p3p8 requirement in test * Updates to address comments in git PR Signed-off-by: ddimatos <[email protected]> * Addressed PR comments regarding error messages needing context. Changed from run_command(tsocommand) to mvs_cmd.ikjeft01 calls for compatibility. * corrections to 2 issues that failed sanity test * corrected a bad default condition. We changed case of mount_opts, but I forgot to uppercase the default. * restored comment about mounting a dataset that is already mounted. * corrected a test looking in old variable. * corrected issue with stderr being blanked out after it was populated. * changed commands to authorized=True * reverted calls to (rc, stdout, stderr) = module.run_command( fullumcmd, use_unsafe_shell=False ) because of error messages in mvs_cmd * changed behavior so if rc=0, any text returned goes to stdout. * needed to handle None return in stdout * modified test (double mount) to check stderr and stdout * modified tests to use persistant/data_store i/s/o data_set_name * corrected parameter error (dataa instead of data) * changed 'tabcomment' to 'comment' in test * updates to zos_mount: moved 'comments' to under persistant grouping corrected test_zos_mount_func to use the new comment format fixed text processor for comments to remove nulls embedded in file * Pushed these changes as a commit for @RadhaV2020 Signed-off-by: ddimatos <[email protected]> * Update copyright to 2021 for files in role samples Signed-off-by: ddimatos <[email protected]> * Updated copyright year to 2021 for files changed in the respective year for GID RadhaV2020 Signed-off-by: ddimatos <[email protected]> * changed test_zos_find_func, removing a test based on pre-existance of files. Co-authored-by: ddimatos <[email protected]> * Update common code used to obtain job content such it is not trucated by the unrelated record count (#283) * Update common code used to obtain job content such it is not trucated by the unrelated record count Signed-off-by: ddimatos <[email protected]> * Removed unused linecount variable Signed-off-by: ddimatos <[email protected]> * Deprecate zos_ssh connection plugin and update zos_ping (#287) * Update zos_ping module to support Ansible 2_9 - 2_11 Signed-off-by: ddimatos <[email protected]> * Add copyright year update Signed-off-by: ddimatos <[email protected]> * Remove doc referring to the deprecated zos_ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Remove the unsued connectionp plugin since the logic is now in zos_ping Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 line to long error Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 continuation line under-indented Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 continuation line under-indented Signed-off-by: ddimatos <[email protected]> * Remove zos_ssh plugin from test configuration to keep from failing tests Signed-off-by: ddimatos <[email protected]> * Remove zos_ssh plugin references from docs since the plugin is deprecated Signed-off-by: ddimatos <[email protected]> * Fix closing quote Signed-off-by: ddimatos <[email protected]> * Updated the ping module code with some logic change on variable use and how changes are restored Signed-off-by: ddimatos <[email protected]> * Correct indentation Signed-off-by: ddimatos <[email protected]> * Temporary work around for failing test test_find_data_sets_in_volume where its expecting 5 DS yet only 1 is cataloged Signed-off-by: ddimatos <[email protected]> * Update zos_copy and zos_fetch to use the SSH conneciton plugin APIs, eliminate spawned SFTP process. (#286) * Update zos_copy module changes to support use of ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Update zos_fetch module changes to support use of ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Add force to module spec to correct module lint error during runtime and fix changed returning null Signed-off-by: ddimatos <[email protected]> * Update copy module per arg spec and sanit test ignore files Signed-off-by: ddimatos <[email protected]> * Update test case to correctly compare changed files Signed-off-by: ddimatos <[email protected]> * Update test case to correctly compare changed files Signed-off-by: ddimatos <[email protected]> * Add stderr to the error case Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Print out steps of a failing test case * Syntax error in test * Erase the ssh_args Signed-off-by: ddimatos <[email protected]> * Null out ssh_args * Add debug to config * Adding verbosity to debug docker * Adding verbosity to debug docker * Reduce verbosity to avoid ssh connection setting up verbosity which causes stderr to be output * Updates to modules verbosity messages and cleanup Signed-off-by: ddimatos <[email protected]> * Correct syntax for default port Signed-off-by: ddimatos <[email protected]> * Add displaying of the playcontext verbosity Signed-off-by: ddimatos <[email protected]> * Added additional logic to handle the failure than can occur when users enabled verbosity > 3 Signed-off-by: ddimatos <[email protected]> * Updated copyright year to include 2021 Signed-off-by: ddimatos <[email protected]> * Set the ignore_stderr=True now that testing has completed Signed-off-by: ddimatos <[email protected]> * Update test case to remove uneeded logic Signed-off-by: ddimatos <[email protected]> * test case assertion back to True Signed-off-by: ddimatos <[email protected]> * Add documentation about co:z for copy and fetch Signed-off-by: ddimatos <[email protected]> * Update failing test case test_find_data_sets_in_volume to not rely on a hard coded number of data sets Signed-off-by: ddimatos <[email protected]> * Update plugin module doc with missing plugins and deprecated content (#288) * Update plugin modudle doc with missing plugins and deprecated content Signed-off-by: ddimatos <[email protected]> * Correct Unknown target name: zos_ping Signed-off-by: ddimatos <[email protected]> * Bug fix not propagating non zero return codes and under certain cases job_name is null for operator action query (#290) * Added on command failure return details , added fix for when no job_name is part of a query Signed-off-by: ddimatos <[email protected]> * Added comments and cleaned up some others Signed-off-by: ddimatos <[email protected]> * Update the failure command, typo and changed what is returned in the message field to stdout only Signed-off-by: ddimatos <[email protected]> * Update failing test case test_find_data_sets_in_volume so that it does not depend on hard coded value Signed-off-by: ddimatos <[email protected]> * Update galaxy yml to reflect the beta and other minor changes Signed-off-by: ddimatos <[email protected]> * Corrected doc that had random back slashes Signed-off-by: ddimatos <[email protected]> * Shorted short description to avoid navigation roll over Signed-off-by: ddimatos <[email protected]> * Update the release notes for version 1.4.0-beta.1 Signed-off-by: ddimatos <[email protected]> * Update ignore_sftp_stderr to simplify the explanation Signed-off-by: ddimatos <[email protected]> * Correct URL RST syntax Signed-off-by: ddimatos <[email protected]> * Fix character set URLs to the updated URL Signed-off-by: ddimatos <[email protected]> * Update modules to use L for URLs Signed-off-by: ddimatos <[email protected]> * Updated module docs to correct the sphinx improper interpretation of VASM followed by an open paren Signed-off-by: ddimatos <[email protected]> * New generated RST from the new zos_mount module Signed-off-by: ddimatos <[email protected]> * Add the updated generated RST to the release for use by unified docs and ansible-doc Signed-off-by: ddimatos <[email protected]> * Update sphinx copyright year to include 2021 Signed-off-by: ddimatos <[email protected]> * Update zos_mount module description to better reflect it does more than just mount Signed-off-by: ddimatos <[email protected]> * Add generated RST that has been updated Signed-off-by: ddimatos <[email protected]> * Module updates to fix linting issues Signed-off-by: ddimatos <[email protected]> * Correct typo in doc Signed-off-by: ddimatos <[email protected]> * commit generated RST for zos_job_query.rst Signed-off-by: ddimatos <[email protected]> * Update release notes with more enhancements Signed-off-by: ddimatos <[email protected]> * Update meta to match the beta release Signed-off-by: ddimatos <[email protected]> * Update release notes Signed-off-by: ddimatos <[email protected]> * Author updates to modules Signed-off-by: ddimatos <[email protected]> Co-authored-by: Rich Parker <[email protected]> Co-authored-by: RadhaV2020 <[email protected]>
ddimatos
added a commit
that referenced
this pull request
Jul 23, 2021
* Update to support job id up to 7 numeric positions (#259) Signed-off-by: ddimatos <[email protected]> * Bugfix/3687 (#251) * 3687: looking for 'JCL ERROR' in return code message, since that will not contain a "CC" value to return. Temporarily assigning that a CC=16 * Expanded changes for JCL error to 'fail fast' + added "JCL ERROR" to job_completion_messages list + corrected grammar in error message@615 (was 'slow to response') * rebuilt timing delay loops for zos_job_submit * fixed variable usage, added more meaningful error for JCL ERROR and ABEND detectors * simplified jcl error search string corrected logic in test (testing for failure) * added debug call/conv strings to return from job_submit, to understand blank returns * argh...typo in the tracking string * continuing to track non-response issue * corrected issue with temp_file.name (may need to add comment that it is a string, not an object) * eliminated duplicate temp file 2 removal * adding badjcl indicator, since the glitch moved * added another fail_json to job_submit * removed failed=true as an assertion * removed tracking statements. * minor corrections on comments... still need to discuss return code and no-wait action changes * changed job to return msg_code=None on JCL error updated documentation of return to note this * Changed documentation to help clarify the wait/wait_time_s issue Fixed another all-caps error message Added JobId info to long-running meessage note. * forgot to switch back to black editor...autopep8 moved the import statements argh! * Added discovered customer issue into description for the 'cmd' parame… (#265) * Added discovered customer issue into description for the 'cmd' parameter. * Update zos_operator.py Minor text edits * removed colon from 'for example', which tripped a sanity test in documentation Co-authored-by: RadhaV2020 <[email protected]> * Bugfix/5950 (#260) * bugfix 5950 (exception thrown when a job info didn't find a matching job) New behavior: job.py returns a single record populated with "NOTFOUND" output, query, and submit functions had a small change to document that a single notfound could be returned test feature added to look for job_name=NOTAJOB to see if the notfound entry is returned * minor change removing a print statement from job.py * added missing blank line between functions (pep8) * corrected two outdated comments, and verified 'not found' code is in place. * standardized on NOTFOUND, tweaked stepname. fixed the 'no jobs found' reaction. * correction on adding a 'not found' message...needed to pass blanks in one case. * corrected issue with job value assignment in 'jobs not found' corrected grammatical typo in error message in submit * changes to get documentation more consistent, and not-found response more system complient * corrected test_job_output to look for the new 'not found' text * Changed the not found response to give None responses on system/subsystem Documentation was general enough to work with this Changed unit test to look for the ret_code/msg value * job.py didn't save, so I had to re-save and re-push * typo in tester: typed "Jobs" instead of "jobs". argh! * Changed return code to JOB NOT FOUND on empty record, to match other response messages. Changed JCLERROR return code to mixed case, for readability. Corrected documentation entries that needed to match code changes. Changed output test to match code changes. * Update zos_mvs_raw.py (#279) Removed the duplicate "name" entry from the examples. Defect fix for https://jsw.ibm.com/browse/NAZARE-6112. * Enhancement/3723 (#256) * added '_parse_stepwise' to job_output .. this should add job["ret_code"]["stepwise"][<step_name>] = <last field of CC string> to the job entries .. assuming that the full response is available when the data is pulled * whoopsie: pep8 issue on a regex string * .. expanded documentation to show 'steps' return in the 3 impacted functions .. added/expanded a test on zos_job_output to make sure steps were being returned * Adjusting return definition for yaml compliance Correcting typo in 'sample' return, that wasn't encapsulated Added 1 more layer to output function test * added pretty print to uss_file_to_mvs_pds_member to see what is happening there * added more output around new sibmit result test * issue moved, so more output messages * added directory list to exception message...temporary * listdir not listdirs... argh * pep8 issue * correction because black formatter couldn't handle [0: * removing tmp literal * simplified vars() to print(result) * changed a dictionary (typo) back to a list, so I can append to it. * more test tweaking, on test zos_job_output * corrected test to look for steps in ret_code, not in main return object * corrected path import * changed a test in zos_data_set, which doesn't account for the response on a pre-existing file. * added tracking output to test_uss_encoding_conversion_mvs_ps_to_mvs_vsam (looks like a case where submit may not return a job list?) * copied 5950 changes to (job.py) into this branch. changed 'stepid' to 'step_name' and 'stepcc' to 'step_name' (snake?). updated documentation in submit and output commands. * turns out ansible sanity checks flake-8 as well as pep-8. Space before semicolon line 305 of encode. * Changed unit test to use new 'step_name' instead of 'stepid'. * Merged job.py 5950 into this ticket. Added 'JOB NOT FOUND' into doc message in job.py. Corrected description of 'abend' example in all 3 zos_job modules. (be instead of better). * Corrected documentation to match field names from job.py. Swapped "The" to replace "Actual" because of implications. * Copy6113 (#282) * Initial proposal to have copy create data set when copying uss file to non-existant data set. Code is there, documentation is there, but I haven't done sanity run yet. * Removal of model_ds from copy function and sub-functions Now through ansible-sanity. This should address Jira 6183 as well as 6113 Note: 2 minor changes to zos_fetch based on a sanity test issue there with 128/130 strings * Changed documentation to set force default to true, which is implemented in action * Changes based on demetri's testing: 1 - Case correction for backup_name member 2 - explicitly generating member name, instead of depending on dcp to generate a value * tried to remove extra blank line on 1905 * Changes based on pull request comments. * Pull request updates to doc Signed-off-by: ddimatos <[email protected]> * partial change moving options to suboptions - not stable * Local *3.8) sanity passing on changed indents and size changes. Need to do playbook testing. * corrected zos_copy to auto-populate destination_dataset if it is not provided * removed a find_test that looked for files it had not created. * corrected suboptions in dictionary definition * bugfixes from playbook: copy to pds was allowing invalid filenames (period in member, member>char, lower member) rebuild workbook tests to make sure these tested okay Co-authored-by: ddimatos <[email protected]> * Feature zos_mount module added (#280) * Initial work on zos_mount. This is NOT complete. Making sure push goes to the right place. * Initial commit of git_mount: needs unit tests * Basic mount/unmount testing passed manually * made interface more ansible-like cleaned up formatting saving before new test runs * passed PEP8 testing in ansible-sanity... still working on documentation * All p3p8 resolved. Down to 6 validte-module issues. * final series of ansible-test issues resolved locally moving to server-level testing * added mount to correct directory * removed zos_mount from wrong folder * GPL3 header correction * Update zos_mount.py Added second line between metadata and documentation, to see if that addresses the license issue * new parameters for mount, aliases for mount, updated ignore files * minor tweak to ignore file * corrected string search and use of tmp directory * corrected code for pep8 provisions * pulled dev back in for code updates. Touched ignore files. * Responded to Blake and Nehnam's recommendations, except: 1 - Still not sure if cp->zos_copy change will help 2 - Need to add unit tests 3 - Need to run my test playbooks again * responding to Blake and Behnam's ssuggestions * code changes per Blake and Behnam's suggestions Still looking at zos_copy Still need to build unit tests * simplified df to search with python for safe shell run... re-tested mount idempotence Need to test multiline comments for bpxprm modifier * corrections for multiline comments. All manually tested. first pass at function test entries. * removed zos_mount functional test until I can get it to pass locally. * forcing rebuild * corrected pep8 errors introduced during recommit * re-pulled dev branch, added basic mount tests component * added functional tests for mount, double mount, and permanent configs * pep8 corrections from ansible-sanity * return value edits * removed 'persistent' from returns because it was tripping up sanity * corrected documentation on comments I missed previously. * pep8 fixes...found vscode was adding spaces when moving lines around. * Reponse changes to December review. No new tests yet. * corrected bad merge. * remerge * Added newer licensing info at top. * updated zos_mount to use 2021 header minor test tweak merged current dev into mount * changing includes and renaming persistent_ds to handle sub-vars * working through pep 8 and reducing imports * Copied backupOper routine because of import issues. * Figured out how to return a sub-dictionary in RETURN doc section * added two detail lines in the routine that is closest to the JSON error * pep8 issues in the test module * changed zos_test to use the upper of the first component of sysname ...interesting that it returns differently when called manually * added to test import funnctionality to see if DataSet class impacts data_set implementation * Moving imports around to get data_set functions working * commenting out one of the routines firing data_set entry, to see if the error moves * change a few os calls to hosts.all.os removed to assertions that couldn't complete because of scope * increased mount space allocation slightly really, previous run was in a bad initial condition * Added missing spaces in zos_mount/unmount function Changed test cleanups from hosts.all.os to hosts.all.shell * argh: black formatter adds a space before a colon (zos_mount: 618) content_lines[startidx : endidx + 1] will trigger a sanity fault * in zos_mount, a discription in the doc was confusing in test_zos_mount_func, 2 values were not passed during an unmount that were mandatory * changed from shell/cp to zos_copy, so that pds members could be created * corrected over-indent in test * test routine was using 2 different filenames * Changed test to fix a bad removal step, and to comment out the tmp_file removal * changed implementation of populate_tmpfile to see if I can get it to stick around * changing tempfile to open 'w' instead of 'w+b' * hard-coding tmpfile creation to see if that lets the file exist * changed copy to shell/cp * adding trace/track messages (stonehenge style), and escaping a DS/member name's parentheses * resolving pep8 issues * zos_data_set call had extransous character in it * correcting print statement * changed testfile generator to be target-local * bad block comment * switching back to zos_copy, now that the tmp file is on the target * fixed block comment * changed comments in test to tabcomment, which is what it is defined as in zos_mount * added tracking on comment tester for last of test chain * space after paren in a print statement * corrected issue where backup_name was not being loaded from persistent data section * changed backup_name mechanism to use mt_backupOper convention * added remote_src=True for copy-back, so the file should get copied (250 of mount test) * changed zos_copy calls to include is_binary=True...trying to reconcile encoding * changing to using outbound temp file on ansible server, instead of target * adding cat of original test_tmp_file_filename to check encoding. * back up step to check copy origin * tryping to use copy_p2uss and uss2mvs instead of zos_copy for testing * back to cp commands: zos_copy adds a 'F', and the copy functions won't import to test * added missing spaces to the cp commands * trying hosts.all.copy to move the tmp file to the target systems * propagated 2-step copy to comment routine un-commented the file deleters * added more output in bcb * space after paren in new print statements * trying copy...content instead of copy...src * removed an extra blank line * changed the file's flag after it landed chtag -t -c ISO8859-1 (destfile) * it looks like chtag is making the file readable... trying for the third display * commented out the dataset assertion until I find the setting to make data_set work * re-introducing zos_copy to see if the sole issue was the file flag * propagated change back to zos_copy + chtag to the other cases. Retesting before moving to the data_set issue * added remote_src=True to the zos_copy * trying remote_src=true for copy in bcb, since it worked in bnn * Added data set removal at the finally stage of tests. Removed comment about zos_copy in the mount module. * In test, removed the -volumes argument * put volumes option back in * changed test aggregate name because of catalog glitch. * 2-10: removed redundant entry for zos_mount 2-11 & 2-12: added an entry for zos_mount * Changed hard-coded "IMSCN1" to "222222", which should usually be a temp volume. * Reponse to latest code reviews. Corrected documentation, variable naming. Added remount test. * added second blank link @141 for p3p8 requirement in test * Updates to address comments in git PR Signed-off-by: ddimatos <[email protected]> * Addressed PR comments regarding error messages needing context. Changed from run_command(tsocommand) to mvs_cmd.ikjeft01 calls for compatibility. * corrections to 2 issues that failed sanity test * corrected a bad default condition. We changed case of mount_opts, but I forgot to uppercase the default. * restored comment about mounting a dataset that is already mounted. * corrected a test looking in old variable. * corrected issue with stderr being blanked out after it was populated. * changed commands to authorized=True * reverted calls to (rc, stdout, stderr) = module.run_command( fullumcmd, use_unsafe_shell=False ) because of error messages in mvs_cmd * changed behavior so if rc=0, any text returned goes to stdout. * needed to handle None return in stdout * modified test (double mount) to check stderr and stdout * modified tests to use persistant/data_store i/s/o data_set_name * corrected parameter error (dataa instead of data) * changed 'tabcomment' to 'comment' in test * updates to zos_mount: moved 'comments' to under persistant grouping corrected test_zos_mount_func to use the new comment format fixed text processor for comments to remove nulls embedded in file * Pushed these changes as a commit for @RadhaV2020 Signed-off-by: ddimatos <[email protected]> * Update copyright to 2021 for files in role samples Signed-off-by: ddimatos <[email protected]> * Updated copyright year to 2021 for files changed in the respective year for GID RadhaV2020 Signed-off-by: ddimatos <[email protected]> * changed test_zos_find_func, removing a test based on pre-existance of files. Co-authored-by: ddimatos <[email protected]> * Update common code used to obtain job content such it is not trucated by the unrelated record count (#283) * Update common code used to obtain job content such it is not trucated by the unrelated record count Signed-off-by: ddimatos <[email protected]> * Removed unused linecount variable Signed-off-by: ddimatos <[email protected]> * Deprecate zos_ssh connection plugin and update zos_ping (#287) * Update zos_ping module to support Ansible 2_9 - 2_11 Signed-off-by: ddimatos <[email protected]> * Add copyright year update Signed-off-by: ddimatos <[email protected]> * Remove doc referring to the deprecated zos_ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Remove the unsued connectionp plugin since the logic is now in zos_ping Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 line to long error Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 continuation line under-indented Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 continuation line under-indented Signed-off-by: ddimatos <[email protected]> * Remove zos_ssh plugin from test configuration to keep from failing tests Signed-off-by: ddimatos <[email protected]> * Remove zos_ssh plugin references from docs since the plugin is deprecated Signed-off-by: ddimatos <[email protected]> * Fix closing quote Signed-off-by: ddimatos <[email protected]> * Updated the ping module code with some logic change on variable use and how changes are restored Signed-off-by: ddimatos <[email protected]> * Correct indentation Signed-off-by: ddimatos <[email protected]> * Temporary work around for failing test test_find_data_sets_in_volume where its expecting 5 DS yet only 1 is cataloged Signed-off-by: ddimatos <[email protected]> * Update zos_copy and zos_fetch to use the SSH conneciton plugin APIs, eliminate spawned SFTP process. (#286) * Update zos_copy module changes to support use of ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Update zos_fetch module changes to support use of ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Add force to module spec to correct module lint error during runtime and fix changed returning null Signed-off-by: ddimatos <[email protected]> * Update copy module per arg spec and sanit test ignore files Signed-off-by: ddimatos <[email protected]> * Update test case to correctly compare changed files Signed-off-by: ddimatos <[email protected]> * Update test case to correctly compare changed files Signed-off-by: ddimatos <[email protected]> * Add stderr to the error case Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Print out steps of a failing test case * Syntax error in test * Erase the ssh_args Signed-off-by: ddimatos <[email protected]> * Null out ssh_args * Add debug to config * Adding verbosity to debug docker * Adding verbosity to debug docker * Reduce verbosity to avoid ssh connection setting up verbosity which causes stderr to be output * Updates to modules verbosity messages and cleanup Signed-off-by: ddimatos <[email protected]> * Correct syntax for default port Signed-off-by: ddimatos <[email protected]> * Add displaying of the playcontext verbosity Signed-off-by: ddimatos <[email protected]> * Added additional logic to handle the failure than can occur when users enabled verbosity > 3 Signed-off-by: ddimatos <[email protected]> * Updated copyright year to include 2021 Signed-off-by: ddimatos <[email protected]> * Set the ignore_stderr=True now that testing has completed Signed-off-by: ddimatos <[email protected]> * Update test case to remove uneeded logic Signed-off-by: ddimatos <[email protected]> * test case assertion back to True Signed-off-by: ddimatos <[email protected]> * Add documentation about co:z for copy and fetch Signed-off-by: ddimatos <[email protected]> * Update failing test case test_find_data_sets_in_volume to not rely on a hard coded number of data sets Signed-off-by: ddimatos <[email protected]> * Update plugin module doc with missing plugins and deprecated content (#288) * Update plugin modudle doc with missing plugins and deprecated content Signed-off-by: ddimatos <[email protected]> * Correct Unknown target name: zos_ping Signed-off-by: ddimatos <[email protected]> * Bug fix not propagating non zero return codes and under certain cases job_name is null for operator action query (#290) * Added on command failure return details , added fix for when no job_name is part of a query Signed-off-by: ddimatos <[email protected]> * Added comments and cleaned up some others Signed-off-by: ddimatos <[email protected]> * Update the failure command, typo and changed what is returned in the message field to stdout only Signed-off-by: ddimatos <[email protected]> * Update failing test case test_find_data_sets_in_volume so that it does not depend on hard coded value Signed-off-by: ddimatos <[email protected]> * Update galaxy yml to reflect the beta and other minor changes Signed-off-by: ddimatos <[email protected]> * Corrected doc that had random back slashes Signed-off-by: ddimatos <[email protected]> * Shorted short description to avoid navigation roll over Signed-off-by: ddimatos <[email protected]> * Update the release notes for version 1.4.0-beta.1 Signed-off-by: ddimatos <[email protected]> * Update ignore_sftp_stderr to simplify the explanation Signed-off-by: ddimatos <[email protected]> * Correct URL RST syntax Signed-off-by: ddimatos <[email protected]> * Fix character set URLs to the updated URL Signed-off-by: ddimatos <[email protected]> * Update modules to use L for URLs Signed-off-by: ddimatos <[email protected]> * Updated module docs to correct the sphinx improper interpretation of VASM followed by an open paren Signed-off-by: ddimatos <[email protected]> * New generated RST from the new zos_mount module Signed-off-by: ddimatos <[email protected]> * Add the updated generated RST to the release for use by unified docs and ansible-doc Signed-off-by: ddimatos <[email protected]> * Update sphinx copyright year to include 2021 Signed-off-by: ddimatos <[email protected]> * Update zos_mount module description to better reflect it does more than just mount Signed-off-by: ddimatos <[email protected]> * Add generated RST that has been updated Signed-off-by: ddimatos <[email protected]> * Module updates to fix linting issues Signed-off-by: ddimatos <[email protected]> * Correct typo in doc Signed-off-by: ddimatos <[email protected]> * commit generated RST for zos_job_query.rst Signed-off-by: ddimatos <[email protected]> * Update release notes with more enhancements Signed-off-by: ddimatos <[email protected]> * Update meta to match the beta release Signed-off-by: ddimatos <[email protected]> * Update release notes Signed-off-by: ddimatos <[email protected]> * Author updates to modules Signed-off-by: ddimatos <[email protected]> * Add the required runtime.yml Signed-off-by: ddimatos <[email protected]> Co-authored-by: Rich Parker <[email protected]> Co-authored-by: RadhaV2020 <[email protected]>
ddimatos
added a commit
that referenced
this pull request
Oct 4, 2022
* Update to support job id up to 7 numeric positions (#259) Signed-off-by: ddimatos <[email protected]> * Bugfix/3687 (#251) * 3687: looking for 'JCL ERROR' in return code message, since that will not contain a "CC" value to return. Temporarily assigning that a CC=16 * Expanded changes for JCL error to 'fail fast' + added "JCL ERROR" to job_completion_messages list + corrected grammar in error message@615 (was 'slow to response') * rebuilt timing delay loops for zos_job_submit * fixed variable usage, added more meaningful error for JCL ERROR and ABEND detectors * simplified jcl error search string corrected logic in test (testing for failure) * added debug call/conv strings to return from job_submit, to understand blank returns * argh...typo in the tracking string * continuing to track non-response issue * corrected issue with temp_file.name (may need to add comment that it is a string, not an object) * eliminated duplicate temp file 2 removal * adding badjcl indicator, since the glitch moved * added another fail_json to job_submit * removed failed=true as an assertion * removed tracking statements. * minor corrections on comments... still need to discuss return code and no-wait action changes * changed job to return msg_code=None on JCL error updated documentation of return to note this * Changed documentation to help clarify the wait/wait_time_s issue Fixed another all-caps error message Added JobId info to long-running meessage note. * forgot to switch back to black editor...autopep8 moved the import statements argh! * Added discovered customer issue into description for the 'cmd' parame… (#265) * Added discovered customer issue into description for the 'cmd' parameter. * Update zos_operator.py Minor text edits * removed colon from 'for example', which tripped a sanity test in documentation Co-authored-by: RadhaV2020 <[email protected]> * Bugfix/5950 (#260) * bugfix 5950 (exception thrown when a job info didn't find a matching job) New behavior: job.py returns a single record populated with "NOTFOUND" output, query, and submit functions had a small change to document that a single notfound could be returned test feature added to look for job_name=NOTAJOB to see if the notfound entry is returned * minor change removing a print statement from job.py * added missing blank line between functions (pep8) * corrected two outdated comments, and verified 'not found' code is in place. * standardized on NOTFOUND, tweaked stepname. fixed the 'no jobs found' reaction. * correction on adding a 'not found' message...needed to pass blanks in one case. * corrected issue with job value assignment in 'jobs not found' corrected grammatical typo in error message in submit * changes to get documentation more consistent, and not-found response more system complient * corrected test_job_output to look for the new 'not found' text * Changed the not found response to give None responses on system/subsystem Documentation was general enough to work with this Changed unit test to look for the ret_code/msg value * job.py didn't save, so I had to re-save and re-push * typo in tester: typed "Jobs" instead of "jobs". argh! * Changed return code to JOB NOT FOUND on empty record, to match other response messages. Changed JCLERROR return code to mixed case, for readability. Corrected documentation entries that needed to match code changes. Changed output test to match code changes. * Update zos_mvs_raw.py (#279) Removed the duplicate "name" entry from the examples. Defect fix for https://jsw.ibm.com/browse/NAZARE-6112. * Enhancement/3723 (#256) * added '_parse_stepwise' to job_output .. this should add job["ret_code"]["stepwise"][<step_name>] = <last field of CC string> to the job entries .. assuming that the full response is available when the data is pulled * whoopsie: pep8 issue on a regex string * .. expanded documentation to show 'steps' return in the 3 impacted functions .. added/expanded a test on zos_job_output to make sure steps were being returned * Adjusting return definition for yaml compliance Correcting typo in 'sample' return, that wasn't encapsulated Added 1 more layer to output function test * added pretty print to uss_file_to_mvs_pds_member to see what is happening there * added more output around new sibmit result test * issue moved, so more output messages * added directory list to exception message...temporary * listdir not listdirs... argh * pep8 issue * correction because black formatter couldn't handle [0: * removing tmp literal * simplified vars() to print(result) * changed a dictionary (typo) back to a list, so I can append to it. * more test tweaking, on test zos_job_output * corrected test to look for steps in ret_code, not in main return object * corrected path import * changed a test in zos_data_set, which doesn't account for the response on a pre-existing file. * added tracking output to test_uss_encoding_conversion_mvs_ps_to_mvs_vsam (looks like a case where submit may not return a job list?) * copied 5950 changes to (job.py) into this branch. changed 'stepid' to 'step_name' and 'stepcc' to 'step_name' (snake?). updated documentation in submit and output commands. * turns out ansible sanity checks flake-8 as well as pep-8. Space before semicolon line 305 of encode. * Changed unit test to use new 'step_name' instead of 'stepid'. * Merged job.py 5950 into this ticket. Added 'JOB NOT FOUND' into doc message in job.py. Corrected description of 'abend' example in all 3 zos_job modules. (be instead of better). * Corrected documentation to match field names from job.py. Swapped "The" to replace "Actual" because of implications. * Copy6113 (#282) * Initial proposal to have copy create data set when copying uss file to non-existant data set. Code is there, documentation is there, but I haven't done sanity run yet. * Removal of model_ds from copy function and sub-functions Now through ansible-sanity. This should address Jira 6183 as well as 6113 Note: 2 minor changes to zos_fetch based on a sanity test issue there with 128/130 strings * Changed documentation to set force default to true, which is implemented in action * Changes based on demetri's testing: 1 - Case correction for backup_name member 2 - explicitly generating member name, instead of depending on dcp to generate a value * tried to remove extra blank line on 1905 * Changes based on pull request comments. * Pull request updates to doc Signed-off-by: ddimatos <[email protected]> * partial change moving options to suboptions - not stable * Local *3.8) sanity passing on changed indents and size changes. Need to do playbook testing. * corrected zos_copy to auto-populate destination_dataset if it is not provided * removed a find_test that looked for files it had not created. * corrected suboptions in dictionary definition * bugfixes from playbook: copy to pds was allowing invalid filenames (period in member, member>char, lower member) rebuild workbook tests to make sure these tested okay Co-authored-by: ddimatos <[email protected]> * Feature zos_mount module added (#280) * Initial work on zos_mount. This is NOT complete. Making sure push goes to the right place. * Initial commit of git_mount: needs unit tests * Basic mount/unmount testing passed manually * made interface more ansible-like cleaned up formatting saving before new test runs * passed PEP8 testing in ansible-sanity... still working on documentation * All p3p8 resolved. Down to 6 validte-module issues. * final series of ansible-test issues resolved locally moving to server-level testing * added mount to correct directory * removed zos_mount from wrong folder * GPL3 header correction * Update zos_mount.py Added second line between metadata and documentation, to see if that addresses the license issue * new parameters for mount, aliases for mount, updated ignore files * minor tweak to ignore file * corrected string search and use of tmp directory * corrected code for pep8 provisions * pulled dev back in for code updates. Touched ignore files. * Responded to Blake and Nehnam's recommendations, except: 1 - Still not sure if cp->zos_copy change will help 2 - Need to add unit tests 3 - Need to run my test playbooks again * responding to Blake and Behnam's ssuggestions * code changes per Blake and Behnam's suggestions Still looking at zos_copy Still need to build unit tests * simplified df to search with python for safe shell run... re-tested mount idempotence Need to test multiline comments for bpxprm modifier * corrections for multiline comments. All manually tested. first pass at function test entries. * removed zos_mount functional test until I can get it to pass locally. * forcing rebuild * corrected pep8 errors introduced during recommit * re-pulled dev branch, added basic mount tests component * added functional tests for mount, double mount, and permanent configs * pep8 corrections from ansible-sanity * return value edits * removed 'persistent' from returns because it was tripping up sanity * corrected documentation on comments I missed previously. * pep8 fixes...found vscode was adding spaces when moving lines around. * Reponse changes to December review. No new tests yet. * corrected bad merge. * remerge * Added newer licensing info at top. * updated zos_mount to use 2021 header minor test tweak merged current dev into mount * changing includes and renaming persistent_ds to handle sub-vars * working through pep 8 and reducing imports * Copied backupOper routine because of import issues. * Figured out how to return a sub-dictionary in RETURN doc section * added two detail lines in the routine that is closest to the JSON error * pep8 issues in the test module * changed zos_test to use the upper of the first component of sysname ...interesting that it returns differently when called manually * added to test import funnctionality to see if DataSet class impacts data_set implementation * Moving imports around to get data_set functions working * commenting out one of the routines firing data_set entry, to see if the error moves * change a few os calls to hosts.all.os removed to assertions that couldn't complete because of scope * increased mount space allocation slightly really, previous run was in a bad initial condition * Added missing spaces in zos_mount/unmount function Changed test cleanups from hosts.all.os to hosts.all.shell * argh: black formatter adds a space before a colon (zos_mount: 618) content_lines[startidx : endidx + 1] will trigger a sanity fault * in zos_mount, a discription in the doc was confusing in test_zos_mount_func, 2 values were not passed during an unmount that were mandatory * changed from shell/cp to zos_copy, so that pds members could be created * corrected over-indent in test * test routine was using 2 different filenames * Changed test to fix a bad removal step, and to comment out the tmp_file removal * changed implementation of populate_tmpfile to see if I can get it to stick around * changing tempfile to open 'w' instead of 'w+b' * hard-coding tmpfile creation to see if that lets the file exist * changed copy to shell/cp * adding trace/track messages (stonehenge style), and escaping a DS/member name's parentheses * resolving pep8 issues * zos_data_set call had extransous character in it * correcting print statement * changed testfile generator to be target-local * bad block comment * switching back to zos_copy, now that the tmp file is on the target * fixed block comment * changed comments in test to tabcomment, which is what it is defined as in zos_mount * added tracking on comment tester for last of test chain * space after paren in a print statement * corrected issue where backup_name was not being loaded from persistent data section * changed backup_name mechanism to use mt_backupOper convention * added remote_src=True for copy-back, so the file should get copied (250 of mount test) * changed zos_copy calls to include is_binary=True...trying to reconcile encoding * changing to using outbound temp file on ansible server, instead of target * adding cat of original test_tmp_file_filename to check encoding. * back up step to check copy origin * tryping to use copy_p2uss and uss2mvs instead of zos_copy for testing * back to cp commands: zos_copy adds a 'F', and the copy functions won't import to test * added missing spaces to the cp commands * trying hosts.all.copy to move the tmp file to the target systems * propagated 2-step copy to comment routine un-commented the file deleters * added more output in bcb * space after paren in new print statements * trying copy...content instead of copy...src * removed an extra blank line * changed the file's flag after it landed chtag -t -c ISO8859-1 (destfile) * it looks like chtag is making the file readable... trying for the third display * commented out the dataset assertion until I find the setting to make data_set work * re-introducing zos_copy to see if the sole issue was the file flag * propagated change back to zos_copy + chtag to the other cases. Retesting before moving to the data_set issue * added remote_src=True to the zos_copy * trying remote_src=true for copy in bcb, since it worked in bnn * Added data set removal at the finally stage of tests. Removed comment about zos_copy in the mount module. * In test, removed the -volumes argument * put volumes option back in * changed test aggregate name because of catalog glitch. * 2-10: removed redundant entry for zos_mount 2-11 & 2-12: added an entry for zos_mount * Changed hard-coded "IMSCN1" to "222222", which should usually be a temp volume. * Reponse to latest code reviews. Corrected documentation, variable naming. Added remount test. * added second blank link @141 for p3p8 requirement in test * Updates to address comments in git PR Signed-off-by: ddimatos <[email protected]> * Addressed PR comments regarding error messages needing context. Changed from run_command(tsocommand) to mvs_cmd.ikjeft01 calls for compatibility. * corrections to 2 issues that failed sanity test * corrected a bad default condition. We changed case of mount_opts, but I forgot to uppercase the default. * restored comment about mounting a dataset that is already mounted. * corrected a test looking in old variable. * corrected issue with stderr being blanked out after it was populated. * changed commands to authorized=True * reverted calls to (rc, stdout, stderr) = module.run_command( fullumcmd, use_unsafe_shell=False ) because of error messages in mvs_cmd * changed behavior so if rc=0, any text returned goes to stdout. * needed to handle None return in stdout * modified test (double mount) to check stderr and stdout * modified tests to use persistant/data_store i/s/o data_set_name * corrected parameter error (dataa instead of data) * changed 'tabcomment' to 'comment' in test * updates to zos_mount: moved 'comments' to under persistant grouping corrected test_zos_mount_func to use the new comment format fixed text processor for comments to remove nulls embedded in file * Pushed these changes as a commit for @RadhaV2020 Signed-off-by: ddimatos <[email protected]> * Update copyright to 2021 for files in role samples Signed-off-by: ddimatos <[email protected]> * Updated copyright year to 2021 for files changed in the respective year for GID RadhaV2020 Signed-off-by: ddimatos <[email protected]> * changed test_zos_find_func, removing a test based on pre-existance of files. Co-authored-by: ddimatos <[email protected]> * Update common code used to obtain job content such it is not trucated by the unrelated record count (#283) * Update common code used to obtain job content such it is not trucated by the unrelated record count Signed-off-by: ddimatos <[email protected]> * Removed unused linecount variable Signed-off-by: ddimatos <[email protected]> * Deprecate zos_ssh connection plugin and update zos_ping (#287) * Update zos_ping module to support Ansible 2_9 - 2_11 Signed-off-by: ddimatos <[email protected]> * Add copyright year update Signed-off-by: ddimatos <[email protected]> * Remove doc referring to the deprecated zos_ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Remove the unsued connectionp plugin since the logic is now in zos_ping Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 line to long error Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 continuation line under-indented Signed-off-by: ddimatos <[email protected]> * Fixes PEP8 continuation line under-indented Signed-off-by: ddimatos <[email protected]> * Remove zos_ssh plugin from test configuration to keep from failing tests Signed-off-by: ddimatos <[email protected]> * Remove zos_ssh plugin references from docs since the plugin is deprecated Signed-off-by: ddimatos <[email protected]> * Fix closing quote Signed-off-by: ddimatos <[email protected]> * Updated the ping module code with some logic change on variable use and how changes are restored Signed-off-by: ddimatos <[email protected]> * Correct indentation Signed-off-by: ddimatos <[email protected]> * Temporary work around for failing test test_find_data_sets_in_volume where its expecting 5 DS yet only 1 is cataloged Signed-off-by: ddimatos <[email protected]> * Update zos_copy and zos_fetch to use the SSH conneciton plugin APIs, eliminate spawned SFTP process. (#286) * Update zos_copy module changes to support use of ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Update zos_fetch module changes to support use of ssh connection plugin Signed-off-by: ddimatos <[email protected]> * Add force to module spec to correct module lint error during runtime and fix changed returning null Signed-off-by: ddimatos <[email protected]> * Update copy module per arg spec and sanit test ignore files Signed-off-by: ddimatos <[email protected]> * Update test case to correctly compare changed files Signed-off-by: ddimatos <[email protected]> * Update test case to correctly compare changed files Signed-off-by: ddimatos <[email protected]> * Add stderr to the error case Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Update test with more verbosity Signed-off-by: ddimatos <[email protected]> * Print out steps of a failing test case * Syntax error in test * Erase the ssh_args Signed-off-by: ddimatos <[email protected]> * Null out ssh_args * Add debug to config * Adding verbosity to debug docker * Adding verbosity to debug docker * Reduce verbosity to avoid ssh connection setting up verbosity which causes stderr to be output * Updates to modules verbosity messages and cleanup Signed-off-by: ddimatos <[email protected]> * Correct syntax for default port Signed-off-by: ddimatos <[email protected]> * Add displaying of the playcontext verbosity Signed-off-by: ddimatos <[email protected]> * Added additional logic to handle the failure than can occur when users enabled verbosity > 3 Signed-off-by: ddimatos <[email protected]> * Updated copyright year to include 2021 Signed-off-by: ddimatos <[email protected]> * Set the ignore_stderr=True now that testing has completed Signed-off-by: ddimatos <[email protected]> * Update test case to remove uneeded logic Signed-off-by: ddimatos <[email protected]> * test case assertion back to True Signed-off-by: ddimatos <[email protected]> * Add documentation about co:z for copy and fetch Signed-off-by: ddimatos <[email protected]> * Update failing test case test_find_data_sets_in_volume to not rely on a hard coded number of data sets Signed-off-by: ddimatos <[email protected]> * Update plugin module doc with missing plugins and deprecated content (#288) * Update plugin modudle doc with missing plugins and deprecated content Signed-off-by: ddimatos <[email protected]> * Correct Unknown target name: zos_ping Signed-off-by: ddimatos <[email protected]> * Bug fix not propagating non zero return codes and under certain cases job_name is null for operator action query (#290) * Added on command failure return details , added fix for when no job_name is part of a query Signed-off-by: ddimatos <[email protected]> * Added comments and cleaned up some others Signed-off-by: ddimatos <[email protected]> * Update the failure command, typo and changed what is returned in the message field to stdout only Signed-off-by: ddimatos <[email protected]> * Update failing test case test_find_data_sets_in_volume so that it does not depend on hard coded value Signed-off-by: ddimatos <[email protected]> * Update galaxy yml to reflect the beta and other minor changes Signed-off-by: ddimatos <[email protected]> * Corrected doc that had random back slashes Signed-off-by: ddimatos <[email protected]> * Shorted short description to avoid navigation roll over Signed-off-by: ddimatos <[email protected]> * Update the release notes for version 1.4.0-beta.1 Signed-off-by: ddimatos <[email protected]> * Update ignore_sftp_stderr to simplify the explanation Signed-off-by: ddimatos <[email protected]> * Correct URL RST syntax Signed-off-by: ddimatos <[email protected]> * Fix character set URLs to the updated URL Signed-off-by: ddimatos <[email protected]> * Update modules to use L for URLs Signed-off-by: ddimatos <[email protected]> * Updated module docs to correct the sphinx improper interpretation of VASM followed by an open paren Signed-off-by: ddimatos <[email protected]> * New generated RST from the new zos_mount module Signed-off-by: ddimatos <[email protected]> * Add the updated generated RST to the release for use by unified docs and ansible-doc Signed-off-by: ddimatos <[email protected]> * Update sphinx copyright year to include 2021 Signed-off-by: ddimatos <[email protected]> * Update zos_mount module description to better reflect it does more than just mount Signed-off-by: ddimatos <[email protected]> * Add generated RST that has been updated Signed-off-by: ddimatos <[email protected]> * Module updates to fix linting issues Signed-off-by: ddimatos <[email protected]> * Correct typo in doc Signed-off-by: ddimatos <[email protected]> * commit generated RST for zos_job_query.rst Signed-off-by: ddimatos <[email protected]> * Update release notes with more enhancements Signed-off-by: ddimatos <[email protected]> * Update meta to match the beta release Signed-off-by: ddimatos <[email protected]> * Update release notes Signed-off-by: ddimatos <[email protected]> * Author updates to modules Signed-off-by: ddimatos <[email protected]> * Add the required runtime.yml Signed-off-by: ddimatos <[email protected]> * Copy ignore-2.12.txt to ignore-2.13.txt * Update galaxy author list Signed-off-by: ddimatos <[email protected]> * Add logic to ensure the module only uses SFTP for sftp_transfer_method so that encodings are maintained as desired Signed-off-by: ddimatos <[email protected]> * Add logic to ensure the module only uses SFTP for sftp_transfer_method so that encodings are maintained as desired Signed-off-by: ddimatos <[email protected]> * Update/copy fetch scp bypass (#305) * Update galaxy author list Signed-off-by: ddimatos <[email protected]> * Add logic to ensure the module only uses SFTP for sftp_transfer_method so that encodings are maintained as desired Signed-off-by: ddimatos <[email protected]> * Add logic to ensure the module only uses SFTP for sftp_transfer_method so that encodings are maintained as desired Signed-off-by: ddimatos <[email protected]> * Bugfix zos_copy user specified data set failure and user specified volume Signed-off-by: ddimatos <[email protected]> * Update data set create with a return return code which could be useful for legacy logic Signed-off-by: ddimatos <[email protected]> * Update meta with a second version of the beta, although it may not be released it can serve as a fixt test. Signed-off-by: ddimatos <[email protected]> * PEP8 corrections Signed-off-by: ddimatos <[email protected]> * Add type PS back to set for mvs_seq check Signed-off-by: ddimatos <[email protected]> * Add check if to only create PDSE when it does not exist Signed-off-by: ddimatos <[email protected]> * Work in progress to modify module Signed-off-by: ddimatos <[email protected]> * Update comments Signed-off-by: ddimatos <[email protected]> * Fixed module to be able to copy into USS * Refactored precedence rules for sequential datasets * Refactored precedence rules for partitioned datasets Still have to implement all the functions to properly allocate datasets on like half of use cases, but this is a checkpoint to see if the rules I proposed would handle every case that the module needs to handle. Most of the refactoring was done by assuming that the function is_compatible works correctly and ignoring a bunch of possible combinations between source and destination types. For example, a sequential destination dataset can only be used when copying a single file, another sequential dataset or a single member from a PDSE. * Added allocation of sequential datasets The module can now allocate and replace destination sequential datasets by taking attributes from the source or setting sensible defaults. * Added allocation of partitioned datasets * Added handling of VSAMs - Added allocation of target VSAMs and simplified the CopyHandler method to copy between types of VSAMs. - Added exceptions to handle errors during allocation of destination data sets - Purged unused methods from the module * Moved allocation to its own function * Simplified how members are copied into a PDSE * Updated tests for copying USS files * Updated tests for sequential data sets * Updated test cases for VSAMs * Updated partitioned data sets test cases * Updated tests for USS dirs and data sets * Fixed linting issues for zos_copy * Changed system calls and cleanup * Updated sanity tests * Simplified use of destination_dataset And also added code to not erase a remote source when an encoding conversion fails. * Added restoration of dest when copy fails * Fixed argument_spec pylint issues * Fixed pep8 and pylint issues detected by Jenkins pipeline * Fixed sanity tests issues from pipeline * More sanity tests fixes Reintroducing the ignore for the shebang test on tests/dependencyfinder.py for Ansible 2.9. As this file gets run by the pipeline, it needs the shebang, but will cause problems with sanity tests. * Changed temp_path for data set member dump Issue raised by Bandit in the pipeline. * Tweaked docs and use of dest_data_set * Removed frozensets from module and dest_exists from handlers * Changed creation and copy of binary data sets * Fixed wrong permissions when copying a directory There was a bug when using the mode option while copying a directory where the parent directory of the destination would end up having the permissions given by the user. This fix makes sure that the module doesn't use the root of the temp directory created to upload local files, resulting in the path returned by copytree (which is used to change permissions) to be the correct directory instead of the parent. A side effect of this change is that whenever a local directory gets copied into a remote system, the module doesn't create another subdirectory in the destination, instead copying the contents into the root of the destination; keep in mind that this is the behavior when copying a remote source. * Updated module documentation * Added latest dependency finder * Fixed test for dest_data_set * Updated other failing test cases * Updated tests for zos_apf * Changed mode behavior to include files and subdirectories * Update doc with some added precedence rules clarity Signed-off-by: ddimatos <[email protected]> * Update release notes rst for authoring Signed-off-by: ddimatos <[email protected]> * add changelog content ready for authoring Signed-off-by: ddimatos <[email protected]> * Fixed wrong parent directory when checking existing members * Updated RST doc source Signed-off-by: ddimatos <[email protected]> * Added preservation of permissions to modified files inside a dir * Correct merge issue and update docs Signed-off-by: ddimatos <[email protected]> * Upadate to satisfy linting Signed-off-by: ddimatos <[email protected]> * update test case with force negative Signed-off-by: ddimatos <[email protected]> * revert test case change Signed-off-by: ddimatos <[email protected]> * Update release notes for 1.4.0-beta.2 Signed-off-by: ddimatos <[email protected]> * Updte content for EEs Signed-off-by: ddimatos <[email protected]> * Updte content for EEs Signed-off-by: ddimatos <[email protected]> Signed-off-by: ddimatos <[email protected]> Co-authored-by: Rich Parker <[email protected]> Co-authored-by: RadhaV2020 <[email protected]> Co-authored-by: Andrew Klychkov <[email protected]> Co-authored-by: Ivan Moreno <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Removed the duplicate "name" entry from the examples. Defect fix for https://jsw.ibm.com/browse/NAZARE-6112.
SUMMARY
ISSUE TYPE
COMPONENT NAME
ADDITIONAL INFORMATION