Issue finder #3
Issue finder #3
Conversation
| method_refs = self.get_method_refs(issue_def.keys()) | ||
| for name, ref in method_refs.items(): | ||
| found_all.add(ref(log_text, issue_def[name])) | ||
| return all(found_all) |
There was a problem hiding this comment.
when this class is extended to add other ways to find Issues, I wonder if it makes sense to use any instead of all?
There was a problem hiding this comment.
The idea is that for any given known issue, there are a set of circumstances that must be true in order to positively identify the issue you're trying to match. Each circumstance may require performing a unique check. For example, profile X in use, operator Y installed and configured and Z error message present in the log. Currently this PR only acts on the cucushift console.html. However, the extended version might also gain access to additional test artifacts such as jenkins logs, must-gather archives, etc. All checks would need to resolve to true to produce a positive match.
| import pathlib | ||
| from urllib.error import HTTPError | ||
| import urllib.request | ||
| import yaml |
There was a problem hiding this comment.
need to add requirements.txt file (or Pipfile if you're using pipenv) for pyyaml?
not part of this commit but perhaps we should also add some python version > 3.6 in requirements.txt file since f strings not supported in python3 versions prior to 3.6
| @@ -0,0 +1,4 @@ | |||
| description: 'https://issues.redhat.com/browse/OCPQE-5204' | |||
There was a problem hiding this comment.
Is the intention here to get known issues no matter which test case id the known issue is logged for as long as the string is matched?
I don't see an association between the known issue and testcase id the known issue is logged for which is fine as long as that is the intention.
There was a problem hiding this comment.
Correct there is no requirement that an issue be mapped 1:1 to a test case. Many bugs or infrastructure issues can cause failures across multiple test cases.
This PR adds a module to support searching for known issues in the given test run ID.
Example call and run output:
./parse_ci_monitor_json.py --runs 20210909-0851 -v 4.8 -i
.
.
.
.
Found issues for OCP-12024: ['https://issues.redhat.com/browse/OCPQE-5204']
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Found issues for OCP-26040: ['https://issues.redhat.com/browse/OCPQE-5743']
.
.
.
Found issues for OCP-12292: ['https://issues.redhat.com/browse/OCPQE-5743']
.
.
Found issues for OCP-43747: ['https://issues.redhat.com/browse/OCPQE-5743']
There is one search method named "match_ordered_strings" defined in this first iteration.
issue_OCPQE_5204.yaml and issue_OCPQE_5743.yaml are examples of how to describe an issue in a yaml definition file.
"description:" holds the string that will be printed if the issue is matched.
"match_ordered_strings:" holds a list of regex strings that will be passed to Issues.match_ordered_strings()
This module can be extended by adding additional methods to the Issues class and then adding an attribute to the yaml with a dict key that matches the method name. For example functionality might be added that checks to see if the tests were run on a particular cloud provider or if certain operators are installed. This may require getting access to additional log files or a must-gather.
The -i option is required to trigger the search, as it can take some time to download each log file and run the search.
To become useful additional issue yaml files will need to be added to support discovery of more existing issues. Scripting identification of known issues can save time for those triaging failures and avoid opening duplicate bugs.