Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix keyword to watch SM cluster provisioning logs and increase timeout #1754

Merged
merged 19 commits into from
Aug 29, 2024

Conversation

bdattoma
Copy link
Contributor

@bdattoma bdattoma commented Aug 28, 2024

Content of the PR:

  1. change logic of Watch Hive Install Log keyword to avoid RHOAIENG-10829 bug
  2. increase timeout to 70m

This won't necessarily fix some of the failures we have seen in CI while provisioning clusters, but would:

  1. ensure to provide provisioning logs when available
  2. reduce the chance of failure in case provisioning takes more than the fixed timeout

PR validation:

  1. PSI provisioning OK
    image

  2. IBM Cloud provisioning OK
    image

@bdattoma bdattoma self-assigned this Aug 28, 2024
@bdattoma bdattoma added needs testing Needs to be tested in Jenkins enhancements Bugfixes, enhancements, refactoring, ... in tests or libraries (PR will be listed in release-notes) labels Aug 28, 2024
END
Run Keyword And Continue On Failure Should Be Equal As Integers ${return_code} ${0}
${hive_pods_status} = Run And Return Rc oc get pod -n ${namespace} --no-headers | awk '{print $3}' | grep -v 'Completed'
IF ${hive_pods_status} != 0

Check notice

Code scanning / Robocop

'{{ block_name }}' condition can be simplified Note

'IF' condition can be simplified
Copy link
Contributor

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
525 0 0 525 100

kobihk
kobihk previously approved these changes Aug 29, 2024
@bdattoma bdattoma requested a review from manosnoam August 29, 2024 12:24
@bdattoma bdattoma requested review from kobihk and manosnoam August 29, 2024 14:00
@bdattoma bdattoma added verified This PR has been tested with Jenkins and removed needs testing Needs to be tested in Jenkins labels Aug 29, 2024
Copy link

@bdattoma bdattoma merged commit 9ed2491 into red-hat-data-services:master Aug 29, 2024
10 checks passed
jgarciao pushed a commit to jgarciao/ods-ci that referenced this pull request Sep 1, 2024
red-hat-data-services#1754)

* draft fix hive watcher

* do not print dots when process is not producing new outputs

* wait for ns to be created before looking for logs

* wait until logs are available to be fetched

* fix wait for pod

* fix wait kw call

* clean new code

* increase provisioning timeout

* extend label selector for cd deployments

* use job type label for cluster pool deployments

* fix catenation separator

* remove commented line

* fix robocop alerts

* minor change

* increase timeout further

* apply suggestions from reviews

* remove wrong return variables

* fix robocop alert

* remove debug lines
@manosnoam
Copy link
Contributor

Hi @bdattoma although you added output_should_contain=install completed successfully you should also fail fast if fatal error occurs, as in this job job/rhods/job/rhods-ci-pr-test/3312/console

Please also add output_should_not_contain=fatal msg

tonyxrmdavidson pushed a commit to tonyxrmdavidson/ods-ci that referenced this pull request Sep 4, 2024
red-hat-data-services#1754)

* draft fix hive watcher

* do not print dots when process is not producing new outputs

* wait for ns to be created before looking for logs

* wait until logs are available to be fetched

* fix wait for pod

* fix wait kw call

* clean new code

* increase provisioning timeout

* extend label selector for cd deployments

* use job type label for cluster pool deployments

* fix catenation separator

* remove commented line

* fix robocop alerts

* minor change

* increase timeout further

* apply suggestions from reviews

* remove wrong return variables

* fix robocop alert

* remove debug lines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancements Bugfixes, enhancements, refactoring, ... in tests or libraries (PR will be listed in release-notes) verified This PR has been tested with Jenkins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants