Skip to content

Commit

Permalink
Adding a Github workflow fo testimony validation
Browse files Browse the repository at this point in the history
This fix should solve CCT-1152 - Fix Testimony for integration-tests of insights-client
  • Loading branch information
zpetrace committed Jan 31, 2025
1 parent df98f19 commit 93656ef
Show file tree
Hide file tree
Showing 16 changed files with 87 additions and 52 deletions.
27 changes: 27 additions & 0 deletions .github/workflows/testimony.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
name: Testimony Validation

on:
pull_request:
paths:
- "integration-tests/**"

jobs:
testimony:
name: testimony validate
runs-on: ubuntu-latest
container:
image: fedora:latest

steps:
- name: Setup for Testimony
run: |
dnf --setopt install_weak_deps=False install -y \
python3-pip
python3 -m pip install testimony
- uses: actions/checkout@v4

- name: Run Testimony
run: |
testimony validate --config \
integration-tests/testimony.yml integration-tests/test*
2 changes: 0 additions & 2 deletions integration-tests/test_checkin.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ def test_ultralight_checkin(insights_client, test_config):
This test verifies that performing an ultra-light check-in with the
insights-client updates the host's 'stale_timestamps' and 'updated'
fields on the server
:reference:
:tags: Tier 2
:steps:
1. Register the insights-client
Expand Down Expand Up @@ -72,7 +71,6 @@ def test_client_checkin_unregistered(insights_client):
:description:
This test verifies that attempting to perform check-in while unregistered
fails with appropriate error message
:reference:
:tags: Tier 1
:steps:
1. Unregister the insights-client if registered
Expand Down
6 changes: 0 additions & 6 deletions integration-tests/test_client_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ def test_no_upload(insights_client):
:description:
This test verifies that no traceback is returned when running --no-upload
command and the archive is created and saved
:reference:
:tags: Tier 1
:steps:
1. Register insights-client
Expand Down Expand Up @@ -91,7 +90,6 @@ def test_group(insights_client, tmp_path):
:description:
This test verifies that the --group option is functional, ensuring that the
group specified is created and packed in the archive
:reference:
:tags: Tier 1
:steps:
1. Run insights-client in offline mode to generate archive
Expand Down Expand Up @@ -137,7 +135,6 @@ def test_support(insights_client):
:description:
This test verifies that the --support option provides the expected
information and generates a support log for Red Hat insights
:reference:
:tags: Tier 1
:steps:
1. Run insights-client with --support option
Expand Down Expand Up @@ -213,7 +210,6 @@ def test_client_checkin_offline(insights_client):
:description:
This test verifies that running the --checkin command in offline
mode logs an appropriate message and exits with a failure code
:reference:
:tags: Tier 1
:steps:
1. Register insights-client
Expand All @@ -237,7 +233,6 @@ def test_client_diagnosis(insights_client):
:description:
This test verifies that on a registered system, the --diagnosis
option retrieves the correct diagnostic information
:reference:
:tags: Tier 1
:steps:
1. Run the --diagnosis option on unregistered system
Expand Down Expand Up @@ -274,7 +269,6 @@ def test_check_show_results(insights_client):
:description:
This test verifies that when checking results from the Advisor (--check-results)
and displaying them (--show-results), a remediation is advised
:reference:
:tags: Tier 1
:steps:
1. Register insights-client
Expand Down
1 change: 0 additions & 1 deletion integration-tests/test_client_systemd.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ def test_data_upload_systemd_timer(insights_client, external_inventory):
:description:
Ensure that the insights-client data upload is triggered by the
systemd timer and the last check-in time is updated accordingly
:reference:
:tags: Tier 1
:steps:
1. Register the system
Expand Down
1 change: 0 additions & 1 deletion integration-tests/test_common_specs.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ def test_common_specs(insights_client, tmp_path):
:description:
This test verifies that the specified specs can be collected,
parsed and contain valid data without errors
:reference:
:tags: Tier 1
:steps:
1. Define the list of common specs to be tested
Expand Down
4 changes: 0 additions & 4 deletions integration-tests/test_display_name_option.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ def test_display_name(insights_client):
:description:
This test verifies that a registered host's display name can be
updated using the --display-name option.
:reference:
:tags: Tier 1
:steps:
1. Generate a unique hostname and register the insights-client
Expand Down Expand Up @@ -78,7 +77,6 @@ def test_register_with_display_name(insights_client):
:description:
This test ensures that registering the insights-client with a custom
display name sets the display name correctly in host details
:reference:
:tags: Tier 1
:steps:
1. Generate a unique hostname
Expand Down Expand Up @@ -112,7 +110,6 @@ def test_register_twice_with_different_display_name(
:description:
This test checks that registering the insights-client twice with different
display names does not change the insights_id and display_name is updated
:reference:
:tags: Tier 1
:steps:
1. Generate a unique hostname
Expand Down Expand Up @@ -172,7 +169,6 @@ def test_invalid_display_name(invalid_display_name, insights_client):
:description:
This test verifies that attempting to set an invalid display_name is rejected
and does not alter the current display_name value
:reference:
:tags: Tier 1
:steps:
1. Register the insights-client
Expand Down
1 change: 0 additions & 1 deletion integration-tests/test_e2e.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ def test_insights_client_version_in_inventory(insights_client, external_inventor
:description:
Ensure that running insights-client creates a new host entry in the
Inventory and includes both the insights-client and egg version information
:reference:
:tags: Tier 1
:steps:
1. Register the system with insights-client
Expand Down
1 change: 0 additions & 1 deletion integration-tests/test_manpage.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ def test_manpage(option):
:description:
This test verifies that the insights-client manual page includes
all the specified options.
:reference:
:tags: Tier 1
:steps:
1. Open the manual page
Expand Down
2 changes: 0 additions & 2 deletions integration-tests/test_motd.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@ def test_motd(insights_client):
This test verifies that the MOTD file exists on an unregistered
system and that it is appropriately removed or not recreated upon
registration and unregistration
:reference:
:tags: Tier 1
:steps:
1. Verify that MOTD file is present on an unregistered system
Expand Down Expand Up @@ -94,7 +93,6 @@ def test_motd_dev_null(insights_client):
This tst ensures that of the MOTD file is a symbolic link to
/dev/null, it is not modified or removed during the client's registration
and unregistration processes
:reference:
:tags: Tier 1
:steps:
1. Create a symlink from MOTD_PATH to /dev/null
Expand Down
4 changes: 0 additions & 4 deletions integration-tests/test_obfuscation.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ def test_ip_obfuscation(insights_client, tmp_path):
This test verifies that when IP obfuscation is enabled in the
insights-client configuration, the system's IP address is not
present in the collected archive
:reference:
:tags: Tier 1
:steps:
1. Record the system's IP address
Expand Down Expand Up @@ -63,7 +62,6 @@ def test_hostname_obfuscation(insights_client, tmp_path):
This test verifies that when hostname obfuscation is enabled in the
insights-client configuration, the system's hostname is not present
in the collected archive
:reference:
:tags: Tier 1
:steps:
1. Record the system's hostname
Expand Down Expand Up @@ -103,7 +101,6 @@ def test_password_obfuscation(insights_client, tmp_path, password_file):
This test ensures that sensitive information such as passwords is obfuscated
in collected files, regardless of the obfuscation setting in the configuration
file
:reference:
:tags: Tier 1
:steps:
1. Backup the original content of the test file
Expand Down Expand Up @@ -200,7 +197,6 @@ def test_no_obfuscation_on_display_name(insights_client, tmp_path):
This test ensures that display_name in package information files
are not incorrectly obfuscated as IP addresses when obfuscation is
enabled
:reference:
:tags: Tier 1
:steps:
1. Enable obfuscation in the configuration file
Expand Down
4 changes: 0 additions & 4 deletions integration-tests/test_redaction.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ def test_redaction_not_on_cmd(insights_client, tmp_path, not_removed_command):
This test verifies that when no commands are configured for redaction in
`/etc/insights-client/file-redaction.yaml`, the outputs of the related
commands are included in the collection archive
:reference:
:tags: Tier 1
:steps:
1. Ensure no command redaction is configured
Expand All @@ -53,7 +52,6 @@ def test_redaction_on_cmd(insights_client, tmp_path, removed_command):
This test verifies that when commands are configured for redaction in
`/etc/insights-client/file-redaction.yaml`, the outputs of the related
commands are excluded from the collection archive
:reference:
:tags: Tier 1
:steps:
1. Configure command redaction for the specified command
Expand Down Expand Up @@ -82,7 +80,6 @@ def test_redaction_not_on_file(insights_client, tmp_path, not_removed_file):
This test verifies that when no files are configured for redaction in
`/etc/insights-client/file-redaction.yaml`, the related files are
included in the collection archive
:reference:
:tags: Tier 1
:steps:
1. Ensure no file redaction is configured
Expand All @@ -105,7 +102,6 @@ def test_redaction_on_file(insights_client, tmp_path, removed_file):
This test verifies that when files are configured for redaction in
`/etc/insights-client/file-redaction.yaml`, the related files are
excluded from the collection archive
:reference:
:tags: Tier 1
:steps:
1. Configure file redaction for the specified file
Expand Down
7 changes: 0 additions & 7 deletions integration-tests/test_registration.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ def test_register(insights_client):
:description:
This test verifies that the --register command successfully registers
an unregistered client
:reference:
:tags: Tier 1
:steps:
1. Run insights-client with --register option
Expand Down Expand Up @@ -51,7 +50,6 @@ def test_register_auth_proxy(insights_client, test_config):
:description:
This test verifies that the --register command successfully registers the
host when an authentication proxy is configured
:reference:
:tags: Tier 1
:steps:
1. Set the proxy configuration in the insights-client.conf file
Expand Down Expand Up @@ -91,7 +89,6 @@ def test_register_noauth_proxy(insights_client, test_config):
:description:
This test verifies that the --register command successfully registers the
host when a unauthenticated proxy is configured
:reference:
:tags: Tier 1
:steps:
1. Set the proxy configuration in the insights-client.conf file
Expand Down Expand Up @@ -124,7 +121,6 @@ def test_machineid_exists_only_when_registered(insights_client):
:description:
This test verifies that the machine ID file is created only when the client
is registered
:reference:
:tags: Tier 1
:steps:
1. Verify the client is not registered and machine ID does not exist
Expand Down Expand Up @@ -163,7 +159,6 @@ def test_machineid_changes_on_new_registration(insights_client):
:description:
This test verifies that the machine ID file content changes when
the client is unregistered and then registered again
:reference:
:tags: Tier 1
:steps:
1. Register insights-client and store current machine ID
Expand Down Expand Up @@ -201,7 +196,6 @@ def test_double_registration(insights_client):
:description:
This test verifies that the --register flag can be passed multiple
times on a system that is already registered without causing errors
:reference:
:tags: Tier 1
:steps:
1. Register insights-client and store its machine ID
Expand Down Expand Up @@ -277,7 +271,6 @@ def test_registered_and_unregistered_files_are_created_and_deleted(insights_clie
This test verifies that the .registered file is created when the client
is registered and the .unregistered file is created when the client is
unregistered
:reference:
:tags: Tier 1
:steps:
1. Verify that the client is not registered and .registered file does not exist
Expand Down
6 changes: 2 additions & 4 deletions integration-tests/test_status.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ def test_status_registered(external_candlepin, insights_client):
:description:
This test verifies that when the insights client is registered, the
`insights-client --status` command outputs the correct registration status
:reference:
:tags: Tier 1
:steps:
1. Register the insights-client
Expand All @@ -32,7 +31,7 @@ def test_status_registered(external_candlepin, insights_client):
2. Wait time completes without issues
3. If 'legacy_upload' is True, output contains "Insights API confirms
registration." If 'legacy_upload' is False, output is "This host
is registered.\n"
is registered."
"""
insights_client.register()
assert conftest.loop_until(lambda: insights_client.is_registered)
Expand All @@ -53,7 +52,6 @@ def test_status_unregistered(external_candlepin, insights_client):
This test verifies that when the insights client is unregistered, the
`insights-client --status` command outputs the correct unregistration
status
:reference:
:tags: Tier 1
:steps:
1. Unregister the insights client to ensure it's unregistered
Expand All @@ -63,7 +61,7 @@ def test_status_unregistered(external_candlepin, insights_client):
2. If 'legacy_upload' is True return code is 1 and output contains
"Insights API says this machine is NOT registered."
If 'legacy_upload' is False return code is 0 and output contains
"This host is unregistered.\n"
"This host is unregistered."
"""
# running unregistration to ensure system is unregistered
with contextlib.suppress(Exception):
Expand Down
23 changes: 9 additions & 14 deletions integration-tests/test_tags.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,24 +22,19 @@ def test_tags(insights_client, external_inventory, test_config):
:title: Test tags
:description:
Test how the tags generated, and check the tags on inventory
:reference:
:tier: Tier 1
steps:
1. Register insights-client, and check satellite related
tags on the inventory if the test env is satellte.
:tags: Tier 1
:steps:
1. Register insights-client, and check satellite related tags
on the inventory if the test env is satellte.
2. Run insights-client with the --group
3. Add a new tag in tags.yaml, and run insights-client,
then check the inventory
then check the inventory
:expectedresults:
1. system is registered to insights, and there will be
satellite related tags supported by branch_info with
satellite env.
1. System is registered to insights, and there will be
satellite related tags supported by branch_info with
satellite env.
2. tags.yaml will be created with group option, and new tag
generated by tags.yaml
generated by tags.yaml
3. The new tag shows on inventory by modifying tags.yaml
"""
# Remove the tags.yaml if it exists
Expand Down
1 change: 0 additions & 1 deletion integration-tests/test_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ def test_version(insights_client):
:description:
This test verifies that running `insights-client --version` outputs
both the client and core version information
:reference:
:tags: Tier 1
:steps:
1. Run `insights-client --version`
Expand Down
Loading

0 comments on commit 93656ef

Please sign in to comment.