Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flakiness of test_metric_collection #4346

Merged
merged 1 commit into from
May 25, 2023

Conversation

bayandin
Copy link
Member

Problem

Test test_metric_collection become flaky (an example):

AssertionError: assert not ['2023-05-25T14:03:41.644042Z ERROR metrics_collection: failed to send metrics: reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(18022), path: "/billing/api/v1/usage_events", query: None, fragment: None }, source: hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" })) }',
                            ...]

I suspect it is caused by having 2 places when we define httpserver_listen_address fixture (which is internally used by pytest-httpserver plugin)

@pytest.fixture(scope="session")
def httpserver_listen_address(port_distributor: PortDistributor):
    port = port_distributor.get_port()
    return ("localhost", port)

Summary of changes

  • Remove definition of httpserver_listen_address from test_runner/regress/test_ddl_forwarding.py and keep one in test_runner/fixtures/neon_fixtures.py
  • Also remote unused httpserver_listen_address parameter from test_proxy_metric_collection

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@bayandin bayandin requested a review from save-buffer May 25, 2023 15:49
@github-actions
Copy link

github-actions bot commented May 25, 2023

1004 tests run: 964 passed, 0 failed, 40 skipped (full report)


Flaky tests (3)

Postgres 15

  • test_remote_storage_upload_queue_retries[local_fs]: ✅ debug
  • test_remote_timeline_client_calls_started_metric[local_fs]: ✅ release
  • test_get_tenant_size_with_multiple_branches: ✅ release
The comment gets automatically updated with the latest test results
6af13ba at 2023-05-25T19:16:57.851Z :recycle:

@bayandin bayandin force-pushed the bayandin/fix-test_metric_collection branch from 078e898 to 6af13ba Compare May 25, 2023 16:24
Copy link
Contributor

@save-buffer save-buffer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this! I guess I didn't realize we had a port distributor in neon_fixtures

@bayandin bayandin enabled auto-merge (squash) May 25, 2023 21:01
@bayandin bayandin merged commit 2b25f0d into main May 25, 2023
@bayandin bayandin deleted the bayandin/fix-test_metric_collection branch May 25, 2023 21:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants