Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set a timeout on kube pod process integration tests to prevent a hang #7087

Merged
merged 12 commits into from
Oct 26, 2021

Conversation

airbyte-jenny
Copy link
Contributor

What

Sets a timeout for KubePodProcessIntegrationTest because it uses some infinite-loop child processes, and if the test case fails, those need to be aborted for the overall script to progress. This protected from a hang condition that is sometimes observed.

How

Adds a class-level timeout to the integration test. The duration of the wait time (2 minutes) is an estimate and open to change.

@airbyte-jenny airbyte-jenny requested a review from jrhizor October 15, 2021 22:50
@github-actions github-actions bot added the area/worker Related to worker label Oct 15, 2021
@airbyte-jenny airbyte-jenny changed the title Airbyte jenny/kube test stability Set a timeout on kube pod process integration tests to prevent a hang Oct 15, 2021
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 15, 2021 22:56 Inactive
@@ -148,7 +152,7 @@ public void testKillingWithoutHeartbeat() throws Exception {
server.stop();

// waiting for process
process.waitFor();
process.waitFor(2, TimeUnit.MINUTES);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we leave this as-is? Internally we call process.waitFor(); and I'd like to make sure the same thing is tested here in case behavior diverges

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should timeout if the test takes too long anyways, right?

@github-actions github-actions bot added the area/platform issues related to the platform label Oct 25, 2021
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 20:02 Inactive
@github-actions github-actions bot added the area/connectors Connector related issues label Oct 25, 2021
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 20:21 Inactive
@github-actions github-actions bot removed the area/connectors Connector related issues label Oct 25, 2021
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 20:48 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 21:40 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 22:16 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 22:23 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 22:56 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 25, 2021 23:32 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 26, 2021 15:01 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 26, 2021 15:58 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 26, 2021 16:47 Inactive
@airbyte-jenny airbyte-jenny temporarily deployed to more-secrets October 26, 2021 17:17 Inactive
@airbyte-jenny airbyte-jenny merged commit c2adf9b into master Oct 26, 2021
@airbyte-jenny airbyte-jenny deleted the airbyte-jenny/kube-test-stability branch October 26, 2021 18:05
schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022
…airbytehq#7087)

* Improved logging for troubleshooting.

* Limiting the number of tests for faster iteration testing.

* Re-enable tests

* Removed unnecessary logging from troubleshooting

* Formatting

* whitespace

* Eliminated duplicate timeout.

* Whitespace/formatting

* Show logs when a sync job fails during testing.

* Formatting

* Use alphanumeric random string, not bytes, to test paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/platform issues related to the platform area/worker Related to worker
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants