Build Cop: duplicate issues for same test failure?

[shollyman]

Unclear why we're getting multiple bugs for the same test:

https://github.com/GoogleCloudPlatform/python-docs-samples/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+test_ncaa_tutorial

Currently yields 4 issues for the same test:
GoogleCloudPlatform/python-docs-samples#2892
GoogleCloudPlatform/python-docs-samples#2891
GoogleCloudPlatform/python-docs-samples#2849
GoogleCloudPlatform/python-docs-samples#2800

[tbpg]

My first thought was that this was a race condition when parallel runs finish at the same time. But, it looks like some of those issues were filed days apart.

GoogleCloudPlatform/python-docs-samples#2892 and GoogleCloudPlatform/python-docs-samples#2891 look like a race condition.

I'm not sure why the other issues aren't being found (or, only found sometimes).

[tbpg]

The problem is pagination. There are > 32 Build Cop issues in python-docs-samples: https://github.com/GoogleCloudPlatform/python-docs-samples/issues?utf8=%E2%9C%93&q=is%3Aissue+label%3A%22buildcop%3Aissue%22

But, we currently only get the first 32 issues. So, it's just luck with whether or not the already-existing issue is returned/found.

repo-automation-bots/packages/buildcop/src/buildcop.ts

Lines 115 to 123 in 4dd6255

	const issues = (
	await context.github.issues.listForRepo({
	owner,
	repo,
	per_page: 32,
	labels: ISSUE_LABEL,
	state: 'all', // Include open and closed issues.
	})
	).data;

Pagination docs are here: https://octokit.github.io/rest.js/#pagination

I'll send a PR.

[tbpg]

Part of the fix needs to include closing duplicate issues, maybe preferring to keep an issue someone other than the bot has commented on.

[tbpg]

For some reason, we're getting many, many duplicate comments on issues (sorry!)

Example: GoogleCloudPlatform/python-docs-samples#2849

Looking.

[tbpg]

All of the duplicate comments are because there are many, many sponge_log.xml files that get sent to the bot.

The bot then has a different instance for each one of the logs. They are all running at the same time, they all try to close the duplicate issues, then they all comment at the same time.

I can't think of a way to prevent this race condition within the bot itself. So, one idea is to add some sort of external locking mechanism.

We could limit concurrency, but the same Cloud Function is used for every repo, so that would just slow things down.