TeamCity: archive debug logs if there are over 1000 of them

[modular-magician]

Description

This PR addresses issues like in this build:

The plugin used for uploading artifacts to S3 has a limit of 1000 artifacts per build. After contacting JetBrains support they suggested archiving files as a solution. The downside to this is that this would make it harder to open specific artifacts in your web browser, instead you'd need to download the entire zip/tar file, open it and then find the logs file you want.

The compromise discussed in the team chat was to archive logs only if a build exceeds the limit. This PR introduces that behaviour.

Testing

I used this PR to create a test project in TeamCity for manual testing of the bash in the new build step. This PR has some commits from when I forced builds to create >1000 or <1000 debug*.txt files and then see how the new build step reacted.

When there are >=1000 artifacts present

This commit GoogleCloudPlatform/magic-modules@7b966a7 shows how I forced builds to look like they had >1000 debug logs.

Here's a build in TeamCity showing the outcome:

In step 1 >1000 debug files are made

In step 2

21:33:51 Step 2/2: Tasks after running nightly tests: archive artifacts(debug logs) if there are >=1000 before S3 upload (Command Line)
21:33:51   Starting: /opt/teamcity-agent/temp/agentTmp/custom_script8203854418285142512
21:33:51   in directory: /opt/teamcity-agent/work/d9c38c33567f381b
21:33:51   Post-test step - archive artifacts(debug logs) if there are >=1000 before S3 upload
21:33:51   Found >=1000 debug artifacts; archiving before upload to S3
21:33:51   Listing files matching the artifact rule value /opt/teamcity-agent/work/d9c38c33567f381b/debug*
21:33:51   debug-google-1-TerraformProviders_SarahTesting1000artifactFix_GOOGLE_NIGHTLYTESTS_GOOGLE_PACKAGE_ACCESSAPPROVAL-archive.tar.gz
21:33:51   Finished
21:33:51   Process exited with code 0

The result is the artifacts tab contains only that .tar.gz file:

When there are <1000 artifacts present

This build shows the opposite scenario, where I made the builds instead create only 10 debug files (GoogleCloudPlatform/magic-modules@bfbca51)

Step 2 now looks like this:

22:10:50 Step 2/2: Tasks after running nightly tests: archive artifacts(debug logs) if there are >=1000 before S3 upload (Command Line)
22:10:50   Starting: /opt/teamcity-agent/temp/agentTmp/custom_script15021977902611658096
22:10:50   in directory: /opt/teamcity-agent/work/d9c38c33567f381b
22:10:50   Post-test step - archive artifacts(debug logs) if there are >=1000 before S3 upload
22:10:50   Found <1000 debug artifacts; we won't archive them before upload to S3
22:10:50 

And the artifacts tab shows the 10 files unchanged:

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

Derived from GoogleCloudPlatform/magic-modules#12083