Fix flaky test SegmentReplicationAllocationIT.testAllocationAndRebalanceWithDisruption

[ltaragi]

Description

• This change fixes the flaky nature of SegmentReplicationAllocationIT.testAllocationAndRebalanceWithDisruption IT.
• In the original test, we needed to make sure that the cluster had more no. of data nodes than the number of shard copies but it was not actually a guarantee leading to the flaky nature of tests.
• We are now pre-generating the random index/shard counts and then spawning the data nodes accordingly.
• Also decreased the upper limit on number of indices because original limit would have caused spawning of as high as 60 data nodes (10 * 2 * (1 + 2)) which was causing transport related failures unrelated to the scope of the test.
• Tested the new changes with ~3.4K consecutive IT runs without any failures

BUILD SUCCESSFUL in 18s
60 actionable tasks: 1 executed, 59 up-to-date
[2025-02-22 23:19:08] : ===================================================================
[2025-02-22 23:19:08] : counter=3412
[2025-02-22 23:19:08] : ===================================================================

Related Issues

Partially resolves #14327

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

✅ Gradle check result for b799100: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.46%. Comparing base (8447737) to head (b799100).
Report is 2 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #17429      +/-   ##
============================================
- Coverage     72.48%   72.46%   -0.02%     
- Complexity    65637    65667      +30     
============================================
  Files          5303     5303              
  Lines        304793   304793              
  Branches      44202    44202              
============================================
- Hits         220926   220879      -47     
- Misses        65711    65869     +158     
+ Partials      18156    18045     -111     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.