Add support for IAM role based credentials in Kinesis Plugin

[KKcorps]

Currently, Kinesis plugin only support DefaultCredentialsProvider but doesn't support IAM role based provider. The PR adds support for this feature.

Supported Configs

• iamRoleBasedAccessEnabled - Set it to true to enable role based access, default: false
• roleArn - specify the ARN of the role the client should assume. This is required.
• roleSessionName - session name to be used when creating a role based session. default: pinot-kineis-uuid
• externalId - string external id value required by role's policy. default: null
• sessionDurationSeconds - The duration, in seconds, of the role session. Default: 900
• asyncSessionUpdateEnabled - Configure whether the provider should fetch credentials asynchronously in the background. If this is true, threads are less likely to block when credentials are loaded, but additional resources are used to maintain the provider. Default - true

[codecov-commenter]

Codecov Report

Merging #9071 (518e168) into master (83410f6) will decrease coverage by 6.46%.
The diff coverage is 76.59%.

❗ Current head 518e168 differs from pull request most recent head 7d7bfe3. Consider uploading reports for the commit 7d7bfe3 to get more accurate results

@@             Coverage Diff              @@
##             master    #9071      +/-   ##
============================================
- Coverage     70.09%   63.62%   -6.47%     
+ Complexity     4741     4718      -23     
============================================
  Files          1831     1784      -47     
  Lines         96428    94897    -1531     
  Branches      14413    14382      -31     
============================================
- Hits          67591    60379    -7212     
- Misses        24169    30184    +6015     
+ Partials       4668     4334     -334     

Flag	Coverage Δ
integration1	?
integration2	?
unittests1	66.92% <79.84%> (+0.10%)	⬆️
unittests2	15.35% <1.18%> (-0.07%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../java/org/apache/pinot/common/utils/DataTable.java	95.45% <ø> (ø)
...che/pinot/core/common/datablock/BaseDataBlock.java	77.41% <0.00%> (-0.28%)	⬇️
...che/pinot/core/common/datatable/BaseDataTable.java	78.94% <ø> (ø)
...reaming/StreamingSelectionOnlyCombineOperator.java	0.00% <0.00%> (-65.31%)	⬇️
...ator/streaming/StreamingSelectionOnlyOperator.java	0.00% <0.00%> (-90.00%)	⬇️
...ctionaryBasedSingleColumnDistinctOnlyExecutor.java	0.00% <0.00%> (ø)
...e/pinot/core/query/reduce/RowBasedBlockValSet.java	15.94% <0.00%> (-0.24%)	⬇️
...main/java/org/apache/pinot/spi/data/FieldSpec.java	82.03% <ø> (ø)
...va/org/apache/pinot/spi/utils/CommonConstants.java	28.57% <ø> (ø)
...lugin/stream/kinesis/KinesisConnectionHandler.java	14.75% <3.70%> (-5.76%)	⬇️
... and 485 more

[navina]

Is it possible to add some unit tests for the configs validation ?

lgtm!

[KKcorps]

I planned to add tests in Kinesis integration but unfortunately the localstack IAM service doesn't work as expected and security token fetched from it is always invalid.

[navina]

I planned to add tests in Kinesis integration but unfortunately the localstack IAM service doesn't work as expected and security token fetched from it is always invalid.

Ah I see. I don't think we need localstack. I just meant a quick unit test for the KinesisConfig class with mock instances, if needed. This shouldn't block the PR from merge.

thanks for working on this!

[navina]

Please tag the PR with feature and release-notes