Updated bedrock.py to handle case of deploying it over Lambda function with attached role

[irshadc]

#84 [Bug]: Code doesn't work on AWS Lambda function when deployed with attached Role to access bedrock service.

Description

Code was not working on AWS Lambda function when deployed with attached Role to access bedrock service.

Why are these changes needed?

When passed credentials locally or via AWS profile, it works fine. However, it doesn't pick automatically session when deployed on AWS with managed services (eg: AWS Lambda function, Amazon ECS task, etc.) or Amazon EC2 with attached role. Although it is recommended and more secure way than creating access credentials.

I have made changes to build bedrock-runtime client directly using boto3 which uses attached Role to managed AWS service.

This is more secure way to running code over AWS. Tested it with AWS Lambda function and ECS container task.

Key Changes:

• Implemented direct bedrock-runtime client creation via boto3
• Added support for IAM Role-based authentication
• Verified functionality on AWS Lambda and ECS container tasks

Related issue number

Fixes #84

Checks

• I've included any doc changes needed for https://ag2ai.github.io/ag2/. See https://ag2ai.github.io/ag2/docs/Contribute#documentation to build and test documentation locally. [NA - as no mention in docs]
• I've added tests (if relevant) corresponding to the changes introduced in this PR. [It requires managed AWS servers to test it]
• I've made sure all auto checks have passed.

Testing Details

• Successfully tested on AWS Lambda function
• Verified on ECS container task
• Confirmed proper IAM Role recognition and authentication

[Hk669]

thanks @irshadc , for the PR. the changes make sense to me.

cc @marklysze, have you tried/tested this locally? please let me know, thanks.

[marklysze]

thanks @irshadc , for the PR. the changes make sense to me.

cc @marklysze, have you tried/tested this locally? please let me know, thanks.

I'm afraid I haven't yet. @irshadc, are you able to run the pre-commit checks, I think the code formatting will need an update. more info here

I'm not familiar with how we could test this as it seems to require running in AWS with an attached role - @irshadc any guidance here?

[qingyun-wu]

@irshadc could you give me permission to your fork and I can help you fix the code format issue. Thank you!

[irshadc]

@marklysze I gave you access to forked-repo. Please update the code. As it is first time I am sending a patch here.

Yes, You need to have AWS account to test it further. I am using my forked repo in requirements.txt to deploy it in AWS Lambda function.

[irshadc]

#85

[sonichi]

@irshadc thanks for your contribution. Could you share your Discord handle and/or email address with @qingyun-wu for CLA?

[marklysze]

@marklysze I gave you access to forked-repo. Please update the code. As it is first time I am sending a patch here.

Yes, You need to have AWS account to test it further. I am using my forked repo in requirements.txt to deploy it in AWS Lambda function.

Thanks @irshadc, I've updated based on the pre-commit and it should pass. Let me check further if I can test it in AWS.

[marklysze]

Thanks @irshadc, I tested in an AWS Lambda function using your repo in the requirements.txt and it worked:

Status: Succeeded
Test Event Name: (unsaved) test event

Response:
null

Function Logs:
model': 'meta.llama3-1-8b-instruct-v1:0', 'aws_region': 'us-west-2'}
/var/task/autogen/oai/bedrock.py:624: UserWarning: Cannot get the costs for meta.llama3-1-8b-instruct-v1:0. The cost will be 0. In your config_list, add field {"price" : [prompt_price_per_1k, completion_token_price_per_1k]} for customized pricing.
warnings.warn(
chatbot (to user_proxy):
{"type":"function","name":"currency_calculator",("type":"function","name":"currency_calculator",)","parameters":{}}
--------------------------------------------------------------------------------
user_proxy (to chatbot):
--------------------------------------------------------------------------------
{'messages': [{'content': "For currency exchange tasks,\n                only use the functions you have been provided with.\n                Output 'TERMINATE' when an answer has been provided.", 'role': 'system'}, {'content': 'How much is 123.45 EUR in USD?', 'role': 'user', 'name': 'user_proxy'}, {'content': '\n\n{"type":"function","name":"currency_calculator",("type":"function","name":"currency_calculator",)","parameters":{}}', 'role': 'assistant', 'name': 'chatbot'}, {'content': '', 'role': 'user', 'name': 'user_proxy'}], 'tools': [{'type': 'function', 'function': {'description': 'Currency exchange calculator.', 'name': 'currency_calculator', 'parameters': {'type': 'object', 'properties': {'base_amount': {'type': 'number', 'description': 'Amount of currency in base_currency'}, 'base_currency': {'enum': ['USD', 'EUR'], 'type': 'string', 'default': 'USD', 'description': 'Base currency'}, 'quote_currency': {'enum': ['USD', 'EUR'], 'type': 'string', 'default': 'EUR', 'description': 'Quote currency'}}, 'required': ['base_amount']}}}], 'model': 'meta.llama3-1-8b-instruct-v1:0', 'aws_region': 'us-west-2'}
chatbot (to user_proxy):
***** Suggested tool call (tooluse_WyccZ8XWTY-UuEk_YAvIQw): currency_calculator *****
Arguments: 
{"base_currency": "EUR", "base_amount": "123", "quote_currency": "USD"}
*************************************************************************************
--------------------------------------------------------------------------------
{'messages': [{'content': 'How much is 123.45 EUR in USD?', 'role': 'user', 'name': 'user_proxy'}, {'content': '\n\n{"type":"function","name":"currency_calculator",("type":"function","name":"currency_calculator",)","parameters":{}}', 'role': 'assistant', 'name': 'chatbot'}, {'content': '', 'role': 'user', 'name': 'user_proxy'}, {'content': '', 'tool_calls': [{'id': 'tooluse_WyccZ8XWTY-UuEk_YAvIQw', 'function': {'arguments': '{"base_currency": "EUR", "base_amount": "123", "quote_currency": "USD"}', 'name': 'currency_calculator'}, 'type': 'function'}], 'role': 'assistant'}, {'role': 'system', 'content': 'Summarize the takeaway from the conversation. Do not add any introductory phrases.'}], 'tools': [{'type': 'function', 'function': {'description': 'Currency exchange calculator.', 'name': 'currency_calculator', 'parameters': {'type': 'object', 'properties': {'base_amount': {'type': 'number', 'description': 'Amount of currency in base_currency'}, 'base_currency': {'enum': ['USD', 'EUR'], 'type': 'string', 'default': 'USD', 'description': 'Base currency'}, 'quote_currency': {'enum': ['USD', 'EUR'], 'type': 'string', 'default': 'EUR', 'description': 'Quote currency'}}, 'required': ['base_amount']}}}], 'model': 'meta.llama3-1-8b-instruct-v1:0', 'aws_region': 'us-west-2'}
Unfortunately, it seems that the conversation got cut off. To get an actual answer to the original question, I'll use an external exchange rate API to get the latest exchange rate.
As of my knowledge cutoff, the exchange rate is approximately 1 EUR = 1.12 USD.
So, 123.45 EUR would be equivalent to:
123.45 EUR x 1.12 USD/EUR ≈ 138.27 USD
Please note that this is an estimate and exchange rates may vary depending on the current market rate.
END RequestId: 5fffd426-f9bb-4799-b1aa-2101e9ba4564
REPORT RequestId: 5fffd426-f9bb-4799-b1aa-2101e9ba4564	Duration: 2928.38 ms	Billed Duration: 2929 ms	Memory Size: 512 MB	Max Memory Used: 161 MB	Init Duration: 3820.60 ms

Request ID: 5fffd426-f9bb-4799-b1aa-2101e9ba4564

I also tested using the original AWS_ACCESS_KEY and AWS_SECRET_KEY and they worked as well. I think we're good to go.

[marklysze]

Tested with an AWS Lambda function and it worked as expected. Also tested with original credentials and still worked. Thanks @irshadc

[marklysze]

Note: I've added a comment in the Bedrock documentation that this is supported.

[marklysze]

@irshadc if you haven't sent your email address to @qingyun-wu, can you please let me know so we can get a CLA across to you, you can catch me on Discord, _msze or email me, mark @ag2.ai.

[irshadc]

Thanks a lot @marklysze, My email id is [email protected] I am not on discord.