HTTP/HTTPS proxy for connections

[asieira]

The old boto library had arguments proxy, proxy_port, proxy_user and proxy_pass to allow connections to the API endpoints to go through a proxy.

What is the boto3 equivalent way of programmatically setting the proxy parameters (i.e., not using the configuration file)?

[aki263]

I have same issue. I cant event set proxy with global conf file placed in /etc/boto.cfg .

[kyleknap]

So boto3 does not require the global config to set proxies. All you need to do is set the general https_proxy environment variable and requests/urllib3 should use the value stored in that environment variable. So something like:

export https_proxy=https://username:password@hostname:port

Let me know if this helps.

[asieira]

I think this is a poor replacement. I might want to override what's in the environment variable without changing it and causing side effects on other libraries or portions of a larger program.

Also, this environment variable is a very *nix-centric standard, which IMHO is poor form for a portable library such as boto3.

I would strongly suggest that a way for this to be specified explicitly is available when a boto3 connection is opened.

[themanifold]

I would agree with the above comment. I think it would be better if the proxies could be set via the config file, and better still, via the interface outlined in #325

[jamesls]

Marking as a feature request.

[bobveznat]

We also require this functionality. We need a clean way of indicating that boto uses the proxy while other services on the same machine do not. Effectively it boils down to "local" services don't require a proxy but "internet" services do.

We're actually going to port our greenfield boto3 code back to boto2 now.

[themanifold]

Has there been any progress on this? We require this for our organisation too.

[themanifold]

This is just another ping to find out what is going on with this feature...

[alborworld]

Same thing here. We need this functionality and currently we're backporting boto3 code to boto2 because of this. Please implement this feature.

[mjs2180]

Same request but for making use of boto3 running on AWS Lambda in a VPC to connect via a corporate proxy. Need to be able to set a proxy in the code as setting the environment variables has to be done before you instantiate the AWS clients. This means I can't pass them as parameters and they have to be hardcoded.

[marekq]

I have the same issue from an internal network, would love to see proper support for proxies!

[feifangit]

any update?

[MikeMcMahon]

any movement on this issue? Seems silly that we'd lose this functionality, it's kind of a pain when running boto3 bundled into something like apache. so it's not really running with the standard environment, we have to utilize apache libs to pass in environment variables.

[jamesls]

So would adding this as an option to the Config object proposed here be sufficient? Something like:

ec2 = boto3.client('ec2', config=Config(proxy='http...'))

[MikeMcMahon]

@jamesls A better solution than current, but still preferred would be having the prior boto functionality of being able to specify it via a configuration file, especially in terms of credentials storage.

[jamesls]

Ok, so to capture both ideas, you could ideally do either:

ec2 = boto3.client('ec2', config=Config(proxy='http....'))

Or you could do:

$ cat ~/.aws/config
[default]
proxy=http://...

$ python
>>> import boto3
>>> boto3.client('ec2') # Would use the proxy settings.

[tmylk]

May I ask if this is on the roadmap?
The lack of this feature stops us from migrating to boto3 from boto2 in the smart_open package.

[skortchmark9]

This is important for our use case also. Specifically, setting the http proxy interferes with pip installs from our pip index.

[alborworld]

Hi, please have a look at the following solution.

https://aws.amazon.com/blogs/security/how-to-add-dns-filtering-to-your-nat-instance-with-squid/

Apparently this would free us from the need of setting the HTTP proxy in every instance of the private subnet. What do you think?

[asieira]

This article describes one particular way of implementing a transparent proxy. However, some of us have to write Python programs using boto that run on environments we don't control and where using an explicit proxy is a requirement.

If you do control your network environment and can use a transparent proxy as a workaround for the lack of explicit proxy control in boto3, then by all means do so. But IMHO the need to allow this configuration in boto3 still remains as important as ever.

[FlorianHeigl]

Why is a regression marked as a feature anyway?

[thewb]

So you guys broke functionality and now consider fixing it, or reverting actually, a feature request? Y'all need Jesus.

[timuralp]

Submitted a PR (boto/botocore#1228) in botocore to plumb the proxies option from Config to the Endpoint. Happy to iterate on it if this is desirable and if there is some feedback on the patch.

[opsroller]

Hmmm, export no_proxy="*.someservice.amazonaws.com"

[JordonPhillips]

Closing since @timuralp 's PR was merged. Feel free to open another issue if you have additional problems.