Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syntax Error when try to upload to S3 #223

Closed
hyphen1370 opened this issue Aug 29, 2018 · 5 comments
Closed

Syntax Error when try to upload to S3 #223

hyphen1370 opened this issue Aug 29, 2018 · 5 comments

Comments

@hyphen1370
Copy link

hyphen1370 commented Aug 29, 2018

I try to upload a file from Twilio URL to S3.

Here is the code I am using:

smart_open_s3_path = 's3://' + s3_bucket + '/' + s3_key_path
                        smart_open_s3_path_credential = 's3://' + aws_access_key + ':' + aws_SecretAccess_key + '@' + s3_bucket + '/' + s3_key_path
                        print(smart_open_s3_path_credential)

                        smart_url_object = smart_open.smart_open(media_url)
                        ##parsed_uri = smart_open.smart_open_lib._parse_uri(smart_open_s3_path_credential)
                        
                        with smart_open.smart_open(smart_open_s3_path_credential, 'wb') as fout:
                            for line in smart_url_object:
                                fout.write(line)

It always gives me syntax error:

invalid syntax (_base.py, line 414): SyntaxError
Traceback (most recent call last):
File "/var/task/index.py", line 118, in lambda_handler
with smart_open.smart_open(smart_open_s3_path_credential, 'wb') as fout:
File "/var/task/smart_open/smart_open_lib.py", line 231, in smart_open
binary, filename = _open_binary_stream(uri, binary_mode, **kw)
File "/var/task/smart_open/smart_open_lib.py", line 321, in _open_binary_stream
return _s3_open_uri(parsed_uri, mode, **kw), filename
File "/var/task/smart_open/smart_open_lib.py", line 383, in _s3_open_uri
return smart_open_s3.open(parsed_uri.bucket_id, parsed_uri.key_id, mode, **kwargs)
File "/var/task/smart_open/s3.py", line 73, in open
fileobj = BufferedOutputBase(bucket_id, key_id, min_part_size=s3_min_part_size, **kwargs)
File "/var/task/smart_open/s3.py", line 357, in __init__
s3 = session.resource('s3', **kwargs)
File "/var/task/boto3/session.py", line 389, in resource
aws_session_token=aws_session_token, config=config)
File "/var/task/boto3/session.py", line 263, in client
aws_session_token=aws_session_token, config=config)
File "/var/task/botocore/session.py", line 885, in create_client
client_config=config, api_version=api_version)
File "/var/task/botocore/client.py", line 70, in create_client
cls = self._create_client_class(service_name, service_model)
File "/var/task/botocore/client.py", line 95, in _create_client_class
base_classes=bases)
File "/var/task/botocore/hooks.py", line 227, in emit
return self._emit(event_name, kwargs)
File "/var/task/botocore/hooks.py", line 360, in _emit
aliased_event_name, kwargs, stop_on_response
File "/var/task/botocore/hooks.py", line 210, in _emit
response = handler(**kwargs)
File "/var/task/boto3/utils.py", line 61, in _handler
module = import_module(module)
File "/var/task/boto3/utils.py", line 52, in import_module
__import__(name)
File "/var/task/boto3/s3/inject.py", line 15, in <module>
from boto3.s3.transfer import create_transfer_manager
File "/var/task/boto3/s3/transfer.py", line 127, in <module>
from s3transfer.exceptions import RetriesExceededError as \
File "/var/task/s3transfer/__init__.py", line 134, in <module>
import concurrent.futures
File "/var/task/concurrent/futures/__init__.py", line 8, in <module>
from concurrent.futures._base import (FIRST_COMPLETED,
File "/var/task/concurrent/futures/_base.py", line 414
raise exception_type, self._exception, self._traceback
^
SyntaxError: invalid syntax

The S3 URI should be correct since I follow the instruction, any clues?

Example S3 URI:
s3://:@omax-mis/twilio-messages-media/final/MEcd7c36e75f87dc6dd9e33702cdcd8fb6

Thanks

@mpenkov
Copy link
Collaborator

mpenkov commented Aug 29, 2018

This is fixed in #204, waiting to merge.

@hyphen1370
Copy link
Author

Do you want when it merge will happen? After that, could I do pip update to update the version?

@mpenkov
Copy link
Collaborator

mpenkov commented Aug 29, 2018

We will merge soon. You can pip update once we do a release.

@menshikh-iv When are we planning the next release?

@menshikh-iv
Copy link
Contributor

@mpenkov let's finish #204 and I'll make bugfix release after it (weekend or next week)

@mpenkov
Copy link
Collaborator

mpenkov commented Sep 2, 2018

Closed via #224

@mpenkov mpenkov closed this as completed Sep 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants