Skip to content

Commit

Permalink
Fix bug where opening files in GCS forgets to define a name field (#506)
Browse files Browse the repository at this point in the history
* Fix bug where opening files in GCS forgets to define a name field

* update unit tests to check .name

* add round trip test

* Update CHANGELOG.md

Co-authored-by: Michael Penkov <[email protected]>
Co-authored-by: Michael Penkov <[email protected]>
  • Loading branch information
3 people authored Jun 17, 2020
1 parent 0a36191 commit bac9ab0
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 2 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

- Correctly pass `newline` parameter to built-in `open` function (PR [#478](https://github.com/RaRe-Technologies/smart_open/pull/478), [@burkovae](https://github.com/burkovae))
- Azure storage blob support ([@nclsmitchell](https://github.com/nclsmitchell) and [@petedannemann](https://github.com/petedannemann))
- Ensure GCS objects always have a .name attribute (PR [#506](https://github.com/RaRe-Technologies/smart_open/pull/506), [@todor-markov](https://github.com/todor-markov))

# 2.0.0, 27 April 2020, "Python 3"

Expand Down
7 changes: 5 additions & 2 deletions smart_open/gcs.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,15 +129,15 @@ def open(
"""
if mode == constants.READ_BINARY:
return Reader(
fileobj = Reader(
bucket_id,
blob_id,
buffer_size=buffer_size,
line_terminator=constants.BINARY_NEWLINE,
client=client,
)
elif mode == constants.WRITE_BINARY:
return Writer(
fileobj = Writer(
bucket_id,
blob_id,
min_part_size=min_part_size,
Expand All @@ -146,6 +146,9 @@ def open(
else:
raise NotImplementedError('GCS support for mode %r not implemented' % mode)

fileobj.name = blob_id
return fileobj


class _RawReader(object):
"""Read an GCS object."""
Expand Down
13 changes: 13 additions & 0 deletions smart_open/tests/test_gcs.py
Original file line number Diff line number Diff line change
Expand Up @@ -888,13 +888,26 @@ def test_read_never_returns_none(self):
"""read should never return None."""
test_string = u"ветер по морю гуляет..."
with smart_open.gcs.open(BUCKET_NAME, BLOB_NAME, "wb") as fout:
self.assertEqual(fout.name, BLOB_NAME)
fout.write(test_string.encode('utf8'))

r = smart_open.gcs.open(BUCKET_NAME, BLOB_NAME, "rb")
self.assertEqual(r.name, BLOB_NAME)
self.assertEqual(r.read(), test_string.encode("utf-8"))
self.assertEqual(r.read(), b"")
self.assertEqual(r.read(), b"")

def test_round_trip(self):
test_string = u"ветер по морю гуляет..."
url = 'gs://%s/%s' % (BUCKET_NAME, BLOB_NAME)
with smart_open.open(url, "w") as fout:
fout.write(test_string)

with smart_open.open(url) as fin:
actual = fin.read()

self.assertEqual(test_string, actual)


class MakeRangeStringTest(unittest.TestCase):
def test_no_stop(self):
Expand Down

0 comments on commit bac9ab0

Please sign in to comment.