Unable to load smart_open.gcs

[andrewasheridan]

I have an issue similar to this : https://stackoverflow.com/questions/61126546/docker-compose-up-d-cannot-finish-because-an-error

Installation in docker works fine but when importing this error crops up.

The issue does not occur with 1.10, only with 1.11

Apologies for the total lack of details / reproducibility. I will try to add those tomorrow if possible.

[Andrew-Sheridan]

I apologize, my above description is just not at all correct. Boo me.

So the issue that I was having was that my tests for a CLI were failing during a build process. That happens in Docker and for some reason I thought that was the issue (it's not).

The test for the CLI was essentially a test to ensure it had installed correctly, that the entrypoint name was available, something like this:

    from subprocess import PIPE
    from subprocess import Popen
    process = Popen(["entrypoint-foo", "--help"], stdout=PIPE, stderr=PIPE)
    stdout, stderr = process.communicate()
    assert stdout.decode().startswith("CLI entrypoint for Foo")
    assert stderr == b""

Foo could essentially have just been a script with import smart_open as open

The line assert stderr == b"" used to work, but as of smart_open==1.11.0, it is failing because stderr is now equal to b"unable to import 'smart_open.gcs', disabling that module\n".

The real issue is just my sucky test.

I'm really happy with the work everyone has put into smart_open, and I saw the recent conversation around using extras_requires to help slim down installations - I will absolutely be moving towards pip install smart-open[aws] for Serverless Lambda installs.

In closing, I don't think there is really an issue here, besides my not-awesome testing of the my CLI.

Also I am definitely commenting using the wrong account but oh well. . Thanks again!

[mpenkov]

Thank you for the kind words.

I think if you check the exit value of the subprocess instead of its stderr, your script will be robust to issues like this.