Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Published Python SDK doesn't include the generated code #3596

Closed
forsberg opened this issue Aug 24, 2023 · 7 comments · Fixed by #3605
Closed

Published Python SDK doesn't include the generated code #3596

forsberg opened this issue Aug 24, 2023 · 7 comments · Fixed by #3605
Labels
area/compatibility area/rest-api area/ui type/bug Something isn't working

Comments

@forsberg
Copy link
Contributor

Description

Registry
Version: 2.4.4

Environment

Linux, Python 3.10

Steps to Reproduce

  1. Create a new virtualenv
  2. Try to pip install apicurioregistrysdk`
  3. Fail.
mkvirtualenv apicurio-registry-sdk
created virtual environment CPython3.10.12.final.0-64 in 148ms
  creator CPython3Posix(dest=/home/forsberg/.virtualenvs/apicurio-registry-sdk, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/forsberg/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.41.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
virtualenvwrapper.user_scripts creating /home/forsberg/.virtualenvs/apicurio-registry-sdk/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/forsberg/.virtualenvs/apicurio-registry-sdk/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/forsberg/.virtualenvs/apicurio-registry-sdk/bin/preactivate
virtualenvwrapper.user_scripts creating /home/forsberg/.virtualenvs/apicurio-registry-sdk/bin/postactivate
virtualenvwrapper.user_scripts creating /home/forsberg/.virtualenvs/apicurio-registry-sdk/bin/get_env_details
(apicurio-registry-sdk) $ pip install apicurioregistrysdk
Looking in indexes: https://pypi.org/simple
Collecting apicurioregistrysdk
  Using cached apicurioregistrysdk-2.4.4.tar.gz (2.4 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting microsoft-kiota-abstractions<0.7.0,>=0.6.0 (from apicurioregistrysdk)
  Obtaining dependency information for microsoft-kiota-abstractions<0.7.0,>=0.6.0 from https://files.pythonhosted.org/packages/d6/5a/8d810173b81051e978f82a2fb0d6821d0743088ce85f6371b0df0b396f38/microsoft_kiota_abstractions-0.6.0-py2.py3-none-any.whl.metadata
  Downloading microsoft_kiota_abstractions-0.6.0-py2.py3-none-any.whl.metadata (3.4 kB)
Collecting microsoft-kiota-http<0.5.0,>=0.4.4 (from apicurioregistrysdk)
  Obtaining dependency information for microsoft-kiota-http<0.5.0,>=0.4.4 from https://files.pythonhosted.org/packages/ee/6c/f7c2d04dde314da76addf71079d0da942a91dadce416a342508286e9523f/microsoft_kiota_http-0.4.4-py2.py3-none-any.whl.metadata
  Downloading microsoft_kiota_http-0.4.4-py2.py3-none-any.whl.metadata (3.3 kB)
Collecting microsoft-kiota-serialization-json<0.4.0,>=0.3.7 (from apicurioregistrysdk)
  Obtaining dependency information for microsoft-kiota-serialization-json<0.4.0,>=0.3.7 from https://files.pythonhosted.org/packages/c3/79/0d2a19e4936b7719d9cf5cc97e22d119fb515f2eb346c905696337a4f030/microsoft_kiota_serialization_json-0.3.7-py2.py3-none-any.whl.metadata
  Downloading microsoft_kiota_serialization_json-0.3.7-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting microsoft-kiota-serialization-text<0.3.0,>=0.2.1 (from apicurioregistrysdk)
  Obtaining dependency information for microsoft-kiota-serialization-text<0.3.0,>=0.2.1 from https://files.pythonhosted.org/packages/b4/41/ffa16e9a9b61182f1badc7fee861fbe2ddeb02b05743e6db377b88d4fd40/microsoft_kiota_serialization_text-0.2.1-py2.py3-none-any.whl.metadata
  Downloading microsoft_kiota_serialization_text-0.2.1-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting uritemplate>=4.1.1 (from microsoft-kiota-abstractions<0.7.0,>=0.6.0->apicurioregistrysdk)
  Using cached uritemplate-4.1.1-py2.py3-none-any.whl (10 kB)
Collecting httpx[http2]>=0.23.0 (from microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Obtaining dependency information for httpx[http2]>=0.23.0 from https://files.pythonhosted.org/packages/ec/91/e41f64f03d2a13aee7e8c819d82ee3aa7cdc484d18c0ae859742597d5aa0/httpx-0.24.1-py3-none-any.whl.metadata
  Using cached httpx-0.24.1-py3-none-any.whl.metadata (7.4 kB)
Collecting python-dateutil>=2.8.2 (from microsoft-kiota-serialization-json<0.4.0,>=0.3.7->apicurioregistrysdk)
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting certifi (from httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Obtaining dependency information for certifi from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
  Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting httpcore<0.18.0,>=0.15.0 (from httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Obtaining dependency information for httpcore<0.18.0,>=0.15.0 from https://files.pythonhosted.org/packages/94/2c/2bde7ff8dd2064395555220cbf7cba79991172bf5315a07eb3ac7688d9f1/httpcore-0.17.3-py3-none-any.whl.metadata
  Using cached httpcore-0.17.3-py3-none-any.whl.metadata (18 kB)
Collecting idna (from httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting sniffio (from httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Using cached sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting h2<5,>=3 (from httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Using cached h2-4.1.0-py3-none-any.whl (57 kB)
Collecting six>=1.5 (from python-dateutil>=2.8.2->microsoft-kiota-serialization-json<0.4.0,>=0.3.7->apicurioregistrysdk)
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting hyperframe<7,>=6.0 (from h2<5,>=3->httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Using cached hyperframe-6.0.1-py3-none-any.whl (12 kB)
Collecting hpack<5,>=4.0 (from h2<5,>=3->httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Using cached hpack-4.0.0-py3-none-any.whl (32 kB)
Collecting h11<0.15,>=0.13 (from httpcore<0.18.0,>=0.15.0->httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Using cached h11-0.14.0-py3-none-any.whl (58 kB)
Collecting anyio<5.0,>=3.0 (from httpcore<0.18.0,>=0.15.0->httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Obtaining dependency information for anyio<5.0,>=3.0 from https://files.pythonhosted.org/packages/19/24/44299477fe7dcc9cb58d0a57d5a7588d6af2ff403fdd2d47a246c91a3246/anyio-3.7.1-py3-none-any.whl.metadata
  Using cached anyio-3.7.1-py3-none-any.whl.metadata (4.7 kB)
Collecting exceptiongroup (from anyio<5.0,>=3.0->httpcore<0.18.0,>=0.15.0->httpx[http2]>=0.23.0->microsoft-kiota-http<0.5.0,>=0.4.4->apicurioregistrysdk)
  Obtaining dependency information for exceptiongroup from https://files.pythonhosted.org/packages/ad/83/b71e58666f156a39fb29417e4c8ca4bc7400c0dd4ed9e8842ab54dc8c344/exceptiongroup-1.1.3-py3-none-any.whl.metadata
  Downloading exceptiongroup-1.1.3-py3-none-any.whl.metadata (6.1 kB)
Using cached microsoft_kiota_abstractions-0.6.0-py2.py3-none-any.whl (33 kB)
Using cached microsoft_kiota_http-0.4.4-py2.py3-none-any.whl (21 kB)
Using cached microsoft_kiota_serialization_json-0.3.7-py2.py3-none-any.whl (10 kB)
Using cached microsoft_kiota_serialization_text-0.2.1-py2.py3-none-any.whl (8.5 kB)
Using cached httpcore-0.17.3-py3-none-any.whl (74 kB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached httpx-0.24.1-py3-none-any.whl (75 kB)
Using cached anyio-3.7.1-py3-none-any.whl (80 kB)
Downloading exceptiongroup-1.1.3-py3-none-any.whl (14 kB)
Building wheels for collected packages: apicurioregistrysdk
  Building wheel for apicurioregistrysdk (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for apicurioregistrysdk (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      Downloading Kiota from URL: https://github.com/microsoft/kiota/releases/download/v1.4.0/linux-x64.zip
      Traceback (most recent call last):
        File "/tmp/pip-install-7ssbyb1x/apicurioregistrysdk_a213301ae1d94386bc795a1ddac03389/kiota-gen.py", line 81, in <module>
          generate_kiota_client_files({})
        File "/tmp/pip-install-7ssbyb1x/apicurioregistrysdk_a213301ae1d94386bc795a1ddac03389/kiota-gen.py", line 53, in generate_kiota_client_files
          shutil.copyfile(
        File "/usr/lib/python3.10/shutil.py", line 254, in copyfile
          with open(src, 'rb') as fsrc:
      FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-install-7ssbyb1x/apicurioregistrysdk_a213301ae1d94386bc795a1ddac03389/../common/src/main/resources/META-INF/openapi.json'
      Traceback (most recent call last):
        File "/home/forsberg/.virtualenvs/apicurio-registry-sdk/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/forsberg/.virtualenvs/apicurio-registry-sdk/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/forsberg/.virtualenvs/apicurio-registry-sdk/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-kd0b0ndw/overlay/lib/python3.10/site-packages/poetry/core/masonry/api.py", line 57, in build_wheel
          return WheelBuilder.make_in(
        File "/tmp/pip-build-env-kd0b0ndw/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 87, in make_in
          wb.build(target_dir=directory)
        File "/tmp/pip-build-env-kd0b0ndw/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 122, in build
          self._build(zip_file)
        File "/tmp/pip-build-env-kd0b0ndw/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 171, in _build
          self._run_build_script(self._package.build_script)
        File "/tmp/pip-build-env-kd0b0ndw/overlay/lib/python3.10/site-packages/poetry/core/masonry/builders/wheel.py", line 231, in _run_build_script
          subprocess.check_call([self.executable.as_posix(), build_script])
        File "/usr/lib/python3.10/subprocess.py", line 369, in check_call
          raise CalledProcessError(retcode, cmd)
      subprocess.CalledProcessError: Command '['/home/forsberg/.virtualenvs/apicurio-registry-sdk/bin/python', 'kiota-gen.py']' returned non-zero exit status 1.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for apicurioregistrysdk
Failed to build apicurioregistrysdk
ERROR: Could not build wheels for apicurioregistrysdk, which is required to install pyproject.toml-based projects

Inspecting the .tar.gz downloaded by pip:

$ tar -zxvf apicurioregistrysdk-2.4.4.tar.gz 
apicurioregistrysdk-2.4.4/README.md
apicurioregistrysdk-2.4.4/apicurioregistrysdk/__init__.py
apicurioregistrysdk-2.4.4/kiota-gen.py
apicurioregistrysdk-2.4.4/pyproject.toml
apicurioregistrysdk-2.4.4/PKG-INFO

..the likely cause is that there simply is no openapi.json included.

Suggested solutions:

  1. Ensure openapi.json is included in source distribution.
  2. Build a pure python wheel instead of a binary one. This seems not entirely easy in poetry according to Creating pure Python wheels when using build scripts python-poetry/poetry#3594

@andreaTP
@forsberg forsberg added the type/bug Something isn't working label Aug 24, 2023
@apicurio-bot
Copy link

apicurio-bot bot commented Aug 24, 2023

Thank you for reporting an issue!

Pinging @jsenko, @EricWittmann to respond or triage.

@andreaTP
Copy link
Member

Hi @forsberg , thanks for this report!

I'm on PTO till Mon, but I'm a bit surprised by the error you see, as the kiota-gen script should copy locally the OpenApi.json before packaging the distribution.

Anyhow, either I'll have a look on Mon or, if you already see where the error is, I'll review your PR.
The logic should be to copy the OpenApi.json locally if not already present and include it in the distribution.(maybe an inclusion/exclusion issue with poetry?)

@forsberg
Copy link
Contributor Author

forsberg commented Aug 25, 2023 •
edited
Loading

Agree it's a bit mysterious, as when I try to repeat it locally, the .tar.gz built has all the generated code + openapi.json.

Yet, if you download the .tar.gz listed under https://pypi.org/project/apicurioregistrysdk/#files, it's very small and only contains the 5 files listed above.

I would suspect some kind of difference in how a local checkout looks vs. how it looks when run in the CI/CD system.

Note: I get a proper .tar.gz when I run poetry build, but CI seems to run poetry install. Not sure it matters.

@andreaTP
Copy link
Member

I just checked, and quite a few things are off in the package automatically published 😢 .
Thanks for reporting, I'm on it.

@andreaTP andreaTP changed the title Python SDK downloaded via pip does not build on python 3.10 Published Python SDK doesn't include the generated code Aug 28, 2023
This was referenced Aug 28, 2023
Merged
Open
@andreaTP
Copy link
Member

@forsberg this published release is good to go:
https://pypi.org/project/apicurioregistrysdk/2.4.5/#files

would you mind confirming?

@forsberg
Copy link
Contributor Author

Tried installing it in a python 3.10 virtualenv, works as a charm.

@andreaTP
Copy link
Member

Thanks for the feedback as always @forsberg 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/compatibility area/rest-api area/ui type/bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix publishing of the Python SDK andreaTP/apicurio-registry
2 participants
@forsberg @andreaTP