Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generated protobuf files with protobuf 3.12.4 on Ubuntu 21.04 #104

Merged

Conversation

baziorek
Copy link

@baziorek baziorek commented Jul 7, 2022

Some time ago I updated python files from protobufs (*.proto) files. It can be done easy with attached scripts:

python3 download-schema.py
python3 scripts/compile-proto.py

Everything was working, I was happy and the iroha-python 1.4.1 was released. Then in our servers with Ubuntu 21.04 I tried to use iroha 1.4.1 instead of iroha 1.0.0, but unfortunately I noticed errors described here: #102

After some investigation I found why it is not working - I have Manjaro, which has almost always newest packages, so generated protobufs was generated with really new version (currently I have protoc --version libprotoc 3.21.1). Everything was working in my OS - Manjaro. But debian-based servers usually have not everything as new as Manjaro, so those files was not compatible with versions in Ubuntu 21.04 (which has protoc 3.12.4).


What is done in the PR is just generating python files from *.proto in Ubuntu 20.04 LTS:

python3 download-schema.py
python3 scripts/compile-proto.py

Tests:

  1. pip3 install iroha==1.0.0 && python3 -c 'import iroha' - works OK, but we want new version
pip3 install iroha==1.4.1 && python3 -c 'import iroha'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/devops/.local/lib/python3.9/site-packages/iroha/__init__.py", line 7, in <module>
    from .iroha import *
  File "/home/devops/.local/lib/python3.9/site-packages/iroha/iroha.py", line 17, in <module>
    from . import endpoint_pb2
  File "/home/devops/.local/lib/python3.9/site-packages/iroha/endpoint_pb2.py", line 38, in <module>
    _descriptor.EnumValueDescriptor(
  File "/home/devops/.local/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
  1. pip3 install iroha==1.4.1 && PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python python3 -c 'import iroha' - works OK, but is may looks strange for new users
  2. pip3 install "git+https://github.com/baziorek/iroha-python.git@generate_protobufs_again" && python3 -c 'import iroha' - works OK

Summary:

So IMO we should downgrade generated protobuf files to be in lower version (I've generated in Manjaro which has protoc --version libprotoc 3.21.1), but rather users would have earlier versions (currently Ubuntu 21.04 has protobuf libprotoc 3.12.4). For users would be easier if they don't need extra flag to run simplest example PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python.

@baziorek baziorek added bug dependencies Pull requests that update a dependency file iroha1 labels Jul 7, 2022
@baziorek baziorek self-assigned this Jul 7, 2022
@baziorek baziorek marked this pull request as ready for review July 18, 2022 14:52
@baziorek baziorek merged commit 30b76e9 into hyperledger-iroha:develop Jul 27, 2022
@baziorek baziorek deleted the generate_protobufs_again branch July 27, 2022 10:11
baziorek added a commit to baziorek/iroha-python that referenced this pull request Aug 8, 2022
baziorek added a commit that referenced this pull request Aug 22, 2022
* Extending TxPaginationMeta in queries (#77)

* quries extended
* example added
* docs comment added
* schema changed
* formatting fixed
* ordering added
* edge case fix, now 0 passed as height or timestamp is passing
* PaginationMeta in GetPendingTransactions
* examples/query_transactions.py

Signed-off-by: Piotr Pawlowski <[email protected]>
Signed-off-by: G.Bazior <[email protected]>

* Update proto files and generated python files to support Iroha 1.4 (#96)

* Update proto files with script `download-schema.py`

Signed-off-by: Grzegorz Bazior <[email protected]>

* Generated python files from protobuf files with script `compile-proto.py`

Signed-off-by: G.Bazior <[email protected]>

Co-authored-by: Grzegorz Bazior <[email protected]>
Signed-off-by: G.Bazior <[email protected]>

* Corrected merge - rerun scripts: download-schema.py and compile-proto.py

Signed-off-by: G.Bazior <[email protected]>

* Tab -> spaces

Signed-off-by: G.Bazior <[email protected]>

* Add ability to provide custom TLS cert (#63)

Signed-off-by: Stepan Lavrentev <[email protected]>
Signed-off-by: G.Bazior <[email protected]>

* Add ordering sequence to params in query (#73)

* Add ordering sequence to params

Makes possible to add ordering sequence to Query. Fixes #72

Signed-off-by: Rafik Naccache <[email protected]>

* Add :param ordering_sequence:

Signed-off-by: Rafik Naccache <[email protected]>

* Fix wrong example for :param ordering_sequence:

Signed-off-by: Rafik Naccache <[email protected]>

* fix wrong ordering message construction in query

After more documentation I got to the way to construct to ordering object

Signed-off-by: Rafik Naccache <[email protected]>

* remove redundant OR clause to manage ordering_sequence

Signed-off-by: Rafik Naccache <[email protected]>
Signed-off-by: G.Bazior <[email protected]>

* Extending TxPaginationMeta in queries (#77)

* quries extended
* example added
* docs comment added
* schema changed
* formatting fixed
* ordering added
* edge case fix, now 0 passed as height or timestamp is passing
* PaginationMeta in GetPendingTransactions
* examples/query_transactions.py

Signed-off-by: Piotr Pawlowski <[email protected]>
Signed-off-by: G.Bazior <[email protected]>

* Added missing changes from Pawlak00@b6d7f42 corrected in #77

Signed-off-by: G.Bazior <[email protected]>

* Examples refactor + added example of MST developed by Leo (#103)

* Refactoring of example tx-example.py

Signed-off-by: G.Bazior <[email protected]>

* Added sample of MST transactions. The code was originally developed by Leo: https://github.com/iptelephony/jubilant-engine but he agreed to add his code to examples

Signed-off-by: G.Bazior <[email protected]>

* Generated protobuf files with protobuf 3.12.4 on Ubuntu 21.04 (#104)

Signed-off-by: G.Bazior <[email protected]>

* Serious refactoring + improvements + adding new examples for Iroha 1 (#107)

* Update README
* Refactoring in tls-example.py
* A little refactoring of batch-example.py
* Small refactoring of blocks-query.py
* Small refactoring of infinite-blocks-stream.py
* Added sample how to use ordering of result transactions
* Refacroting in query_transactions.py
* Renamed file with convention used in other files
* Added examples in comments
* Some refactoring in examples + added docstring in each file
* chmod +x
* Updated iroha version from 1.4 -> 1.5

Signed-off-by: G.Bazior <[email protected]>

* Add autorestart to iroha

Signed-off-by: Alexey Rodionov <[email protected]>

* Cherry-pick fix

Signed-off-by: G.Bazior <[email protected]>

* Generated python files from protobuf on Ubuntu:20.04

Signed-off-by: G.Bazior <[email protected]>

* Dependencies fix

Signed-off-by: G.Bazior <[email protected]>

* Added space in logging after review

Signed-off-by: G.Bazior <[email protected]>

* Made import in code more readable after review

Signed-off-by: G.Bazior <[email protected]>

Signed-off-by: Piotr Pawlowski <[email protected]>
Signed-off-by: G.Bazior <[email protected]>
Signed-off-by: Stepan Lavrentev <[email protected]>
Signed-off-by: Rafik Naccache <[email protected]>
Signed-off-by: Alexey Rodionov <[email protected]>
Co-authored-by: Piotr Pawłowski <[email protected]>
Co-authored-by: Grzegorz Bazior <[email protected]>
Co-authored-by: Stepan Lavrentev <[email protected]>
Co-authored-by: Rafik NACCACHE <[email protected]>
Co-authored-by: Alexey Rodionov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug dependencies Pull requests that update a dependency file iroha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants