Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build fails with Python 3.11 #144

Closed
davide125 opened this issue Nov 15, 2022 · 9 comments
Closed

Build fails with Python 3.11 #144

davide125 opened this issue Nov 15, 2022 · 9 comments
Assignees
@quark-zju
Labels
fix ready A fix was created. It's pending review or push.

Comments

@davide125
Copy link
Member

This currently fails to build on Fedora Rawhide, which uses Python 3.11

/builddir/build/BUILD/sapling-20221115-080554-34470671/eden/scm/build/traitlets-4.3.3-py2.py3-none-any/traitlets/config/loader.py:795: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if len(key) is 1:
/builddir/build/BUILD/sapling-20221115-080554-34470671/eden/scm/build/traitlets-4.3.3-py2.py3-none-any/traitlets/config/loader.py:804: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if len(key) is 1:
edenscm/cext/mpatch.c: In function 'cpygetitem':
edenscm/cext/mpatch.c:66:3: warning: 'PyObject_AsCharBuffer' is deprecated [-Wdeprecated-declarations]
   66 |   if (PyObject_AsCharBuffer(tmp, &buffer, (Py_ssize_t*)&blen))
      |   ^~
In file included from /usr/include/python3.11/Python.h:100,
                 from edenscm/cext/mpatch.c:31:
/usr/include/python3.11/abstract.h:326:17: note: declared here
  326 | PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj,
      |                 ^~~~~~~~~~~~~~~~~~~~~
edenscm/cext/mpatch.c: In function 'patches':
edenscm/cext/mpatch.c:94:3: warning: 'PyObject_AsCharBuffer' is deprecated [-Wdeprecated-declarations]
   94 |   if (PyObject_AsCharBuffer(text, &in, &inlen))
      |   ^~
/usr/include/python3.11/abstract.h:326:17: note: declared here
  326 | PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj,
      |                 ^~~~~~~~~~~~~~~~~~~~~
edenscm/cext/dirs.c: In function '_addpath':
edenscm/cext/dirs.c:79:7: warning: 'ob_shash' is deprecated [-Wdeprecated-declarations]
   79 |       ((PyBytesObject*)key)->ob_shash = -1;
      |       ^
In file included from /usr/include/python3.11/bytesobject.h:62,
                 from /usr/include/python3.11/Python.h:50,
                 from edenscm/cext/dirs.c:18:
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
edenscm/cext/dirs.c:88:18: error: lvalue required as left operand of assignment
   88 |     Py_SIZE(key) = pos;
      |                  ^
error: command '/usr/lib64/ccache/gcc' failed with exit code 1
make: *** [Makefile:75: oss] Error 1
@davide125
Copy link
Member Author

Updating the traitlets dependency would probably fix this.

@davide125
Copy link
Member Author

Nope, that was a red herring, this failure is due to sapling itself, not traitlets.

@davide125
Copy link
Member Author

Here's a better error trace (with make -j1):

building 'edenscmnative.mpatch' extension
gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I. -I../.. -I/usr/local/include -I/usr/include/python3.11 -c edenscm/cext/mpatch.c -o build/temp.linux-x86_64-3.11/edenscm/cext/mpatch.o
edenscm/cext/mpatch.c: In function 'cpygetitem':
edenscm/cext/mpatch.c:66:3: warning: 'PyObject_AsCharBuffer' is deprecated [-Wdeprecated-declarations]
   66 |   if (PyObject_AsCharBuffer(tmp, &buffer, (Py_ssize_t*)&blen))
      |   ^~
In file included from /usr/include/python3.11/Python.h:100,
                 from edenscm/cext/mpatch.c:31:
/usr/include/python3.11/abstract.h:326:17: note: declared here
  326 | PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj,
      |                 ^~~~~~~~~~~~~~~~~~~~~
edenscm/cext/mpatch.c: In function 'patches':
edenscm/cext/mpatch.c:94:3: warning: 'PyObject_AsCharBuffer' is deprecated [-Wdeprecated-declarations]
   94 |   if (PyObject_AsCharBuffer(text, &in, &inlen))
      |   ^~
/usr/include/python3.11/abstract.h:326:17: note: declared here
  326 | PyAPI_FUNC(int) PyObject_AsCharBuffer(PyObject *obj,
      |                 ^~~~~~~~~~~~~~~~~~~~~
gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I. -I../.. -I/usr/local/include -I/usr/include/python3.11 -c edenscm/mpatch.c -o build/temp.linux-x86_64-3.11/edenscm/mpatch.o
gcc -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection build/temp.linux-x86_64-3.11/edenscm/cext/mpatch.o build/temp.linux-x86_64-3.11/edenscm/mpatch.o -L/usr/lib64 -Lbuild/temp.linux-x86_64-3.11 -lmpatch -lchg -o /builddir/build/BUILD/sapling-20221115-080554-34470671/eden/scm/edenscmnative/mpatch.cpython-311-x86_64-linux-gnu.so
building 'edenscmnative.parsers' extension
gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I. -I../.. -I/usr/local/include -I/usr/include/python3.11 -c edenscm/cext/charencode.c -o build/temp.linux-x86_64-3.11/edenscm/cext/charencode.o
gcc -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -I. -I../.. -I/usr/local/include -I/usr/include/python3.11 -c edenscm/cext/dirs.c -o build/temp.linux-x86_64-3.11/edenscm/cext/dirs.o
edenscm/cext/dirs.c: In function '_addpath':
edenscm/cext/dirs.c:79:7: warning: 'ob_shash' is deprecated [-Wdeprecated-declarations]
   79 |       ((PyBytesObject*)key)->ob_shash = -1;
      |       ^
In file included from /usr/include/python3.11/bytesobject.h:62,
                 from /usr/include/python3.11/Python.h:50,
                 from edenscm/cext/dirs.c:18:
/usr/include/python3.11/cpython/bytesobject.h:7:35: note: declared here
    7 |     Py_DEPRECATED(3.11) Py_hash_t ob_shash;
      |                                   ^~~~~~~~
edenscm/cext/dirs.c:88:18: error: lvalue required as left operand of assignment
   88 |     Py_SIZE(key) = pos;
      |                  ^
error: command '/usr/lib64/ccache/gcc' failed with exit code 1
make: *** [Makefile:75: oss] Error 1

@davide125
Copy link
Member Author

I can confirm this builds fine on Fedora 36 (which uses Python 3.10), so it's specifically Python 3.11 incompatibility.

@safarmer
Copy link

Looks like the Py_SIZE API has changed.

From Python docs:

Py_ssize_t Py_SIZE(PyVarObject *o)
Get the size of the Python object o.

Use the Py_SET_SIZE() function to set an object size.

Changed in version 3.11: Py_SIZE() is changed to an inline static function. The parameter type is no longer const PyVarObject*.

I think this is the least of the problems with the internal changes to the Python frame object handling: python/cpython#88756

TL;DR: the traceprof related code needs changes to how PyFrameObject is used (no longer a complete struct/type in the currently used headers). I am not a Python expert so I am not sure how to approach this.

@quark-zju
Copy link
Contributor

Not resolved. There are other users of Py_SIZE.

@quark-zju quark-zju reopened this Nov 16, 2022
@leahneukirchen
Copy link

Fixing Py_SIZE, it goes on with different issues:

./edenscm/ext/extlib/traceprofimpl.cpp: In member function 'void FrameInfo::assign(PyFrameObject*, frameid_t)':
./edenscm/ext/extlib/traceprofimpl.cpp:52:17: error: invalid use of incomplete type 'PyFrameObject' {aka 'struct _frame'}
   52 |     code = frame->f_code;

later:

edenscmnative/traceprof.cpp: In function 'int __Pyx__GetException(PyThreadState*, PyObject**, PyObject**, PyObject**)':
edenscmnative/traceprof.cpp:3923:30: error: '_PyErr_StackItem' {aka 'struct _err_stackitem'} has no member named 'exc_type'
 3923 |         tmp_type = exc_info->exc_type;

@sggutier sggutier mentioned this issue Nov 21, 2022
Closed
6 tasks
@quark-zju quark-zju self-assigned this Nov 21, 2022
@sggutier sggutier mentioned this issue Nov 22, 2022
Open
@quark-zju quark-zju added the fix ready A fix was created. It's pending review or push. label Nov 22, 2022
facebook-github-bot pushed a commit that referenced this issue Nov 23, 2022
@quark-zju @facebook-github-bot
cext: drop pathencode C path
7a78bc7 
Summary:
`pathencode` is only used by legacy revlog repos currently in tests.
The C code does not compile with Python 3.11. Let's just drop the code with the
hope that we can drop pathencode entirely once we cleaned up the tests. The
feature still have a Python implementation so tests should still work.

The Python `lowerencode` logic was updated to work correctly.

Related: #144

Reviewed By: muirdm

Differential Revision: D41446882

fbshipit-source-id: 05b0ca1117750ca095d9bd833f8a977d1a6bf453
facebook-github-bot pushed a commit that referenced this issue Nov 23, 2022
@quark-zju @facebook-github-bot
setup: upgrade Cython to its latest version
3fc550c 
Summary:
The previous version generates code that does not compile with Python 3.11.
Related: #144

Reviewed By: sggutier

Differential Revision: D41447770

fbshipit-source-id: 1a58760a62a998d2f4b476fe29ee9a0bdcf95158
@jtagcat
Copy link

jtagcat commented Nov 24, 2022 •
edited
Loading

edit: Yep, works (after cleaning git worktree).

@leahneukirchen
Copy link

Worked for me after git clean -fdx, just make clean isn't enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@quark-zju quark-zju

Labels
fix ready A fix was created. It's pending review or push.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@leahneukirchen @davide125 @quark-zju @safarmer @jtagcat