Split up _testcapimodule.c #93649

encukou · 2022-06-09T15:53:22Z

Modules/_testcapimodule.c is a nearly-8000-line behemoth with no clear structure or organization. It is getting hard to maintain.

It also doesn't work well with testing (a) feature macros that affect Python.h and (b) module initialization, so we have additional C-API testing modules: _testmultiphase, _testimportmultiple, _testinternalcapi.
_testbuffer is already split out, but there are many other aspects of the API that would use a similar dedicated test suite.

We should split and combine these, ideally without polluting the namespace of top-level modules.

PR: gh-93649: Split vectorcall testing from _testcapimodule.c #94549

PR: gh-93649: Fix dependencies of _testcapi #94695

PR: gh-93649: Fix linkage of _PyTestCapi_Init_Vectorcall #94702

PR: gh-93649: Split heaptype tests from _testcapimodule.c #95386

PR: gh-93649: Undefine NDEBUG in Modules/_testcapi/* to enable assert() #95793

PR: gh-93649: Split unicode tests from _testcapimodule.c & add some more #95819

PR: gh-93649: Split getargs testing from _testcapimodule.c #99346

PR: gh-93649: Split pytime and datetime tests from _testcapimodule.c #99494

PR: gh-93649: Split memory and docstring tests from _testcapimodule.c #99517

PR: gh-93649: Split watcher API tests from _testcapimodule.c #99532

PR: gh-93649: Split float/long tests from _testcapimodule.c #99549

PR: gh-93649: Split tracemalloc tests from _testcapimodule.c #99551

PR: gh-93649: Split exception tests from _testcapimodule.c #102173

PR: gh-93649: Split gc- and allocation tests from _testcapimodule.c #104403

PR: gh-93649: Split _testcapimodule.c into smaller files #129400

PR: gh-93649: Move PyFrame C API tests to test_capi #129512

PR: gh-93649: Add Modules/_testcapi/type.c file #129516

PR: gh-93649: Add Modules/_testcapi/function.c file #129521

PR: gh-93649: Move _testcapi tests to specific files #129544

The text was updated successfully, but these errors were encountered:

arhadthedev · 2022-06-09T18:14:02Z

ideally without polluting the namespace of top-level modules

Can hypothetical Lib/_testcapimodule/*.c export their objects as test.capi.*?

encukou · 2022-06-10T08:59:17Z

As far as I know they can, as long as test.capi itself is not an extension module. It would need some extra support in both make and Windows builds, though.

The _testcapimodule.c file is getting too large to work with effectively. Vectorcall tests aren't the biggest issue -- it's just an area I want to work on next, so I started there. It does make it clear that MethodDescriptor2 is related to testing vectorcall, which wasn't clear before (the /* Test PEP 590 */ section had an ambiguous end). This PR lays out a general structure of how tests can be split up, with more splitting to come later if the structure is OK.

encukou · 2022-07-04T16:45:07Z

PR: #94549

vstinner · 2022-07-05T11:18:46Z

I tried to group de 299 attributes of the _testcapi extension module. I'm not
sure about each attribute, I only looked quickly at the name, sometimes at the
implementation. But it should give a coarse idea of which "groups" make sense
or not.

Python/getargs.c and Python/modsupport.c (parse and build arguments) (51):

parse_tuple_and_keywords
getargs_B
getargs_C
getargs_D
getargs_H
getargs_I
getargs_K
getargs_L
getargs_S
getargs_U
getargs_Y
getargs_Z
getargs_Z_hash
getargs_b
getargs_c
getargs_d
getargs_es
getargs_es_hash
getargs_et
getargs_et_hash
getargs_f
getargs_h
getargs_i
getargs_k
getargs_keyword_only
getargs_keywords
getargs_l
getargs_n
getargs_p
getargs_positional_only_and_keywords
getargs_s
getargs_s_hash
getargs_s_hash_int
getargs_s_star
getargs_tuple
getargs_u
getargs_u_hash
getargs_w_star
getargs_y
getargs_y_hash
getargs_y_star
getargs_z
getargs_z_hash
getargs_z_star
test_L_code
test_k_code
test_s_code
test_buildvalue_N
test_buildvalue_issue38913
test_empty_argparse
argparsing

datetime, pytime.c (33):

PyDateTime_DATE_GET
PyDateTime_DELTA_GET
PyDateTime_GET
PyDateTime_TIME_GET
PyTime_AsMicroseconds
PyTime_AsMilliseconds
PyTime_AsSecondsDouble
PyTime_AsTimespec
PyTime_AsTimespec_clamp
PyTime_AsTimeval
PyTime_AsTimeval_clamp
PyTime_FromSeconds
PyTime_FromSecondsObject
datetime_check_date
datetime_check_datetime
datetime_check_delta
datetime_check_time
datetime_check_tzinfo
get_date_fromdate
get_date_fromtimestamp
get_datetime_fromdateandtime
get_datetime_fromdateandtimeandfold
get_datetime_fromtimestamp
get_delta_fromdsu
get_time_fromtime
get_time_fromtimeandfold
get_timezone_utc_capi
get_timezones_offset_zero
make_timezones_capi
pytime_object_to_time_t
pytime_object_to_timespec
pytime_object_to_timeval
test_datetime_capi

Types, metaclasses, type inheritance (wide category, not sure if it makes sense) (30):

_test_structmembersType
awaitType
instancemethod
ipowType
create_type_from_repeated_slots
matmulType
NullTpDocType
Generic
GenericAlias
MyList
ContainerNoGC
HeapCTypeMetaclass
HeapCTypeMetaclassCustomNew
HeapCTypeSetattr
HeapCTypeSubclass
HeapCTypeSubclassWithFinalizer
HeapCTypeWithBuffer
HeapCTypeWithDict
HeapCTypeWithNegativeDict
HeapCTypeWithWeakref
HeapDocCType
HeapGcCType
test_from_spec_invalid_metatype_inheritance
pytype_fromspec_meta
test_from_spec_metatype_inheritance
test_get_statictype_slots
test_get_type_name
test_get_type_qualname
test_lazy_hash_inheritance
test_type_from_ephemeral_spec

Function calls and spawn threads to call functions (25):

_test_thread_state
call_in_temporary_c_thread
get_kwargs
get_args
create_cfunction
pyobject_vectorcall
pyvectorcall_call
pyobject_fastcall
pyobject_fastcalldict
meth_fastcall
meth_fastcall_keywords
meth_noargs
meth_o
meth_varargs
meth_varargs_keywords
MethClass
MethInstance
MethStatic
MethodDescriptor2
MethodDescriptorBase
MethodDescriptorDerived
MethodDescriptorNopGet
return_null_without_error
return_result_with_error
stack_pointer

Unicode, codecs (12):

codec_incrementaldecoder
codec_incrementalencoder
test_unicode_compare_with_ascii
test_widechar
test_string_from_format
unicode_asucs4
unicode_asutf8
unicode_asutf8andsize
unicode_aswidechar
unicode_aswidecharstring
unicode_copycharacters
unicode_findchar

Memory allocators (17):

WITH_PYMALLOC
pyobject_malloc_without_gil
pymem_api_misuse
pymem_buffer_overflow
pymem_getallocatorsname
pymem_malloc_without_gil
test_pymem_alloc0
test_pymem_setallocators
test_pymem_setrawallocators
test_pyobject_new
test_pyobject_setallocators
set_nomemory
check_pyobject_forbidden_bytes_is_freed
check_pyobject_freed_is_freed
check_pyobject_null_is_freed
check_pyobject_uninitialized_is_freed
remove_mem_hooks

Exceptions, "error handling", C "errno" variable and signals (13):

RecursingInfinitelyError
raise_memoryerror
make_exception_with_doc
write_unraisable_exc
traceback_print
set_exception
raise_exception
set_exc_info
exception_print
fatal_error
error
set_errno
raise_SIGINT_then_send_None

C types limit and size (20):

CHAR_MAX
CHAR_MIN
INT_MAX
INT_MIN
LLONG_MAX
LLONG_MIN
LONG_MAX
LONG_MIN
PY_SSIZE_T_MAX
PY_SSIZE_T_MIN
SHRT_MAX
SHRT_MIN
UCHAR_MAX
UINT_MAX
ULLONG_MAX
ULONG_MAX
USHRT_MAX
SIZEOF_TIME_T
test_config
test_sizeof_c_types

Float, PyLongObject and PyNumber C API (16):

test_long_and_overflow
test_long_api
test_long_as_size_t
test_long_as_unsigned_long_long_mask
test_long_long_and_overflow
test_long_numbits
test_longlong_api
pynumber_tobase
DBL_MAX
DBL_MIN
FLT_MAX
FLT_MIN
float_pack
float_unpack
test_string_to_double
test_long_as_double

Docstring (10):

docstring_empty
docstring_no_signature
docstring_with_invalid_signature
docstring_with_invalid_signature2
docstring_with_signature
docstring_with_signature_and_extra_newlines
docstring_with_signature_but_no_doc
docstring_with_signature_with_defaults
no_docstring
test_with_docstring

Garbage collector (3):

test_gc_control
without_gc
with_tp_del

Tracemalloc (3):

tracemalloc_get_traceback
tracemalloc_track
tracemalloc_untrack

os module helpers (1):

W_STOPCODE

Subinterpreters (1):

run_in_subinterp

C API (57):

PyObject and PyVarObject C API (9):
- negative_refcount
- test_decref_doesnt_leak
- test_incref_decref_API
- test_incref_doesnt_leak
- test_refcount_funcs
- test_refcount_macros
- test_xdecref_doesnt_leak
- test_xincref_doesnt_leak
- test_set_type_size
PyObject_Bytes(), PyObject_Repr(), PyObject_Str() (3):
- pyobject_bytes_from_null
- pyobject_repr_from_null
- pyobject_str_from_null
PyTypeObject C API (2):
- negative_dictoffset
- type_get_version
C API macros (3):
- test_macros
- test_py_is_funcs
- test_py_is_macros
PyCodeObject C API (2):
- test_code_api
- code_newempty
PyFrameObject C API (5):
- frame_getbuiltins
- frame_getgenerator
- frame_getglobals
- frame_getlasti
- frame_getlocals
PyDictObject C API (3):
- dict_get_version
- dict_getitem_knownhash
- test_dict_iteration
PyThreadState C API (2):
- test_tstate_capi
- crash_no_current_thread
PyListObject C API (1):
- test_list_api
PySequence and PyMapping C API, PyObject_GetItem() (6):
- sequence_getitem
- get_mapping_items
- get_mapping_keys
- get_mapping_values
- getitem_with_error
- bad_get
C API "feature" macros like HAVE_FORK (1):
- get_feature_macros
HAMT C API (1):
- hamt
PyBuffer and memoryview C API (5):
- getbuffer_with_null_view
- make_memoryview_from_NULL_pointer
- test_from_contiguous
- PyBuffer_SizeFromFormat
- test_pep3118_obsolete_write_locks
PyCapsule C API (1):
- test_capsule
PyThread_tss C API (1):
- test_pythread_tss_key_state
PyStructSeq C API (2):
- test_structseq_newtype_doesnt_leak
- test_structseq_newtype_null_descr_doc
PyEval_SetTrace C API (1):
- settrace_to_record
PyMarshal C API (6):
- pymarshal_read_last_object_from_file
- pymarshal_read_long_from_file
- pymarshal_read_object_from_file
- pymarshal_read_short_from_file
- pymarshal_write_long_to_file
- pymarshal_write_object_to_file
Py_AddPendingCall C API (1):
- _pending_threadfunc
Misc C API (2):
- Py_Version
- Py_CompileString

Misc (7):

__doc__
__file__
__loader__
__name__
__package__
__spec__
the_number_three

vstinner · 2022-07-05T11:23:06Z

It might be interesting to split the large _testcapi module into multiple modules. But I like the idea of starting by splitting the long C file into multiple C files.

@vstinner

The `_testcapimodule.c` file is getting too large to work with effectively. This PR lays out a general structure of how tests can be split up, with more splitting to come later if the structure is OK. Vectorcall tests aren't the biggest issue -- it's just an area I want to work on next, so I'm starting here. An issue specific to vectorcall tests is that it wasn't clear that e.g. `MethodDescriptor2` is related to testing vectorcall: the `/* Test PEP 590 */` section had an ambiguous end. Separate file should make things like this much clearer. OTOH, for some pieces it might not be clear where they should be -- I left `meth_fastcall` with tests of the other calling conventions. IMO, even with the ambiguity it's still worth it to split the huge file up. I'm not sure about the buildsystem changes, hopefully CI will tell me what's wrong. @vstinner, @markshannon: Do you think this is a good idea? Automerge-Triggered-By: GH:encukou

tiran · 2022-07-08T16:37:53Z

You do not need to include _testcapi/vectorcall.c in MODULE__TESTCAPI_DEPS. It is already listed as dependency for _testcapi in the Setup file.

- header files are located in $(srcdir) - dependencies must not list C files that are also in a makesetup Setup file

- header files are located in $(srcdir) - dependencies must not list C files that are also in a makesetup Setup file - generate SRCDIRS for OOT builds

encukou · 2022-07-11T14:27:18Z

Thank you for the fix! I'll 🔨 test-with-buildbots next time I touch the buildsystem.
(I wonder why this issue didn't get the broken buildbot notification, though...)

tiran · 2022-07-11T14:33:18Z

Thank you for the fix! I'll hammer test-with-buildbots next time I touch the buildsystem. (I wonder why this issue didn't get the broken buildbot notification, though...)

We think that there was a problem in the buildbot hook, python/buildmaster-config#333

This removes the unused negative_dictoffset function: the type this function would create is available as _testcapi.HeapCTypeWithNegativeDict

philg314 · 2022-08-01T18:06:37Z

How should raiseTestError be handled?

cpython/Modules/_testcapimodule.c

Lines 52 to 62 in c7e5bba

    
           static PyObject *TestError;     /* set to exception object in init */ 
        
           /* Raise TestError with test_name + ": " + msg, and return NULL. */ 
        
           static PyObject * 
        
           raiseTestError(const char* test_name, const char* msg) 
        
           { 
        
               PyErr_Format(TestError, "%s: %s", test_name, msg); 
        
               return NULL; 
        
           }

~~Edit: For the Unicode tests I copied it and renamed it to _testcapi.unicode_error.~~

* main: pythongh-91896: Fixup some docs issues following ByteString deprecation (python#104422) pythonGH-104371: check return value of calling `mv.release` (python#104417) pythongh-104415: Fix refleak tests for `typing.ByteString` deprecation (python#104416) pythonGH-86275: Implementation of hypothesis stubs for property-based tests, with zoneinfo tests (python#22863) pythonGH-103082: Filter LINE events in VM, to simplify tool implementation. (pythonGH-104387) pythongh-93649: Split gc- and allocation tests from _testcapimodule.c (pythonGH-104403) pythongh-104389: Add 'unused' keyword to Argument Clinic C converters (python#104390) pythongh-101819: Prepare _io._IOBase for module state (python#104386) pythongh-104413: Fix refleak when super attribute throws AttributeError (python#104414) Fix refleak in `super_descr_get` (python#104408) pythongh-87526: Remove dead initialization from _zoneinfo parse_abbr() (python#24700) pythongh-91896: Improve visibility of `ByteString` deprecation warnings (python#104294) pythongh-104371: Fix calls to `__release_buffer__` while an exception is active (python#104378) pythongh-104377: fix cell in comprehension that is free in outer scope (python#104394) pythongh-104392: Remove _paramspec_tvars from typing (python#104393) pythongh-104396: uuid.py to skip platform check for emscripten and wasi (pythongh-104397) pythongh-99108: Refresh HACL* from upstream (python#104401) pythongh-104301: Allow leading whitespace in disambiguated pdb statements (python#104342)

vstinner · 2023-08-22T23:16:33Z

_testcapi extension was splitted into sub-tests: multiple extensions and test_capi package with multiple tests.

The issue can now be closed, no?

encukou · 2023-08-23T07:37:10Z

I wouldn't close it -- the module still has a lot of tests that can be split out.

erlend-aasland · 2023-08-23T07:53:27Z

I agree, there are still useful refactors to be made.

vstinner · 2024-03-19T16:35:31Z

See also issue gh-78453.

vstinner · 2024-03-19T16:36:29Z

I created a _testlimitedcapi extension to test the C API via the limited C API: see issue gh-116417 (completed).

…H-102173) Automerge-Triggered-By: GH:erlend-aasland

* Move many functions from _testcapimodule.c into more specific files in Modules/_testcapi/. * Add files: * Modules/_testcapi/frame.c * Modules/_testcapi/function.c * Modules/_testcapi/type.c * In moved code: * Replace get_testerror() with PyExc_AssertionError. * Replace raiseTestError() with PyErr_Format(PyExc_AssertionError, ...).

* Move C API tests from test_frame to test_capi.test_frame. * Add Modules/_testcapi/frame.c file. * Add Lib/test/test_capi/test_frame.py file.

* Add Lib/test/test_capi/test_frame.py file. * Move C API tests from test_frame to test_capi.test_frame. * Add Modules/_testcapi/frame.c file. * Move C API tests from _testcapimodule.c to frame.c

Move PyType C API tests to a new file. Move following tests from test_capi.test_misc to test_capi.test_type: * BuiltinStaticTypesTests * test_get_type_name() * test_get_base_by_token()

* Move PyFunction C API tests to a new file. * Add Lib/test/test_capi/test_function.py. * Move tests from test_capi.test_misc to test_capi.test_function.

Move many functions from _testcapimodule.c into more specific files in Modules/_testcapi/. In moved code: * Replace get_testerror() with PyExc_AssertionError. * Replace raiseTestError() with PyErr_Format(PyExc_AssertionError, ...).

encukou added tests Tests in the Lib/test dir topic-C-API labels Jun 9, 2022

bedevere-bot mentioned this issue Jul 4, 2022

gh-93649: Split vectorcall testing from _testcapimodule.c #94549

Merged

gpshead mentioned this issue Jul 8, 2022

gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization #28525

Closed

tiran added a commit to tiran/cpython that referenced this issue Jul 8, 2022

pythongh-93649: Fix dependencies of _testcapi

35b1153

- header files are located in $(srcdir) - dependencies must not list C files that are also in a makesetup Setup file

bedevere-bot mentioned this issue Jul 8, 2022

gh-93649: Fix dependencies of _testcapi #94695

Merged

brandtbucher mentioned this issue Jul 9, 2022

GH-93252: Fix error handling for failed Python calls #94693

Merged

bedevere-bot mentioned this issue Jul 9, 2022

gh-93649: Fix linkage of _PyTestCapi_Init_Vectorcall #94702

Merged

tiran added a commit that referenced this issue Jul 9, 2022

gh-93649: Fix dependencies of _testcapi (#94695)

18ef240

- header files are located in $(srcdir) - dependencies must not list C files that are also in a makesetup Setup file - generate SRCDIRS for OOT builds

encukou pushed a commit that referenced this issue Jul 13, 2022

gh-93649: Fix linkage of _PyTestCapi_Init_Vectorcall (#94702)

fc21a43

encukou added a commit to encukou/cpython that referenced this issue Jul 28, 2022

pythongh-93649: Split heaptype tests from _testcapimodule.c

fb2101c

bedevere-bot mentioned this issue Jul 28, 2022

gh-93649: Split heaptype tests from _testcapimodule.c #95386

Merged

encukou added a commit that referenced this issue Aug 1, 2022

gh-93649: Split heaptype tests from _testcapimodule.c (GH-95386)

664e96a

This removes the unused negative_dictoffset function: the type this function would create is available as _testcapi.HeapCTypeWithNegativeDict

philg314 mentioned this issue Aug 1, 2022

gh-95504: Fix sign placement in PyUnicode_FromFormat #95505

Closed

philg314 added a commit to philg314/cpython that referenced this issue Aug 2, 2022

pythongh-93649: Split unicode tests from _testcapimodule.c

63e9afd

JelleZijlstra mentioned this issue May 22, 2023

PPC64LE RHEL8 3.x buildbot is timing out on test_gdb #104735

Closed

colesbury mentioned this issue Sep 1, 2023

Split up _testinternalcapi.c #108777

Closed

serhiy-storchaka mentioned this issue Oct 11, 2023

[3.12] gh-110525: Add CAPI tests for set and frozenset objects (GH-110526). #110547

Merged

vstinner mentioned this issue Mar 19, 2024

Reorganize C API tests #78453

Closed

JelleZijlstra pushed a commit to JelleZijlstra/cpython that referenced this issue Sep 10, 2024

pythongh-93649: Split exception tests from _testcapimodule.c (pythonG…

2131bed

…H-102173) Automerge-Triggered-By: GH:erlend-aasland

erlend-aasland mentioned this issue Dec 6, 2024

Add type-refactor label python/core-workflow#563

Open

bedevere-app bot mentioned this issue Jan 28, 2025

gh-93649: Split _testcapimodule.c into smaller files #129400

Closed

vstinner added a commit to vstinner/cpython that referenced this issue Jan 31, 2025

pythongh-93649: Move PyFrame C API tests to test_capi

7f9f563

* Move C API tests from test_frame to test_capi.test_frame. * Add Modules/_testcapi/frame.c file. * Add Lib/test/test_capi/test_frame.py file.

vstinner added a commit to vstinner/cpython that referenced this issue Jan 31, 2025

pythongh-93649: Move PyFrame C API tests to test_capi

b305bcd

* Move C API tests from test_frame to test_capi.test_frame. * Add Modules/_testcapi/frame.c file. * Add Lib/test/test_capi/test_frame.py file.

bedevere-app bot mentioned this issue Jan 31, 2025

gh-93649: Move PyFrame C API tests to test_capi #129512

Merged

bedevere-app bot mentioned this issue Jan 31, 2025

gh-93649: Add Modules/_testcapi/type.c file #129516

Merged

bedevere-app bot mentioned this issue Jan 31, 2025

gh-93649: Add Modules/_testcapi/function.c file #129521

Merged

vstinner added a commit that referenced this issue Jan 31, 2025

gh-93649: Add Modules/_testcapi/function.c file (#129521)

60a8541

* Move PyFunction C API tests to a new file. * Add Lib/test/test_capi/test_function.py. * Move tests from test_capi.test_misc to test_capi.test_function.

bedevere-app bot mentioned this issue Feb 1, 2025

gh-93649: Move _testcapi tests to specific files #129544

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Split up _testcapimodule.c #93649

Split up _testcapimodule.c #93649

encukou commented Jun 9, 2022 •

edited by bedevere-app bot

Loading

arhadthedev commented Jun 9, 2022

encukou commented Jun 10, 2022

encukou commented Jul 4, 2022 •

edited

Loading

vstinner commented Jul 5, 2022 •

edited by encukou

Loading

vstinner commented Jul 5, 2022

tiran commented Jul 8, 2022

encukou commented Jul 11, 2022

tiran commented Jul 11, 2022

philg314 commented Aug 1, 2022 •

edited

Loading

vstinner commented Aug 22, 2023

encukou commented Aug 23, 2023

erlend-aasland commented Aug 23, 2023

vstinner commented Mar 19, 2024

vstinner commented Mar 19, 2024

Split up _testcapimodule.c #93649

Split up _testcapimodule.c #93649

Comments

encukou commented Jun 9, 2022 • edited by bedevere-app bot Loading

arhadthedev commented Jun 9, 2022

encukou commented Jun 10, 2022

encukou commented Jul 4, 2022 • edited Loading

vstinner commented Jul 5, 2022 • edited by encukou Loading

vstinner commented Jul 5, 2022

tiran commented Jul 8, 2022

encukou commented Jul 11, 2022

tiran commented Jul 11, 2022

philg314 commented Aug 1, 2022 • edited Loading

vstinner commented Aug 22, 2023

encukou commented Aug 23, 2023

erlend-aasland commented Aug 23, 2023

vstinner commented Mar 19, 2024

vstinner commented Mar 19, 2024

encukou commented Jun 9, 2022 •

edited by bedevere-app bot

Loading

encukou commented Jul 4, 2022 •

edited

Loading

vstinner commented Jul 5, 2022 •

edited by encukou

Loading

philg314 commented Aug 1, 2022 •

edited

Loading