Skip to content

Commit

Permalink
Merge branch 'main' into optimise-path-pickle
Browse files Browse the repository at this point in the history
  • Loading branch information
barneygale authored Dec 22, 2023
2 parents 9720628 + 9c3ddf3 commit 49f5c27
Show file tree
Hide file tree
Showing 165 changed files with 3,800 additions and 5,869 deletions.
1 change: 0 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@ Programs/test_frozenmain.h generated
Python/Python-ast.c generated
Python/executor_cases.c.h generated
Python/generated_cases.c.h generated
Python/abstract_interp_cases.c.h generated
Python/opcode_targets.h generated
Python/stdlib_module_names.h generated
Tools/peg_generator/pegen/grammar_parser.py generated
Expand Down
42 changes: 26 additions & 16 deletions Doc/library/ast.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Node classes

This is the base of all AST node classes. The actual node classes are
derived from the :file:`Parser/Python.asdl` file, which is reproduced
:ref:`above <abstract-grammar>`. They are defined in the :mod:`_ast` C
:ref:`above <abstract-grammar>`. They are defined in the :mod:`!_ast` C
module and re-exported in :mod:`ast`.

There is one class defined for each left-hand side symbol in the abstract
Expand Down Expand Up @@ -128,14 +128,14 @@ Node classes

.. deprecated:: 3.8

Old classes :class:`ast.Num`, :class:`ast.Str`, :class:`ast.Bytes`,
:class:`ast.NameConstant` and :class:`ast.Ellipsis` are still available,
Old classes :class:`!ast.Num`, :class:`!ast.Str`, :class:`!ast.Bytes`,
:class:`!ast.NameConstant` and :class:`!ast.Ellipsis` are still available,
but they will be removed in future Python releases. In the meantime,
instantiating them will return an instance of a different class.

.. deprecated:: 3.9

Old classes :class:`ast.Index` and :class:`ast.ExtSlice` are still
Old classes :class:`!ast.Index` and :class:`!ast.ExtSlice` are still
available, but they will be removed in future Python releases.
In the meantime, instantiating them will return an instance of
a different class.
Expand Down Expand Up @@ -1935,8 +1935,7 @@ Function and class definitions
.. class:: arg(arg, annotation, type_comment)

A single argument in a list. ``arg`` is a raw string of the argument
name, ``annotation`` is its annotation, such as a :class:`Str` or
:class:`Name` node.
name; ``annotation`` is its annotation, such as a :class:`Name` node.

.. attribute:: type_comment

Expand Down Expand Up @@ -2210,7 +2209,7 @@ and classes for traversing abstract syntax trees:
Added ``type_comments``, ``mode='func_type'`` and ``feature_version``.

.. versionchanged:: 3.13
The minimum supported version for feature_version is now (3,7)
The minimum supported version for ``feature_version`` is now ``(3, 7)``.
The ``optimize`` argument was added.


Expand Down Expand Up @@ -2286,8 +2285,8 @@ and classes for traversing abstract syntax trees:
.. function:: get_source_segment(source, node, *, padded=False)

Get source code segment of the *source* that generated *node*.
If some location information (:attr:`lineno`, :attr:`end_lineno`,
:attr:`col_offset`, or :attr:`end_col_offset`) is missing, return ``None``.
If some location information (:attr:`~ast.AST.lineno`, :attr:`~ast.AST.end_lineno`,
:attr:`~ast.AST.col_offset`, or :attr:`~ast.AST.end_col_offset`) is missing, return ``None``.

If *padded* is ``True``, the first line of a multi-line statement will
be padded with spaces to match its original position.
Expand All @@ -2298,7 +2297,7 @@ and classes for traversing abstract syntax trees:
.. function:: fix_missing_locations(node)

When you compile a node tree with :func:`compile`, the compiler expects
:attr:`lineno` and :attr:`col_offset` attributes for every node that supports
:attr:`~ast.AST.lineno` and :attr:`~ast.AST.col_offset` attributes for every node that supports
them. This is rather tedious to fill in for generated nodes, so this helper
adds these attributes recursively where not already set, by setting them to
the values of the parent node. It works recursively starting at *node*.
Expand All @@ -2313,8 +2312,8 @@ and classes for traversing abstract syntax trees:

.. function:: copy_location(new_node, old_node)

Copy source location (:attr:`lineno`, :attr:`col_offset`, :attr:`end_lineno`,
and :attr:`end_col_offset`) from *old_node* to *new_node* if possible,
Copy source location (:attr:`~ast.AST.lineno`, :attr:`~ast.AST.col_offset`, :attr:`~ast.AST.end_lineno`,
and :attr:`~ast.AST.end_col_offset`) from *old_node* to *new_node* if possible,
and return *new_node*.


Expand Down Expand Up @@ -2360,14 +2359,18 @@ and classes for traversing abstract syntax trees:
visited unless the visitor calls :meth:`generic_visit` or visits them
itself.

.. method:: visit_Constant(node)

Handles all constant nodes.

Don't use the :class:`NodeVisitor` if you want to apply changes to nodes
during traversal. For this a special visitor exists
(:class:`NodeTransformer`) that allows modifications.

.. deprecated:: 3.8

Methods :meth:`visit_Num`, :meth:`visit_Str`, :meth:`visit_Bytes`,
:meth:`visit_NameConstant` and :meth:`visit_Ellipsis` are deprecated
Methods :meth:`!visit_Num`, :meth:`!visit_Str`, :meth:`!visit_Bytes`,
:meth:`!visit_NameConstant` and :meth:`!visit_Ellipsis` are deprecated
now and will not be called in future Python versions. Add the
:meth:`visit_Constant` method to handle all constant nodes.

Expand Down Expand Up @@ -2396,7 +2399,7 @@ and classes for traversing abstract syntax trees:
)

Keep in mind that if the node you're operating on has child nodes you must
either transform the child nodes yourself or call the :meth:`generic_visit`
either transform the child nodes yourself or call the :meth:`~ast.NodeVisitor.generic_visit`
method for the node first.

For nodes that were part of a collection of statements (that applies to all
Expand All @@ -2405,7 +2408,7 @@ and classes for traversing abstract syntax trees:

If :class:`NodeTransformer` introduces new nodes (that weren't part of
original tree) without giving them location information (such as
:attr:`lineno`), :func:`fix_missing_locations` should be called with
:attr:`~ast.AST.lineno`), :func:`fix_missing_locations` should be called with
the new sub-tree to recalculate the location information::

tree = ast.parse('foo', mode='eval')
Expand Down Expand Up @@ -2457,6 +2460,13 @@ effects on the compilation of a program:
Generates and returns an abstract syntax tree instead of returning a
compiled code object.

.. data:: PyCF_OPTIMIZED_AST

The returned AST is optimized according to the *optimize* argument
in :func:`compile` or :func:`ast.parse`.

.. versionadded:: 3.13

.. data:: PyCF_TYPE_COMMENTS

Enables support for :pep:`484` and :pep:`526` style type comments
Expand Down
2 changes: 1 addition & 1 deletion Doc/library/collections.abc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ the required methods (unless those methods have been set to

class E:
def __iter__(self): ...
def __next__(next): ...
def __next__(self): ...

.. doctest::

Expand Down
7 changes: 7 additions & 0 deletions Doc/library/functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1800,6 +1800,13 @@ are always available. They are listed here in alphabetical order.
the second argument is a type, ``issubclass(type2, type)`` must be true (this
is useful for classmethods).

When called directly within an ordinary method of a class, both arguments may
be omitted ("zero-argument :func:`!super`"). In this case, *type* will be the
enclosing class, and *obj* will be the first argument of the immediately
enclosing function (typically ``self``). (This means that zero-argument
:func:`!super` will not work as expected within nested functions, including
generator expressions, which implicitly create nested functions.)

There are two typical use cases for *super*. In a class hierarchy with
single inheritance, *super* can be used to refer to parent classes without
naming them explicitly, thus making the code more maintainable. This use
Expand Down
30 changes: 20 additions & 10 deletions Doc/library/importlib.metadata.rst
Original file line number Diff line number Diff line change
Expand Up @@ -171,16 +171,18 @@ group. Read `the setuptools docs
<https://setuptools.pypa.io/en/latest/userguide/entry_point.html>`_
for more information on entry points, their definition, and usage.

*Compatibility Note*

The "selectable" entry points were introduced in ``importlib_metadata``
3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted
no parameters and always returned a dictionary of entry points, keyed
by group. With ``importlib_metadata`` 5.0 and Python 3.12,
``entry_points`` always returns an ``EntryPoints`` object. See
`backports.entry_points_selectable <https://pypi.org/project/backports.entry-points-selectable>`_
for compatibility options.

.. versionchanged:: 3.12
The "selectable" entry points were introduced in ``importlib_metadata``
3.6 and Python 3.10. Prior to those changes, ``entry_points`` accepted
no parameters and always returned a dictionary of entry points, keyed
by group. With ``importlib_metadata`` 5.0 and Python 3.12,
``entry_points`` always returns an ``EntryPoints`` object. See
`backports.entry_points_selectable <https://pypi.org/project/backports.entry-points-selectable>`_
for compatibility options.

.. versionchanged:: 3.13
``EntryPoint`` objects no longer present a tuple-like interface
(:meth:`~object.__getitem__`).

.. _metadata:

Expand Down Expand Up @@ -342,9 +344,17 @@ instance::
>>> dist.metadata['License'] # doctest: +SKIP
'MIT'

For editable packages, an origin property may present :pep:`610`
metadata::

>>> dist.origin.url
'file:///path/to/wheel-0.32.3.editable-py3-none-any.whl'

The full set of available metadata is not described here.
See the `Core metadata specifications <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_ for additional details.

.. versionadded:: 3.13
The ``.origin`` property was added.

Distribution Discovery
======================
Expand Down
21 changes: 18 additions & 3 deletions Doc/library/os.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4570,7 +4570,8 @@ written in Python, such as a mail server's external command delivery program.
Most users should use :func:`subprocess.run` instead of :func:`posix_spawn`.

The positional-only arguments *path*, *args*, and *env* are similar to
:func:`execve`.
:func:`execve`. *env* is allowed to be ``None``, in which case current
process' environment is used.

The *path* parameter is the path to the executable file. The *path* should
contain a directory. Use :func:`posix_spawnp` to pass an executable file
Expand Down Expand Up @@ -4600,10 +4601,17 @@ written in Python, such as a mail server's external command delivery program.

Performs ``os.dup2(fd, new_fd)``.

.. data:: POSIX_SPAWN_CLOSEFROM

(``os.POSIX_SPAWN_CLOSEFROM``, *fd*)

Performs ``os.closerange(fd, INF)``.

These tuples correspond to the C library
:c:func:`!posix_spawn_file_actions_addopen`,
:c:func:`!posix_spawn_file_actions_addclose`, and
:c:func:`!posix_spawn_file_actions_adddup2` API calls used to prepare
:c:func:`!posix_spawn_file_actions_addclose`,
:c:func:`!posix_spawn_file_actions_adddup2`, and
:c:func:`!posix_spawn_file_actions_addclosefrom_np` API calls used to prepare
for the :c:func:`!posix_spawn` call itself.

The *setpgroup* argument will set the process group of the child to the value
Expand Down Expand Up @@ -4645,6 +4653,13 @@ written in Python, such as a mail server's external command delivery program.

.. versionadded:: 3.8

.. versionchanged:: 3.13
*env* parameter accepts ``None``.

.. versionchanged:: 3.13
``os.POSIX_SPAWN_CLOSEFROM`` is available on platforms where
:c:func:`!posix_spawn_file_actions_addclosefrom_np` exists.

.. availability:: Unix, not Emscripten, not WASI.

.. function:: posix_spawnp(path, argv, env, *, file_actions=None, \
Expand Down
2 changes: 1 addition & 1 deletion Doc/library/ssl.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ probably additional platforms, as long as OpenSSL is installed on that platform.

Some behavior may be platform dependent, since calls are made to the
operating system socket APIs. The installed version of OpenSSL may also
cause variations in behavior. For example, TLSv1.3 with OpenSSL version
cause variations in behavior. For example, TLSv1.3 comes with OpenSSL version
1.1.1.

.. warning::
Expand Down
67 changes: 56 additions & 11 deletions Doc/library/tarfile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ Some facts and figures:
``'filemode|[compression]'``. :func:`tarfile.open` will return a :class:`TarFile`
object that processes its data as a stream of blocks. No random seeking will
be done on the file. If given, *fileobj* may be any object that has a
:meth:`read` or :meth:`write` method (depending on the *mode*). *bufsize*
:meth:`~io.TextIOBase.read` or :meth:`~io.TextIOBase.write` method (depending on the *mode*). *bufsize*
specifies the blocksize and defaults to ``20 * 512`` bytes. Use this variant
in combination with e.g. ``sys.stdin``, a socket :term:`file object` or a tape
device. However, such a :class:`TarFile` object is limited in that it does
Expand Down Expand Up @@ -255,6 +255,51 @@ The following constants are available at the module level:
The default character encoding: ``'utf-8'`` on Windows, the value returned by
:func:`sys.getfilesystemencoding` otherwise.

.. data:: REGTYPE
AREGTYPE

A regular file :attr:`~TarInfo.type`.

.. data:: LNKTYPE

A link (inside tarfile) :attr:`~TarInfo.type`.

.. data:: SYMTYPE

A symbolic link :attr:`~TarInfo.type`.

.. data:: CHRTYPE

A character special device :attr:`~TarInfo.type`.

.. data:: BLKTYPE

A block special device :attr:`~TarInfo.type`.

.. data:: DIRTYPE

A directory :attr:`~TarInfo.type`.

.. data:: FIFOTYPE

A FIFO special device :attr:`~TarInfo.type`.

.. data:: CONTTYPE

A contiguous file :attr:`~TarInfo.type`.

.. data:: GNUTYPE_LONGNAME

A GNU tar longname :attr:`~TarInfo.type`.

.. data:: GNUTYPE_LONGLINK

A GNU tar longlink :attr:`~TarInfo.type`.

.. data:: GNUTYPE_SPARSE

A GNU tar sparse file :attr:`~TarInfo.type`.


Each of the following constants defines a tar archive format that the
:mod:`tarfile` module is able to create. See section :ref:`tar-formats` for
Expand Down Expand Up @@ -325,7 +370,7 @@ be finalized; only the internally used file object will be closed. See the

*name* is the pathname of the archive. *name* may be a :term:`path-like object`.
It can be omitted if *fileobj* is given.
In this case, the file object's :attr:`name` attribute is used if it exists.
In this case, the file object's :attr:`!name` attribute is used if it exists.

*mode* is either ``'r'`` to read from an existing archive, ``'a'`` to append
data to an existing file, ``'w'`` to create a new file overwriting an existing
Expand Down Expand Up @@ -359,7 +404,7 @@ be finalized; only the internally used file object will be closed. See the
messages). The messages are written to ``sys.stderr``.

*errorlevel* controls how extraction errors are handled,
see :attr:`the corresponding attribute <~TarFile.errorlevel>`.
see :attr:`the corresponding attribute <TarFile.errorlevel>`.

The *encoding* and *errors* arguments define the character encoding to be
used for reading or writing the archive and how conversion errors are going
Expand Down Expand Up @@ -645,8 +690,8 @@ It does *not* contain the file's data itself.
:meth:`~TarFile.getmember`, :meth:`~TarFile.getmembers` and
:meth:`~TarFile.gettarinfo`.

Modifying the objects returned by :meth:`~!TarFile.getmember` or
:meth:`~!TarFile.getmembers` will affect all subsequent
Modifying the objects returned by :meth:`~TarFile.getmember` or
:meth:`~TarFile.getmembers` will affect all subsequent
operations on the archive.
For cases where this is unwanted, you can use :mod:`copy.copy() <copy>` or
call the :meth:`~TarInfo.replace` method to create a modified copy in one step.
Expand Down Expand Up @@ -795,8 +840,8 @@ A ``TarInfo`` object has the following public data attributes:

A dictionary containing key-value pairs of an associated pax extended header.

.. method:: TarInfo.replace(name=..., mtime=..., mode=..., linkname=...,
uid=..., gid=..., uname=..., gname=...,
.. method:: TarInfo.replace(name=..., mtime=..., mode=..., linkname=..., \
uid=..., gid=..., uname=..., gname=..., \
deep=True)

.. versionadded:: 3.12
Expand All @@ -816,7 +861,7 @@ A :class:`TarInfo` object also provides some convenient query methods:

.. method:: TarInfo.isfile()

Return :const:`True` if the :class:`Tarinfo` object is a regular file.
Return :const:`True` if the :class:`TarInfo` object is a regular file.


.. method:: TarInfo.isreg()
Expand Down Expand Up @@ -952,7 +997,7 @@ reused in custom filters:
path (after following symlinks) would end up outside the destination.
This raises :class:`~tarfile.OutsideDestinationError`.
- Clear high mode bits (setuid, setgid, sticky) and group/other write bits
(:const:`~stat.S_IWGRP`|:const:`~stat.S_IWOTH`).
(:const:`~stat.S_IWGRP` | :const:`~stat.S_IWOTH`).

Return the modified ``TarInfo`` member.

Expand All @@ -977,9 +1022,9 @@ reused in custom filters:
- For regular files, including hard links:

- Set the owner read and write permissions
(:const:`~stat.S_IRUSR`|:const:`~stat.S_IWUSR`).
(:const:`~stat.S_IRUSR` | :const:`~stat.S_IWUSR`).
- Remove the group & other executable permission
(:const:`~stat.S_IXGRP`|:const:`~stat.S_IXOTH`)
(:const:`~stat.S_IXGRP` | :const:`~stat.S_IXOTH`)
if the owner doesn’t have it (:const:`~stat.S_IXUSR`).

- For other files (directories), set ``mode`` to ``None``, so
Expand Down
Loading

0 comments on commit 49f5c27

Please sign in to comment.