Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-123523: Rework typing documentation for generators and coroutines, and link to it from collections.abc docs #123544

Merged
merged 18 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions Doc/library/collections.abc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,9 @@ Collections Abstract Base Classes -- Detailed Descriptions

ABC for classes that provide the :meth:`~object.__call__` method.

See :ref:`annotating-callables` for details on how to use
:class:`Callable` and :class:`~typing.Callable` in type annotations.

.. class:: Iterable

ABC for classes that provide the :meth:`~container.__iter__` method.
Expand Down Expand Up @@ -250,6 +253,10 @@ Collections Abstract Base Classes -- Detailed Descriptions
:meth:`~generator.send`,
:meth:`~generator.throw` and :meth:`~generator.close` methods.

Refer to
:ref:`annotating generators and coroutines <annotating-generators-and-coroutines>`
for details on using this class in type annotations.

.. versionadded:: 3.5

.. class:: Sequence
Expand Down Expand Up @@ -280,6 +287,11 @@ Collections Abstract Base Classes -- Detailed Descriptions

ABCs for read-only and mutable :term:`mappings <mapping>`.

Example use in type annotations::

def get_position_in_index(word_list: Mapping[str, int], word: str) -> int:
return word_list[word]

.. class:: MappingView
ItemsView
KeysView
Expand Down Expand Up @@ -321,6 +333,12 @@ Collections Abstract Base Classes -- Detailed Descriptions
Using ``isinstance(gencoro, Coroutine)`` for them will return ``False``.
Use :func:`inspect.isawaitable` to detect them.

Refer to
:ref:`annotating generators and coroutines <annotating-generators-and-coroutines>`
for details on using this class in type annotations.
The variance and order of type parameters correspond to those of
:class:`Generator`.

.. versionadded:: 3.5

.. class:: AsyncIterable
Expand All @@ -342,6 +360,10 @@ Collections Abstract Base Classes -- Detailed Descriptions
ABC for :term:`asynchronous generator` classes that implement the protocol
defined in :pep:`525` and :pep:`492`.

Refer to
:ref:`annotating generators and coroutines <annotating-generators-and-coroutines>`
for details on using this class in type annotations.

.. versionadded:: 3.6

.. class:: Buffer
Expand Down
174 changes: 93 additions & 81 deletions Doc/library/typing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,87 @@ For example::
``type[Any]`` is equivalent to :class:`type`, which is the root of Python's
:ref:`metaclass hierarchy <metaclasses>`.


.. _annotating-generators-and-coroutines:

Annotating generators and coroutines
====================================

A generator can be annotated using the generic type
:class:`Generator[YieldType, SendType, ReturnType] <collections.abc.Generator>`.
For example::

def echo_round() -> Generator[int, float, str]:
sent = yield 0
while sent >= 0:
sent = yield round(sent)
return 'Done'

Note that unlike many other generic classes in the standard library,
the ``SendType`` of :class:`~collections.abc.Generator` behaves
contravariantly, not covariantly or invariantly.

The ``SendType`` and ``ReturnType`` parameters default to :const:`!None`::

def infinite_stream(start: int) -> Generator[int]:
while True:
yield start
start += 1

It is also possible to set these types explicitly::

def infinite_stream(start: int) -> Generator[int, None, None]:
while True:
yield start
start += 1

Simple generators that only ever yield values can also be annotated
as having a return type of either
:class:`Iterable[YieldType] <collections.abc.Iterable>`
or :class:`Iterator[YieldType] <collections.abc.Iterator>`::

def infinite_stream(start: int) -> Iterator[int]:
while True:
yield start
start += 1

Async generators are handled in a similar fashion, but don't
expect a ``ReturnType`` type argument
(:class:`AsyncGenerator[YieldType, SendType] <collections.abc.AsyncGenerator>`).
The ``SendType`` argument defaults to :const:`!None`, so the following definitions
are equivalent::

async def infinite_stream(start: int) -> AsyncGenerator[int]:
while True:
yield start
start = await increment(start)

async def infinite_stream(start: int) -> AsyncGenerator[int, None]:
while True:
yield start
start = await increment(start)

As in the synchronous case,
:class:`AsyncIterable[YieldType] <collections.abc.AsyncIterable>`
and :class:`AsyncIterator[YieldType] <collections.abc.AsyncIterator>` are
available as well::

async def infinite_stream(start: int) -> AsyncIterator[int]:
while True:
yield start
start = await increment(start)

Coroutines can be annotated using
:class:`Coroutine[YieldType, SendType, ReturnType] <collections.abc.Coroutine>`.
Generic arguments correspond to those of :class:`~collections.abc.Generator`,
for example::

from collections.abc import Coroutine
c: Coroutine[list[str], str, int] # Some coroutine defined elsewhere
x = c.send('hi') # Inferred type of 'x' is list[str]
async def bar() -> None:
y = await c # Inferred type of 'y' is int

.. _user-defined-generics:

User-defined generic types
Expand Down Expand Up @@ -3544,11 +3625,6 @@ Aliases to container ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.Mapping`.

This type can be used as follows::

def get_position_in_index(word_list: Mapping[str, int], word: str) -> int:
return word_list[word]

.. deprecated:: 3.9
:class:`collections.abc.Mapping` now supports subscripting (``[]``).
See :pep:`585` and :ref:`types-genericalias`.
Expand Down Expand Up @@ -3612,14 +3688,10 @@ Aliases to asynchronous ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.Coroutine`.

The variance and order of type variables
correspond to those of :class:`Generator`, for example::

from collections.abc import Coroutine
c: Coroutine[list[str], str, int] # Some coroutine defined elsewhere
x = c.send('hi') # Inferred type of 'x' is list[str]
async def bar() -> None:
y = await c # Inferred type of 'y' is int
See
:ref:`annotating generators and coroutines <annotating-generators-and-coroutines>`
for details on using :class:`collections.abc.Coroutine`
and ``typing.Coroutine`` in type annotations.

.. versionadded:: 3.5.3

Expand All @@ -3631,40 +3703,10 @@ Aliases to asynchronous ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.AsyncGenerator`.

An async generator can be annotated by the generic type
``AsyncGenerator[YieldType, SendType]``. For example::

async def echo_round() -> AsyncGenerator[int, float]:
sent = yield 0
while sent >= 0.0:
rounded = await round(sent)
sent = yield rounded

Unlike normal generators, async generators cannot return a value, so there
is no ``ReturnType`` type parameter. As with :class:`Generator`, the
``SendType`` behaves contravariantly.

The ``SendType`` defaults to :const:`!None`::

async def infinite_stream(start: int) -> AsyncGenerator[int]:
while True:
yield start
start = await increment(start)

It is also possible to set this type explicitly::

async def infinite_stream(start: int) -> AsyncGenerator[int, None]:
while True:
yield start
start = await increment(start)

Alternatively, annotate your generator as having a return type of
either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``::

async def infinite_stream(start: int) -> AsyncIterator[int]:
while True:
yield start
start = await increment(start)
See
:ref:`annotating generators and coroutines <annotating-generators-and-coroutines>`
for details on using :class:`collections.abc.AsyncGenerator`
and ``typing.AsyncGenerator`` in type annotations.

.. versionadded:: 3.6.1

Expand Down Expand Up @@ -3746,40 +3788,10 @@ Aliases to other ABCs in :mod:`collections.abc`

Deprecated alias to :class:`collections.abc.Generator`.

A generator can be annotated by the generic type
``Generator[YieldType, SendType, ReturnType]``. For example::

def echo_round() -> Generator[int, float, str]:
sent = yield 0
while sent >= 0:
sent = yield round(sent)
return 'Done'

Note that unlike many other generics in the typing module, the ``SendType``
of :class:`Generator` behaves contravariantly, not covariantly or
invariantly.

The ``SendType`` and ``ReturnType`` parameters default to :const:`!None`::

def infinite_stream(start: int) -> Generator[int]:
while True:
yield start
start += 1

It is also possible to set these types explicitly::

def infinite_stream(start: int) -> Generator[int, None, None]:
while True:
yield start
start += 1

Alternatively, annotate your generator as having a return type of
either ``Iterable[YieldType]`` or ``Iterator[YieldType]``::

def infinite_stream(start: int) -> Iterator[int]:
while True:
yield start
start += 1
See
:ref:`annotating generators and coroutines <annotating-generators-and-coroutines>`
for details on using :class:`collections.abc.Generator`
and ``typing.Generator`` in type annotations.

.. deprecated:: 3.9
:class:`collections.abc.Generator` now supports subscripting (``[]``).
Expand Down
Loading