diff --git a/peps/pep-0835.rst b/peps/pep-0835.rst index 68cfa2b299d..16c3761dabc 100644 --- a/peps/pep-0835.rst +++ b/peps/pep-0835.rst @@ -228,46 +228,11 @@ origin: Supported Left-Hand Operands ----------------------------- -The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the -metatype (``type``) and to several typing-related types. The operator is -supported for any left-hand operand that currently supports the ``|`` -operator for making a union. +The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the metatype (``type``) and to several typing-related types. The operator is supported for instances of ``type`` and other typing constructs that currently support the ``|`` operator for unions. The specific handling of ``None`` is currently unsettled (see Open Issues). For all other left-hand operands, the operator returns ``NotImplemented``, allowing normal ``__matmul__`` dispatch to proceed. -Parsing and Grammar -=================== - -This proposal requires no changes to the Python grammar. Because ``@`` is -already a valid operator, it is natively parsed as a binary operation. The -shorthand is resolved during semantic analysis, entirely bypassing the need -to patch grammar files or update the parser. - -How to Teach This -================= - -In Python, the ``@`` symbol already has an established association with -metadata through decorators. The annotation shorthand extends this -intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``, -decorated with ``Field(gt=0)``." - -For beginners, the key rule is simple: **in a type annotation, ``@`` means -"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax -remains available and is entirely equivalent for those who find it clearer. - -For experienced developers, the precedence rules follow standard Python -operator precedence (``@`` binds tighter than ``|``), and chaining -``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does. - -Documentation and teaching materials should introduce the shorthand alongside -``Annotated``, not as a replacement. The longhand form is still preferred in -contexts where multiple metadata items are passed as a group and chaining -would be unwieldy. - -Backwards Compatibility -======================= - Forward References and Deferred Evaluation ------------------------------------------- @@ -277,8 +242,9 @@ module provides several formats for retrieving annotations: - ``Format.VALUE``: Fully evaluates the annotation. Raises ``NameError`` if any name is unresolvable. - ``Format.FORWARDREF``: Wraps unresolvable names in ``ForwardRef`` objects. - However, compound expressions using operators (``@``, ``|``) produce an - opaque ``ForwardRef`` string containing the entire expression. + However, when operators like ``@`` or ``|`` fail to evaluate because of an + unresolvable name, this format falls back to returning the entire expression + as an opaque string wrapped in a single ``ForwardRef`` object. - ``Format.STRING``: Returns the raw source text with no evaluation. For the ``@`` operator, ``Format.FORWARDREF`` is insufficient. Consider:: @@ -293,8 +259,8 @@ forward reference is resolved. This is a blocking issue for libraries like Pydantic and FastAPI, which inspect metadata at class-definition time. This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``. -This format assumes typing semantics and evaluates compound type expressions -**structurally**. It always returns an ``AnnotatedType`` for ``@``, a union +This format assumes typing semantics and evaluates type expressions involving +operators (like ``@`` and ``|``) **structurally**. It always returns an ``AnnotatedType`` for ``@``, a union for ``|``, and a ``GenericAlias`` for subscripting. When a name cannot be resolved, only that name is wrapped in ``ForwardRef``; the surrounding operators are still evaluated. The example above produces:: @@ -321,6 +287,38 @@ coarser than from :pep:`749` thunks. When a name is unresolvable, the rather than just a name. :pep:`749` provides a strictly better experience and is the recommended path forward. +Parsing and Grammar +=================== + +This proposal requires no changes to the Python grammar. Because ``@`` is +already a valid operator, it is natively parsed as a binary operation. The +shorthand is resolved during semantic analysis, entirely bypassing the need +to patch grammar files or update the parser. + +How to Teach This +================= + +In Python, the ``@`` symbol already has an established association with +metadata through decorators. The annotation shorthand extends this +intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``, +decorated with ``Field(gt=0)``." + +For beginners, the key rule is simple: **in a type annotation, ``@`` means +"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax +remains available and is entirely equivalent for those who find it clearer. + +For experienced developers, the precedence rules follow standard Python +operator precedence (``@`` binds tighter than ``|``), and chaining +``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does. + +Documentation and teaching materials should introduce the shorthand alongside +``Annotated``, not as a replacement. The longhand form is still preferred in +contexts where multiple metadata items are passed as a group and chaining +would be unwieldy. + +Backwards Compatibility +======================= + Operator Overloading -------------------- @@ -354,7 +352,7 @@ metaclass. The private ``typing._AnnotatedAlias`` class is retained as a deprecated compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)`` will continue to work but emit a ``DeprecationWarning``. The shim is -scheduled for removal in Python 3.18. +scheduled for removal in Python 3.21. Code that should be updated: @@ -481,6 +479,26 @@ References - :pep:`727` -- Documentation metadata in typing (Withdrawn) - :pep:`749` -- Implementing PEP 649 +Open Issues +=========== + +Supporting ``None`` +------------------- + +While the ``@`` operator naturally applies to instances of ``type``, it is currently unsettled how it should handle ``None``. + +There are two primary options, both with notable downsides: + +1. **Implement ``__matmul__`` on ``NoneType``:** This provides the best ergonomics, allowing ``None @ Metadata``. However, it introduces a runtime risk. If a user accidentally uses matrix multiplication on a ``None`` value (e.g., ``a @ b`` where ``a`` is ``None`` instead of a numpy array), it will silently return an ``Annotated`` object instead of raising a ``TypeError``. +2. **Do not support ``None @ Metadata``:** This avoids the runtime risk, but creates an ergonomic cliff. Users would be forced to write ``type(None) @ Metadata`` or fall back to ``typing.Annotated[None, Metadata]``. + +The authors of this PEP lean slightly in favor of implementing ``__matmul__`` on ``NoneType`` for its ergonomic benefits, but ultimately defer to the Typing Council and community discussion to decide the best path forward. + +Deprecation Timeline +-------------------- + +This PEP schedules the removal of the ``typing._AnnotatedAlias`` compatibility shim for Python 3.21, following the standard 5-year deprecation policy (:pep:`387`). However, given the widespread use of ``typing.Annotated``, the exact timeline for removal—or whether the shim should be retained indefinitely to prevent churn in legacy code—remains open for community discussion. + Copyright =========