Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-32227: functools.singledispatch supports registering via type annotations #4733

Merged
merged 5 commits into from
Dec 11, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 21 additions & 7 deletions Doc/library/functools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -281,23 +281,34 @@ The :mod:`functools` module defines the following functions:
... print(arg)

To add overloaded implementations to the function, use the :func:`register`
attribute of the generic function. It is a decorator, taking a type
parameter and decorating a function implementing the operation for that
type::
attribute of the generic function. It is a decorator. For functions
annotated with types, the decorator will infer the type of the first
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a glossary link for function annotations?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

argument automatically::

>>> @fun.register(int)
... def _(arg, verbose=False):
>>> @fun.register
... def _(arg: int, verbose=False):
... if verbose:
... print("Strength in numbers, eh?", end=" ")
... print(arg)
...
>>> @fun.register(list)
... def _(arg, verbose=False):
>>> @fun.register
... def _(arg: list, verbose=False):
... if verbose:
... print("Enumerate this:")
... for i, elem in enumerate(arg):
... print(i, elem)

For code which doesn't use type annotations, the appropriate type
argument can be passed explicitly to the decorator itself::

>>> @fun.register(complex)
... def _(arg, verbose=False):
... if verbose:
... print("Better than complicated.", end=" ")
... print(arg.real, arg.imag)
...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would put these examples in the opposite order. Show the traditional way first, and then say, "if you are using type annotations in your code, you don't need to repeat the type information in the @register call:"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the current order is OK, but the lead in sentence for the different examples need to be changed. Something like:

... If using type annotations in your code, just apply the decorator, and it will infer the appropriate overload based on the first parameter's annotation::

And then:

For code which doesn't use type annotations, the appropriate type parameter can instead be passed explicitly to the decorator itself:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why: because type hinting is still new and not known or used by most people on most projects.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like Nick's suggestion and applied it in the latest commit.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still prefer for the default (that is the first presented) to be no-type-hints, but I'm not going to block this, The reason is stronger than Merwok's "type hinting is still new". I don't want type hinting in my code (having it in external stubs is fine with me), and I would prefer that python's default remain no type hinting. That is the stance we have taken for the standard library and the documentation, and this is part of our documentation ;)

Copy link
Contributor Author

@ambv ambv Dec 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not quite right.

Historically the syntactic feature of annotations didn't have a standardized meaning, so the standard library couldn't use it internally and we couldn't document any APIs that would use it. Otherwise we'd be creating a "de facto" standard.

The situation changed with PEP 484. Since this was a very large feature, we have taken the stance that we won't be covering the standard library with type hints, at least not until typing graduates from provisional status and further discussion on python-dev (or maybe even a PEP about this).

However, at this point we never said we wouldn't allow for the standard library to implement features that allow the users of the library to utilize annotations. The typing module itself is an obvious example: it presents users with a feature that requires using the syntactic construct of annotations.
Data classes for Python 3.7 also utilize annotations to define fields. And in this pull request, singledispatch uses annotations for a similar purpose: to inform runtime type decisions.

And by the way, since the entire point of PEP 557 was to bring an "attrs" implementation to the standard library in order for the standard library to be able to use it itself, I think with Python 3.7 effectively type annotations are going to start appearing in the standard library by means of data classes alone. At least with singledispatch I'm leaving the classic syntax alone so users with a visceral reaction to annotations can keep using it.



To enable registering lambdas and pre-existing functions, the
:func:`register` attribute can be used in a functional form::

Expand Down Expand Up @@ -368,6 +379,9 @@ The :mod:`functools` module defines the following functions:

.. versionadded:: 3.4

.. versionchanged:: 3.7
The :func:`register` attribute supports using type annotations.


.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)

Expand Down
18 changes: 17 additions & 1 deletion Lib/functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -793,7 +793,23 @@ def register(cls, func=None):
"""
nonlocal cache_token
if func is None:
return lambda f: register(cls, f)
if isinstance(cls, type):
return lambda f: register(cls, f)
ann = getattr(cls, '__annotations__', {})
if not ann:
raise TypeError(
f"Invalid first argument to `register()`: {cls!r}. "
f"Use either `@register(some_class)` or plain `@register` "
f"on an annotated function."
)
func = cls

# only import typing if annotation parsing is necessary
from typing import get_type_hints
argname, cls = next(iter(get_type_hints(func).items()))
assert isinstance(cls, type), (
f"Invalid annotation for {argname!r}. {cls!r} is not a class."
)
registry[cls] = func
if cache_token is None and hasattr(cls, '__abstractmethods__'):
cache_token = get_cache_token()
Expand Down
68 changes: 68 additions & 0 deletions Lib/test/test_functools.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
from test import support
import threading
import time
import typing
import unittest
import unittest.mock
from weakref import proxy
Expand Down Expand Up @@ -2119,6 +2120,73 @@ class X:
g._clear_cache()
self.assertEqual(len(td), 0)

def test_annotations(self):
@functools.singledispatch
def i(arg):
return "base"
@i.register
def _(arg: collections.abc.Mapping):
return "mapping"
@i.register
def _(arg: "collections.abc.Sequence"):
return "sequence"
self.assertEqual(i(None), "base")
self.assertEqual(i({"a": 1}), "mapping")
self.assertEqual(i([1, 2, 3]), "sequence")
self.assertEqual(i((1, 2, 3)), "sequence")
self.assertEqual(i("str"), "sequence")

# Registering classes as callables doesn't work with annotations,
# you need to pass the type explicitly.
@i.register(str)
class _:
def __init__(self, arg):
self.arg = arg

def __eq__(self, other):
return self.arg == other
self.assertEqual(i("str"), "str")

def test_invalid_registrations(self):
msg_prefix = "Invalid first argument to `register()`: "
msg_suffix = (
". Use either `@register(some_class)` or plain `@register` on an "
"annotated function."
)
@functools.singledispatch
def i(arg):
return "base"
with self.assertRaises(TypeError) as exc:
@i.register(42)
def _(arg):
return "I annotated with a non-type"
self.assertTrue(str(exc.exception).startswith(msg_prefix + "42"))
self.assertTrue(str(exc.exception).endswith(msg_suffix))
with self.assertRaises(TypeError) as exc:
@i.register
def _(arg):
return "I forgot to annotate"
self.assertTrue(str(exc.exception).startswith(msg_prefix +
"<function TestSingleDispatch.test_invalid_registrations.<locals>._"
))
self.assertTrue(str(exc.exception).endswith(msg_suffix))

# FIXME: The following will only work after PEP 560 is implemented.
return

with self.assertRaises(TypeError) as exc:
@i.register
def _(arg: typing.Iterable[str]):
# At runtime, dispatching on generics is impossible.
# When registering implementations with singledispatch, avoid
# types from `typing`. Instead, annotate with regular types
# or ABCs.
return "I annotated with a generic collection"
self.assertTrue(str(exc.exception).startswith(msg_prefix +
"<function TestSingleDispatch.test_invalid_registrations.<locals>._"
))
self.assertTrue(str(exc.exception).endswith(msg_suffix))


if __name__ == '__main__':
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
``functools.singledispatch`` now supports registering implementations using
type annotations.