MethodType incorrectly rejected as subtype of Callable

[mthuurne]

When I run mypy on the following code:

from types import MethodType
from typing import Callable

def describe(func: Callable[[], None]) -> str:
    if isinstance(func, MethodType):
        return 'bound method'
    else:
        return 'other callable'

class C:
    def m(self) -> None:
        pass

print(describe(C().m))

It outputs:

$ mypy --warn-unreachable methodtype.py 
methodtype.py:6: error: Statement is unreachable

This statement is return 'bound method'.

If you run the test case, you will see "bound method" being printed, so the code is reachable in practice.

For some reason the problem disappears if the annotation func: Callable[[], None] is changed to just func: Callable.

I'm using mypy 0.770 on Python 3.8.2, with the --warn-unreachable option.

[beezee]

Methods always have at least one arg (self) - would that justify mypy concluding that a nullary callable would never unify with MethodType?

[mthuurne]

In Python 3 (not sure about 2), an instance of MethodType always represents a bound method; a non-bound method is considered an instance of FunctionType instead:

>>> from types import *
>>> class C:
...   def m(self):
...     pass
... 
>>> isinstance(C.m, FunctionType)
True
>>> isinstance(C.m, MethodType)
False
>>> isinstance(C().m, FunctionType)
False
>>> isinstance(C().m, MethodType)
True

The instance that a method is bound to is stored in the __self__ attribute of the bound method object:

>>> o = C()
>>> o
<__main__.C object at 0x7f926f2a3d30>
>>> o.m
<bound method C.m of <__main__.C object at 0x7f926f2a3d30>>
>>> o.m.__self__
<__main__.C object at 0x7f926f2a3d30>
>>> o.m()
>>>

As you can see, it's correct to call a bound method that only takes self as an argument with 0 additional arguments.

[beezee]

That makes sense. In the interest of narrowing things down, are you able to clear the error if you tweak the callable type? Either specifically to take a C or Any or some other option?

[mthuurne]

Changing the argument types of the Callable does not seem to have any effect, but changing the return type to Any makes the warning go away, which doesn't make sense to me.

[JukkaL]

Mypy has no special understanding of MethodType. It's defined as a regular class in typeshed, and mypy assumes that's the whole story.

A potential fix would be to special case MethodType when deciding whether the isinstance check can be true. The relevant code seems to be in is_overlapping_types (mypy.meet)?

[mthuurne]

MethodType in typeshed:

class MethodType:
    __func__: _StaticFunctionType
    __self__: object
    __name__: str
    __qualname__: str
    def __init__(self, func: Callable[..., Any], obj: object) -> None: ...
    def __call__(self, *args: Any, **kwargs: Any) -> Any: ...

MethodType.__call__ returns Any, so perhaps that is why changing the return type makes the message go away? A subclass is allowed to narrow the return type of a method though, so I still don't understand why mypy thinks isinstance(func, MethodType) is always false.

[mthuurne]

Here is a test case that doesn't use MethodType but does exhibit the same behavior that I don't understand:

from typing import Any, Callable

class ProtoAny:
    def __call__(self) -> Any: ...

class ProtoNone:
    def __call__(self) -> None: ...

def describe(func: Callable[[], None]) -> str:
    if isinstance(func, ProtoAny):
        return 'ProtoAny'
    elif isinstance(func, ProtoNone):
        return 'ProtoNone'
    else:
        return 'other callable'

class C(ProtoAny):
    def __call__(self) -> None: ...

def f():
    pass

print(describe(ProtoAny()))
print(describe(C()))
print(describe(ProtoNone()))
print(describe(f))

mypy reports:

$ mypy --warn-unreachable testcase.py
testcase.py:11: error: Statement is unreachable

Which is the return 'ProtoAny' line, so isinstance(func, ProtoAny) is considered always false, but isinstance(func, ProtoNone) is not.

[mthuurne]

A test case without Any does work in the way I expect it:

from typing import Callable

class A:
    pass

class B(A):
    pass

class ProtoA:
    def __call__(self) -> A: ...

class ProtoB:
    def __call__(self) -> B: ...

def describe(func: Callable[[], A]) -> str:
    if isinstance(func, ProtoA):
        return 'ProtoA'
    elif isinstance(func, ProtoB):
        return 'ProtoB'
    else:
        return 'other callable'

class C(ProtoA):
    def __call__(self) -> B: ...

class D(ProtoB):
    def __call__(self) -> A: ...

def f():
    pass

print(describe(C()))
print(describe(D()))
print(describe(f))

mypy reports:

$ mypy --warn-unreachable testcase.py 
testcase.py:27: error: Return type "A" of "__call__" incompatible with return type "B" in supertype "ProtoB"

Line 27 is the body of class D, which is indeed incorrect, because it widens the return type of __call__ compared to its superclass.

[ods]

One more case:

from types import MethodType

class C:
    def m(self):
        pass

def f(meth: MethodType):
    meth.__self__

f(C().m)

The output:
error: Argument 1 to "f" has incompatible type "Callable[[], Any]"; expected "MethodType"

[JelleZijlstra]

This is now also reproducible with TypeIs; found it as a result of python/typeshed#11823. Here is an example that doesn't rely on any non-basic types:

from typing import Any, Callable, reveal_type
from typing_extensions import TypeIs


class FunctionType:
    def __call__(self, *args: Any, **kwargs: Any) -> Any: ...

def isfunction(x: object) -> TypeIs[FunctionType]:
    raise NotImplementedError


def decorate(wrapped: Callable[..., object]) -> Any:
    if isfunction(wrapped):
        reveal_type(wrapped)
    if isinstance(wrapped, FunctionType):
        reveal_type(wrapped)

--warn-unreachable will say the two reveal_types are unreachable. https://mypy-play.net/?mypy=latest&python=3.10&flags=warn-unreachable&gist=61455819df1bb90f40587091a810851d