mypy 1.6.1, ParamSpec: Cannot infer type argument 1 of "function_name"

[orsinium]

In pytypest, inference for code using ParamSpec started to fail in mypy 1.6.1.

To Reproduce

from typing import Callable, Generic, ParamSpec
from dataclasses import dataclass

P = ParamSpec('P')

@dataclass
class Case(Generic[P]):
    pass

def case(*args: P.args, **kwargs: P.kwargs) -> Case[P]:
    return Case()

def _test(a: int):
    assert a

def parametrize(func: Callable[P, None], case: Case[P]):
    pass

parametrize(_test, case(1))

Expected Behavior

On mypy==1.5.1, no issues are reported.

Actual Behavior

On mypy==1.6.1, inference fails:

$ python3.10 -m mypy tmp2.py
tmp2.py:19: error: Cannot infer type argument 1 of "parametrize"  [misc]
Found 1 error in 1 file (checked 1 source file)

Your Environment

• Mypy version used: 1.6.1
• Mypy command-line flags: none
• Mypy configuration options from mypy.ini (and other config files): none
• Python version used: tested with python 3.8 and 3.10.

I also tried using --new-type-inference, it doesn't help.

[AlexWaygood]

This is probably the same thing as #16296, though I haven't looked too deeply

[orsinium]

It might, I'm not 100% sure. Their example uses Concatenate, mine doesn't. Either way, if either issue has a fix, it's worth checking if that solves the other issue as well.

[erictraut]

The function case in this example uses P.args and P.kwargs in a manner that is expressly forbidden by PEP 612.

def case(*args: P.args, **kwargs: P.kwargs) -> Case[P]:
    return Case()

Mypy should generate an error here. P.args and P.kwargs can be used only if P is defined in an outer scope or defined in a parameter earlier in the signature.

Pyre was the reference implementation for PEP 612, and it emits an error here. Pyright does as well.

Perhaps this bug should be changed to cover this false negative.

[AlexWaygood]

Thanks @erictraut, I missed that (but in my defence, I only skimmed it earlier ;)

In that case, yes -- there's a mypy bug here, but the bug is that it's not rejecting the case function, which is improperly using ParamSpec.

[ilevkivskyi]

The error itself is a duplicate of #16296.

About the PEP 612: on one hand I understand the motivation, there is no meaningful mapping from actual kinds to formal kinds, but on the other hand this looks somewhat arbitrary, since e.g. a similar case def apply(fn: Callable[P, R], *args: P.args, **kwargs: P.kwargs) -> R: ... is allowed. If it feels important to anyone to prohibit this, please open a separate dedicated issue.

[orsinium]

So, what's the conclusion here? #16296 is going to be eventually fixed and with that my code example will start working again as well? Or it won't be fixed because the PEP says it shouldn't be supported and so I should find a different API for my package?

Another thing to note is that while the case function might be illegal, the inference fails not for case but for parametrize. I know type inference is hard but my expectation is that the type for parametrize should be inferred based on the first argument because P is invariant and known for the _test function.

[orsinium]

I changed the code to get rid of the constructor and not use the case forbidden by PEP, but the issue persists. The updated code example:

from typing import Callable, Generic, ParamSpec

P = ParamSpec('P')

class Case(Generic[P]):
    def __init__(self, *args: P.args, **kwargs: P.kwargs):
        pass

def _test(a: int):
    assert a

def parametrize(func: Callable[P, None], case: Case[P]):
    pass

parametrize(_test, Case(1))

[ilevkivskyi]

@orsinium Thanks for this example! It shows why the restriction in the PEP is arbitrary. PEP talks something about ParamSpec being "in scope", here it is clearly in scope. The real problem is, as I said, that actual kinds don't map meaningfully to formal kinds.

So the conclusion here is that your code will start working again when the other issue is fixed. Mypy will continue to support this (arguably tricky) use case on best effort basis. It may prohibit it at some point, if people will strongly prefer to do so.

[q0w]

with 1.7 it works for the valid case but fails with invalid

parametrize(_test, case(1)) # OK
parametrize(_test, case("1")) # Cannot infer type argument 1 of "parametrize"  [misc]

[orsinium]

It still fails for me for more involved cases, even for valid ones. The original example I posted passes for valid case, though. I'll try to make another example that reproduces the current failure and open a new issue.

[orsinium]

Created a new issue: #16485

Now it fails if b is passed first as a positional argument and then as a keyword one.