Using "attrgetter" as a parameter in sorted() method produces an error

[nvishnya]

When using attrgetter as a key to sorted(), mypy treats attrgetter as a type, not as sorted() second parameter. Although when using lambda as a key, mypy doesn't treat it that way.

from _operator import attrgetter
from typing import List

class Fruit:
    def __init__(self, fruit_type: str, color: str)-> None:
        self.fruit_type = fruit_type
        self.color = color  
    def __str__(self):
        return "%s: (%s, %s)" %(self.__class__.__name__, self.fruit_type, self.color)


def sort_lambda(list_to_sort: List[Fruit]):
    return sorted(list_to_sort, key = lambda fruit: fruit.fruit_type)


def sort_getter(list_to_sort: List[Fruit]):
        return sorted(list_to_sort, key =  attrgetter('color', 'fruit_type'))

Here's what I get when I try to type check the code above:

$ mypy sorting_objects.py
sorting_objects.py: note: In function "sort_getter":
sorting_objects.py:23: error: Argument 2 to "sorted" has incompatible type "attrgetter"; expected Callable[[Any], Any]

[gvanrossum]

Hmm... That's an interesting one!

Why are you using _operator instead of operator? The latter is the public, documented API, and doesn't give this error.

That said, the reason it works with operator is that the definition there (in typeshed/stdlib/2and3/operator.pyi) returns a Callable[[Any], Any] which means that it's not type-safe.

(I think it should have worked with the definition from typeshed/stdlib/3/_operator.pyi as well, but that's a known problem: #797.)

The problem with avoiding false negatives for attrgetter is that unless we special-case operator.attrgetter in mypy, the arguments to attrgetter are strings, and the return type of the function is not known until we know the argument type to which it will be applied. With a lambda, which is special syntax, mypy knows what's going on, but attrgetter currently is "just" another function.

It's similar for functools.partial: #1484.

It's also similar for getattr. Example:

class C:
  def __init__(self) -> None:
    self.x = 12
c = C()
x = c.x
y = getattr(c, 'x')
reveal_type(x)  # Revealed type is 'builtins.int'
reveal_type(y)  #  Revealed type is 'Any'

[nvishnya]

It can be quite misleading when importing attrgetter using PyDev's code completion because it suggests two modules:

[gvanrossum]

Does PyDev get that information from typeshed or from somewhere else?

[elazarg]

PyDev knows nothing about typeshed.

(Based on https://github.com/fabioz/Pydev/search?utf8=%E2%9C%93&q=typeshed )

[gvanrossum]

OK, then PyDev should learn not to suggest the builtin module corresponding to a .py module -- this would probably cause confusion for many other such module pairs (e.g. _pickle, _heapq, who knows what else, it's a common pattern). There's still the mypy issue with attrgetter of course. (And we should probably delete the `_operator` module -- or fix it so it's identical to operator?)

[nvishnya]

Maybe mypy should show a warning when a module like _operator is imported.

[gvanrossum]

How would you configure which imports deserve warnings?

[nvishnya]

This module can be deprecated somehow and warning can be shown for all deprecated modules.

[nvishnya]

Anyway, if mypy checks types, it should be able to check deprecations as well.

[gvanrossum]

I've got a feeling there's something missing. Maybe you could try to come up with a PR to see if you can make something that works.

[JukkaL]

Can you create a new issue about deprecated modules first? It's a separate issue from this, at least as it was originally reported, and it's confusing to mix two discussions.

[nvishnya]

Here is the issue about deprecation

[JukkaL]

The original issue is due to typeshed (created an issue: python/typeshed#835) so closing this now.