What does it mean to inherit from a generic type without specifying type parameters?

[vlasovskikh]

In the current implementation of typing.py the Iterator type is defined as follows:

class Iterable(Generic[T_co], extra=collections_abc.Iterable):
    pass

class Iterator(Iterable, extra=collections_abc.Iterator):  # Note: No generic types here
    pass

From the implementation point of view this means that Iterator is a generic type with T_co as its type parameter. @JukkaL argues that it should mean that Iterator is actually the concrete type Iterator[Any].

[gvanrossum]

Good catch. I only implemented this behavior (in GenericMeta IIRC) because I wasn't sure and I think Jukka wasn't around to respond in real time. I think Jukka's interpretation is fine. Can you start adding explicit type variables to all those cases and perhaps also think about how to implement Jukka's behavior? (And I guess the PEP needs to mention this, unless it's already there.)

[gvanrossum]

@JukkaL - I'm not sure what to do here. Is the PEP text up to date? Then it would just be a bug in my typing.py. Or does the PEP need to be updated? Or do we still need to decide what to do?

[gvanrossum]

I just chatted with @JukkaL. The intention is clear. The PEP text could be clearer. The implementation in typing.py should be changed; at the very least the generic classes should all use explicitly type variables.

[gvanrossum]

Updated the PEP. Still need to fix this in typing.py.

[ilevkivskyi]

Currently typing.py contains

class ItemsView(MappingView, Generic[KT, VT_co],
                extra=collections_abc.ItemsView):
    pass

But in the context of this issue should MappingView, Generic[KT, VT_co] be also replaced with MappingView[KT], Generic[VT_co]?

[gvanrossum]

I think a type checker should infer that if d is a Dict[str, int] then
d.items() is a MappingView[Tuple[str, int]] -- which suggests that
ItemsView should really just be like this:

class ItemsView(MappingView[T_co],
                extra=collections_abc.ItemsView):
    pass

Or alternatively we could use a type variable that is bound to Tuple? I'm
not quite sure how to spell this, TBH.

[ilevkivskyi]

I think ideally it would be:

class ItemsView(MappingView[Tuple[KT, VT_co]],
                extra=collections_abc.ItemsView):
    pass

however, this is currently impossible. There is a TODO in typing.py for enabling this construct and I think it would be useful in general. In any case, having MappingView without type parameter is not good. As a quick workaround, I would agree with your idea of having a type parameter bound to Tuple.

[gvanrossum]

I've got a feeling there's still an issue here (separate from #115). Several examples in the PEP create generic classes by simply subclassing an existing generic class and providing new type variables; e.g.

class LinkedList(Iterable[T], Container[T]):
    ...

However mypy reject this -- you have to add Generic[T] into the mix. I'm not sure whether to claim that mypy is wrong here or whether to change this in the PEP. For simple cases it seems that the need to add Generic[T] is just redundant -- but there are also complex cases that typing.py currently doesn't allow at all but that would be disambiguated by this rule, e.g.

class B(Generic[X, Y]): ...
class C(B[Tuple[Y, X], Z]): ...

What should the parameters of C be in this case, and in what order? If we require

class C(B[Tuple[Y, X], Z], Generic[X, Y, Z]): ...

there would be no ambiguity.

[gvanrossum]

Argh. This issue is actually about the interpretation of class C(Iterator): ... -- should it mean C(Iterator[T]) or C(Iterator[Any])?

I'm sorry for the mess. There are too many related problems and when I wrote typing.py I wasn't aware of either how mypy behaves or what the corner cases were.

In any case we should decide what to do for these problems in the PEP and in typing.py before CPython 3.5.2 goes out, and change mypy (if needed) when we have a decision.

[JukkaL]

I think that leaving Generic out from the base class list can be supported at least in many cases, but we should spell out when and how this works exactly. Also, for more complex cases we may want to require an explicit Generic.

For example, there rules would cover a lot of interesting cases and they would be unambiguous (though a little ad-hoc):

1. In cases where only a single base class in generic and this base class is of form C[T1, T2, ...], we'd infer Generic[T1, T2, ...] automatically. Example: class D(C[T1, T2], CC): ... would be shorthand for class D(C[T1, T2], CC, Generic[T1, T2]): ...

2. There is only a single type variable T in the list of base classes, but it can be repeated. We'd infer Generic[T] automatically. Example: class C(Iterable[T], Container[T]): ... would be shorthand for class C(Iterable[T], Container[T], Generic[T]): ...

3. In all other cases an explicit Generic[...] would be required. This would be fine:

   # Generic[...] required, because not in form 1 or 2
   class C(B[Tuple[Y, X], Z], Generic[X, Y, Z]): ...
   

Alternatively, we could make Generic always optional without restricting the form of generic base classes. Here is a possible way to define it:

If there is no Generic base class, find all type variable references in the base class list from left to right (in textual order). Remove duplicates (keep the leftmost one in case of duplicates). If the remaining type variables are T1, T2, ... (and this list is non-empty), add an implicit Generic[T1, T2, ...] base class. Not sure whether the implicit base class should be visible at runtime, or whether it should be a type check time thing only. Example:

 class C(B[Tuple[Y, X], Z]): ... # implicit Generic[Y, X, Z] base class

Using Iterator as a base class should probably be equivalent to Iterator[Any] for consistency. It would be strange to have that mean Iterator[T] since we wouldn't know which type variable T should be bound in the class body. Also, elsewhere Iterator means Iterator[Any]. A type checker can require an explicit type argument in a 'strict' mode.

[ilevkivskyi]

I agree with @JukkaL , it is more consistent if Iterator means Iterator[Any], moreover it is already mentioned in documentation https://docs.python.org/3/library/typing.html that:
"""
...Subclassing a generic class without specifying type parameters assumes Any for each position. In the following example, MyIterable is not generic but implicitly inherits from Iterable[Any]...
"""
Also, I like the second option proposed by Jukka about implicitly inserting a Generic with type variables in textual order, it seems more consistent with what is discussed in #115

[gvanrossum]

OK, let's go with Jukka's most elaborate plan for the PEP. It's fine if mypy catches up in two phases: first the simple rules, later the more advanced version. After all, we can always make mypy accept something by just adding Generic[...], which is always fine for the PEP. Also agreed plain Iterator means Iterator[Any].

[gvanrossum]

The PEP already includes the correct language about Iterator meaning Iterator[Any]; it's just typing.py that doesn't implement this.

I've just added some words (examples, really) to the PEP that imply Jukka's simpler set of rules in rev eeb7393. Please review.

[gvanrossum]

I'm closing this issue, because (a) the original question has been answered (it means X[Any]) and (b) I've updated the PEP with text that follows Jukka's proposal. I'm keeping #115 open to track the necessary changes to typing.py for both issues.