mypy not inferring that variable cannot be None after assignment from dict.get

[sersorrel]

Code

from typing import Dict, Optional

d: Dict[str, str] = {}

def foo(arg: Optional[str] = None) -> None:
    if arg is None:
        arg = d.get("a", "foo bar")
    print(arg.split())

Expected output

None, since the code is correctly typed (afaict, at least)

Actual output

with --strict-optional:

mvce.py:10: error: Item "None" of "Optional[str]" has no attribute "split"

Inserting reveal_type(arg) before the print gives Union[builtins.str, builtins.None].

The following typechecks fine, so mypy definitely understands the return type of dict.get with two arguments:

d: Dict[str, str] = {}

a: str = d.get("a", "a")

Similarly, mypy can infer that arg cannot be None after an is None and simple assignment:

d: Dict[str, str] = {}

def foo(arg: Optional[str] = None) -> None:
    if arg is None:
        arg = "foo bar"
    reveal_type(arg)  # str
    print(arg.split())

but combining the two fails.

• Python 3.6.3
• mypy 0.580

[gvanrossum]

Good research! I can also make the error go away through a temporary variable:

a = d.get("a", "foo bar")
arg = a

I've got a feeling it's an bug in the type solver.

[komuw]

In my project, I have code like:

import typing


class Task:
    def __init__(self, name: typing.Union[None, str] = None) -> None:
        self.name = name
        if not self.name:
            self.name = "Komu"

    def caps(self) -> str:
        return self.name.upper()

when I run mypy on it I get:
error: Item "None" of "Optional[str]" has no attribute "upper"

I'm been forced to now right my code as:

class Task:

    def caps(self) -> str:
        # this assert is here to make mypy happy
        assert isinstance(self.name, str)
        return self.name.upper()

[JukkaL]

@komuw That looks like a different problem. Can you open a new issue for it?

[ilevkivskyi]

@komuw This is unrelated to this issue, and not actually a bug, mypy can't guess your intention for the ("long living" and externally visible) .name. You should rewrite your code to make your intention explicit. For example:

class Task:
    def __init__(self, name: Union[None, str] = None) -> None:
        if name is not None:
            self.name = name
        else:
            self.name = "Komu"

Or even simpler (assuming this is the actual code you wanted to type):

class Task:
    def __init__(self, name: str = "Komu") -> None:
        self.name = name

[komuw]

@ilevkivskyi

thanks for the pointer, re-writing as suggested makes mypy happy and is what I'll result to in my project.
I'm just surprised that mypy does not recognise that this two snippets are equivalent, and it should apply the same rules to both;

self.name = name
if not self.name:
    self.name = "Komu"

if name is not None:
    self.name = name
else:
    self.name = "Komu"

[ilevkivskyi]

and it should apply the same rules to both

No, it shouldn't.

Within the method body they result in same types for self.name (you can check this with reveal_type()). But outside the method (and outside the class) mypy uses the first inferred type for instance variables. In this case it is Optional[str] in first snippet and str in the second one.

If we would infer str for instance variables this would cause errors in other places:

task = Task()
name: Optional[str]
task.name = name  # Should this be valid?

Since this is a backwards incompatible change with high potential for breaking currently passing code, there is no way this will be changed.

[leahein]

I'm having a similar issue, but with Callables.

def outer(func: Optional[Callable] = None) -> Callable:
    if func is None:
        func = lambda: None
    def inner(*args, **kwargs) -> Any:
        return func(*args, **kwargs)
    return inner

error: "None" not callable

As gvanrossum mentioned, the workaround of assigning it to a new variable makes the error go away.

def outer(
    func: Optional[Callable] = None,
) -> Callable:
    if func is None:
        func = lambda: None
    my_func = func
    def inner(*args, **kwargs) -> Any:
        '''Inner function'''
        return my_func(*args, **kwargs)
    return inner

No error

[ilevkivskyi]

@leahein Your issue is actually different, see #2608 (the workaround is however the same).

[andresdelfino]

I've come across this issue (I believe).

Sharing a similar piece of code in case it helps in troubleshooting.

from typing import Optional

KUBERNETES_REGISTRIES_ALIASES = {
    'production': 'melchior-1.myorg.com',
    'staging': 'balthasar-2.myorg.com',
}

kubernetes_registry: Optional[str] = 'casper-3.myorg.com'  # from user input, may be None, an alias, or a full FQDN

if kubernetes_registry:
    reveal_type(kubernetes_registry)  # Revealed type is "builtins.str"
    kubernetes_registry = KUBERNETES_REGISTRIES_ALIASES.get(kubernetes_registry, kubernetes_registry)  # resolve alias (if it is an alias), otherwise use it as is
    reveal_type(kubernetes_registry)  # Revealed type is "Union[builtins.str, None]"

With mypy 0.910

[xtuchyna]

Hello, any update on this? I've encountered a very similar problem with v0.910

[alextriaca]

The same thing happens if you guard against Nones with an exception:

def __init__(self, my_var_id: str, my_var: Optional[MyVar] = None):
    self._my_var = my_var
    if self._my_var is None:
        self._my_var = self._lookup_my_var(my_var_id) # Function that may retrieve a result i.e. from a DB
    if self._my_var is None:
        raise Exception("My var does not exist")

self._my_var will be considered Optional in the rest of the code and produce the Item "None" of "Optional[MyVar]" has no attribute "..." error.

[shaybensasson]

In my case, I was certain that the instance is not None (although the library that I use defined it as Optional[TYPE]).
My solution was to use typing.cast to the underlying type.

# logger.parent is defined as Optional[logging.Logger]
prefect_logger = cast(logging.Logger, logger.parent)

One can argue if this is elegant or not, but this is a solution I'm happy with (it silenced the FA).

[shawalli]

@shaybensasson this solution worked for me as well, when I knew that if I didn't return by a certain point my variable was guaranteed not to be None

[erictraut]

@shaybensasson, rather than using a cast, you might consider using assert logger.parent is not None. This not only eliminates the type checking error, it also clearly documents your assumption in the code and provides runtime verification when running your program in debug mode.