mypy incremental issues with `--warn-unused-ignores` #2960

timabbott · 2017-03-04T23:55:33Z

I just had the following sequence happen, which suggests to me that the mypy cache is broken with this option:

(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy 
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy --py3
zerver/lib/cache.py:287: note: unused 'type: ignore' comment
zproject/jinja2/__init__.py:32: note: unused 'type: ignore' comment
zerver/tornado/event_queue.py:474: note: unused 'type: ignore' comment
zerver/lib/actions.py:1199: note: unused 'type: ignore' comment
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy --py3^C
(zulip-py3-venv) tabbott@zaset:~/zulip$ rm -rf var/mypy-cache/
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy 
api/integrations/asana/zulip_asana_mirror:81: note: unused 'type: ignore' comment
api/integrations/asana/zulip_asana_mirror:82: note: unused 'type: ignore' comment
api/integrations/google/google-calendar:133: note: unused 'type: ignore' comment
api/integrations/google/google-calendar:143: note: unused 'type: ignore' comment
scripts/setup/generate_secrets.py:54: note: unused 'type: ignore' comment
tools/replace-tarball-shebang:22: note: unused 'type: ignore' comment
tools/show-profile-results.py:22: note: unused 'type: ignore' comment
tools/tests/test_template_parser.py:31: note: unused 'type: ignore' comment
zerver/lib/actions.py:1199: note: unused 'type: ignore' comment
zerver/lib/camo.py:18: note: unused 'type: ignore' comment
zerver/lib/parallel.py:32: note: unused 'type: ignore' comment
zerver/lib/test_helpers.py:175: note: unused 'type: ignore' comment
zerver/management/commands/email_mirror.py:112: note: unused 'type: ignore' comment
zerver/management/commands/email_mirror.py:119: note: unused 'type: ignore' comment
zerver/management/commands/email_mirror.py:128: note: unused 'type: ignore' comment
zerver/storage.py:45: note: unused 'type: ignore' comment
zerver/storage.py:75: note: unused 'type: ignore' comment
zerver/tests/test_upload.py:746: note: unused 'type: ignore' comment
zerver/tornado/ioloop_logging.py:24: note: unused 'type: ignore' comment
zproject/jinja2/__init__.py:32: note: unused 'type: ignore' comment
zproject/local_settings.py:14: note: unused 'type: ignore' comment
(zulip-py3-venv) tabbott@zaset:~/zulip$ ./tools/run-mypy 
(zulip-py3-venv) tabbott@zaset:~/zulip$

The text was updated successfully, but these errors were encountered:

gvanrossum · 2017-03-05T02:00:42Z

I think what happens here is that the data for --warn-unused-ignores is only collected when it parses a file, not when it reads the data for a file from the cache. So I guess in your first run some files were already in the cache.

I don't believe there's a bug to be fixed here (or at least I would prefer not to have to serialize the # type: ignore map into the cache). Maybe we can add a note to the docs.

timabbott · 2017-03-05T03:45:58Z

Hmm. Our model for how we use mypy is that we want to run it as part of our linter suite, along with pyflakes and friends (with the caching, it's basically fast enough). And the easy thing to do there is to just pass the -i option to mypy unconditionally (we do that today).

But if we do that, then suddenly this flag doesn't work at all, unless you know to delete the mypy cache each time you want to run it.

So I guess my vote is for updating the cache to include that map.

gvanrossum · 2017-03-05T05:06:27Z

I hear you -- but supporting this use case is a low priority for us.

Daenyth · 2017-06-06T14:22:34Z

I've been able to work around this for CI by making sure mypy_cache is deleted between runs.

msullivan · 2018-02-27T23:05:01Z

I am deep in a rabbit hole of trying to understand if there are going to be any unexpected ramifications to a change to the handling of suppressed that I want to make, and in the process I produced these two test cases that I believe are for this bug:

[case testIgnoreWarning1]
# cmd: mypy -m a b
# cmd2: mypy -m a b
# cmd3: mypy -m a b c
# flags: --follow-imports=error --warn-unused-ignores --verbose
[file a.py]
import b
import c  # type: ignore
[file b.py]
x = 1
[file b.py.2]
x = 'hi'
[file c.py]
pass
[out]
[out2]
[out3]
tmp/a.py:1: note: unused 'type: ignore' comment

[case testIgnoreWarning2]
# flags: --warn-unused-ignores --verbose
import a
[file a.py]
import b
import c  # type: ignore
[file b.py]
x = 1
[file b.py.2]
x = 'hi'
[file c.py.3]
pass
[out]
[out2]
[out3]
tmp/a.py:1: note: unused 'type: ignore' comment

They both fail in stage 2 with spurious unused type:ignore warnings.

msullivan · 2018-03-27T02:31:36Z

We may want to address this somehow (perhaps just by disallowing the combination for now) before making incremental the default mode per #4800.

msullivan · 2018-03-27T02:45:35Z

The snag here also does not actually seem to be that we don't write out the ignore errors map (which I think is fine), but that we don't generate error messages for failed imports from modules that are loaded from the cache initially but then later parsed due to changes in their dependencies.

JukkaL · 2018-03-27T08:25:42Z

Good point, I had forgotten about this. I've encountered myself. This may affect a non-trivial number of users.

gvanrossum · 2018-03-27T16:31:40Z

So what's the action item here?

JukkaL · 2018-03-27T19:30:09Z

We have to figure out what's the best way to get rid of these bogus messages -- either fix the underlying issue or disable the flag in incremental mode. I haven't spent enough time thinking about this to say how hard this would be to fix. If somebody has spare cycles to look at this, great. Otherwise I can probably look at this next week or so.

msullivan · 2018-03-27T20:27:59Z

I can take this. I have a half-finished refactor of the production of import diagnostics that should make both this and #4798 easy to fix

Daenyth · 2018-03-28T17:22:01Z

As a user I'd prefer the flag to not be disabled because it helps keep a project clean without having different test code paths for ci and local dev

…

On Tue, Mar 27, 2018, 4:28 PM Michael J. Sullivan ***@***.***> wrote: I can take this. I have a half-finished refactor of the production of import diagnostics that should make both this and #4798 <#4798> easy to fix — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2960 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA5NGBk0eDHjxIkyMTKRoExNP1UE0IQks5tiqDYgaJpZM4MTQX0> .

This is accomplished by generating diagnostics for suppressed dependencies before generating unused ignore notes. This requires that we store line information for suppressed dependencies in our cache files. Fixes #2960.

This is accomplished by generating diagnostics for suppressed dependencies before generating unused ignore notes. This requires that we store line information for suppressed dependencies in our cache files. Fixes #2960

#4841) This is accomplished by generating diagnostics for suppressed dependencies before generating unused ignore notes. This requires that we store line information for suppressed dependencies in our cache files. Fixes #2960

There is a problem with MyPy's implementation of --warn-unused-ignore flag, that depending on it's incremental or full run it will sometimes throw an "unused ignore" error (entirely randomly it seems). The problem is described (but only workarounded) in python/mypy#2960. The workaround is to disable --warn-unused-ignore flag. There is little harm in having unused ignores and we can clean them up from time to time easily.

Discussion at python/mypy#2960

There is a problem with MyPy's implementation of --warn-unused-ignore flag, that depending on it's incremental or full run it will sometimes throw an "unused ignore" error (entirely randomly it seems). The problem is described (but only workarounded) in python/mypy#2960. The workaround is to disable --warn-unused-ignore flag. There is little harm in having unused ignores and we can clean them up from time to time easily. GitOrigin-RevId: d93c1fdb26aba280d907ef431bae6c9984155a9c

Discussion at python/mypy#2960

There is a problem with MyPy's implementation of --warn-unused-ignore flag, that depending on it's incremental or full run it will sometimes throw an "unused ignore" error (entirely randomly it seems). The problem is described (but only workarounded) in python/mypy#2960. The workaround is to disable --warn-unused-ignore flag. There is little harm in having unused ignores and we can clean them up from time to time easily. GitOrigin-RevId: d93c1fdb26aba280d907ef431bae6c9984155a9c

gvanrossum added feature priority-2-low labels Mar 5, 2017

msullivan mentioned this issue Feb 27, 2018

Invalid "unused 'type ignore' comment" messages #4288

Closed

msullivan changed the title ~~mypy caching issues with --warn-unused-ignores~~ mypy incremental issues with --warn-unused-ignores Feb 27, 2018

JukkaL added priority-0-high and removed priority-2-low labels Mar 27, 2018

msullivan self-assigned this Mar 27, 2018

msullivan mentioned this issue Apr 2, 2018

Don't produce spurious unused type ignore messages in incremental mode #4841

Merged

msullivan closed this as completed in #4841 Apr 4, 2018

The-Compiler mentioned this issue Nov 27, 2018

Unused section warning with warn_unused_configs and cache #5957

Closed

potiuk mentioned this issue Sep 11, 2020

Disables --warn-unused-ignore flag for mypy apache/airflow#10880

Merged

unkcpz mentioned this issue May 25, 2021

BaseRestartWorkChain: deal with namespaced outputs aiidateam/aiida-core#4961

Merged

jace mentioned this issue Jun 24, 2021

mypy's warn_unused_ignores is unreliable, so disable it hasgeek/funnel#1213

Merged

jace added a commit to hasgeek/funnel that referenced this issue Jun 24, 2021

mypy's warn_unused_ignores is unreliable, so disable it (#1213)

86696f7

Discussion at python/mypy#2960

vidya-ram pushed a commit to hasgeek/funnel that referenced this issue Nov 23, 2021

mypy's warn_unused_ignores is unreliable, so disable it (#1213)

fadeba2

Discussion at python/mypy#2960

aaronkaplan mentioned this issue Feb 16, 2023

mypy cache issue in the github actions D4-project/IPASN-History#19

Closed

llucax mentioned this issue Aug 29, 2023

Upgrade to repo-config v0.5.2 frequenz-floss/frequenz-sdk-python#587

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mypy incremental issues with `--warn-unused-ignores` #2960

mypy incremental issues with `--warn-unused-ignores` #2960

timabbott commented Mar 4, 2017

gvanrossum commented Mar 5, 2017

timabbott commented Mar 5, 2017

gvanrossum commented Mar 5, 2017

Daenyth commented Jun 6, 2017

msullivan commented Feb 27, 2018

msullivan commented Mar 27, 2018

msullivan commented Mar 27, 2018

JukkaL commented Mar 27, 2018

gvanrossum commented Mar 27, 2018 via email

JukkaL commented Mar 27, 2018

msullivan commented Mar 27, 2018

Daenyth commented Mar 28, 2018 via email

mypy incremental issues with --warn-unused-ignores #2960

mypy incremental issues with --warn-unused-ignores #2960

Comments

timabbott commented Mar 4, 2017

gvanrossum commented Mar 5, 2017

timabbott commented Mar 5, 2017

gvanrossum commented Mar 5, 2017

Daenyth commented Jun 6, 2017

msullivan commented Feb 27, 2018

msullivan commented Mar 27, 2018

msullivan commented Mar 27, 2018

JukkaL commented Mar 27, 2018

gvanrossum commented Mar 27, 2018 via email

JukkaL commented Mar 27, 2018

msullivan commented Mar 27, 2018

Daenyth commented Mar 28, 2018 via email

mypy incremental issues with `--warn-unused-ignores` #2960

mypy incremental issues with `--warn-unused-ignores` #2960