Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: When setting python.analysis.typeshedPaths, use its stubs for typing absolute imports #5209

Closed
Avasam opened this issue Dec 4, 2023 · 6 comments
Assignees
Labels
user responded Was "waiting for user response" and they responded

Comments

@Avasam
Copy link

Avasam commented Dec 4, 2023

For example, in typeshed repository, I have the following configured:

    "python.analysis.typeshedPaths": [
        "${workspaceFolder}"
    ],

I'd expect absolute imports to find symbols from typeshed's own stubs, and "Go to type definition" to lead to typeshed's own .pyi (when it exists) instead of the installed library.

Here's a couple example:
image
image
When new modules get added (or a new stub gets worked on, which is full of new modules):
image

Here's the full config:

/*
 * Copy this file as `.vscode/settings.json` to configure VSCode for this workspace.
 * Unfortunately, VSCode doesn't (yet) offer any way to have "workspace defaults" or "user-worspace settings",
 * so offering defaults to copy is the best we can do at the moment.
 *
 * ⚠ Disclaimer: The typeshed team doesn't commit to maintaining this file. It exists purely for your ease of use.
*/
{
    // Don't format on save for formatters we don't explicitely control
    "editor.formatOnSave": false,
    "editor.codeActionsOnSave": {
        "source.fixAll": false
    },
    // Set file associations to support comments syntax highlight
    "files.associations": {
        "settings.default.json": "jsonc",
        "pyrightconfig*.json": "jsonc",
        ".flake8": "properties",
        "stubtest_allowlist*.txt": "properties",
        "**/stubtest_allowlists/*.txt": "properties",
        "pytype_exclude_list.txt": "properties"
    },
    "files.exclude": {
        "**/.*_cache": true, // mypy and Ruff cache
        "**/__pycache__": true
    },
    "files.eol": "\n",
    "files.insertFinalNewline": true,
    "files.trimFinalNewlines": true,
    "files.trimTrailingWhitespace": true,
    "editor.comments.insertSpace": true,
    "editor.insertSpaces": true,
    "editor.detectIndentation": false,
    "editor.tabSize": 2,
    "[json][jsonc][python][toml]": {
        "editor.tabSize": 4
    },
    "editor.rulers": [
        90,
        130
    ],
    "[git-commit]": {
        "editor.rulers": [
            72
        ]
    },
    // Format on save for formatters we explicitely control
    "[json][jsonc][yaml][python][toml]": {
        "editor.formatOnSave": true,
        "editor.codeActionsOnSave": {
            "source.fixAll": true,
            // Allow isort or Ruff to organize imports
            "source.organizeImports": true
        }
    },
    "[json][jsonc]": {
        "editor.defaultFormatter": "vscode.json-language-features"
    },
    "[yaml]": {
        "editor.defaultFormatter": "redhat.vscode-yaml"
    },
    "[toml]": {
        "editor.rulers": [
            90
        ],
        "editor.defaultFormatter": "tamasfe.even-better-toml"
    },
    "[python]": {
        "editor.rulers": [
            130
        ],
        "editor.defaultFormatter": "ms-python.black-formatter"
    },
    // python.analysis is Pylance (pyright) configurations
    "python.analysis.fixAll": [
        "source.unusedImports"
        // Explicitly omiting "source.convertImportFormat", some stubs use relative imports
    ],
    "python.analysis.typeshedPaths": [
        "${workspaceFolder}"
    ],
    "python.analysis.extraPaths": [
        "tests"
    ],
    "mypy-type-checker.importStrategy": "fromEnvironment",
    "mypy-type-checker.args": [
        "--custom-typeshed-dir=${workspaceFolder}",
        "--python-version=3.8",
        "--strict",
        // Needed because a library stubbed in typeshed won't necessarily be installed in the dev's environment
        "--ignore-missing-imports"
    ],
    // Ensure typeshed's configs are used, and not user's VSCode settings
    "flake8.args": [
        "--config=.flake8"
    ],
    "flake8.importStrategy": "fromEnvironment",
    "black-formatter.importStrategy": "fromEnvironment",
    // Using Ruff instead of isort
    "isort.check": false,
    "ruff.importStrategy": "fromEnvironment",
    "ruff.fixAll": true,
    "ruff.organizeImports": true,
    "evenBetterToml.formatter.alignComments": false,
    "evenBetterToml.formatter.alignEntries": false,
    "evenBetterToml.formatter.allowedBlankLines": 1,
    "evenBetterToml.formatter.arrayAutoCollapse": true,
    "evenBetterToml.formatter.arrayAutoExpand": true,
    "evenBetterToml.formatter.arrayTrailingComma": true,
    "evenBetterToml.formatter.columnWidth": 90,
    "evenBetterToml.formatter.compactArrays": true,
    "evenBetterToml.formatter.compactEntries": false,
    "evenBetterToml.formatter.compactInlineTables": false,
    "evenBetterToml.formatter.indentEntries": false,
    "evenBetterToml.formatter.indentTables": false,
    "evenBetterToml.formatter.inlineTableExpand": false,
    "evenBetterToml.formatter.reorderArrays": true,
    "evenBetterToml.formatter.trailingNewline": true,
    // We like keeping TOML keys in a certain non-alphabetical order that feels more natural
    "evenBetterToml.formatter.reorderKeys": false
}
@github-actions github-actions bot added the needs repro Issue has not been reproduced yet label Dec 4, 2023
@erictraut
Copy link
Contributor

What you're proposing would violate PEP 561, which says that typeshed stubs should always be resolved last. If you want a stub package to override the library's own type information, you can install the stub library in your Python environment. As you can see from PEP 561, stub libraries are resolved before normal packages. All of the typeshed third-party stubs are regularly published as stub packages in pypi, so installing them in your environment is easy to do.

@debonte debonte added waiting for user response Requires more information from user and removed needs repro Issue has not been reproduced yet labels Dec 4, 2023
@Avasam
Copy link
Author

Avasam commented Dec 4, 2023

If you want a stub package to override the library's own type information, you can install the stub library in your Python environment. [...] All of the typeshed third-party stubs are regularly published as stub packages in pypi, so installing them in your environment is easy to do.

That can't be done when the stub library, or the module, doesn't exist yet. Or even if the current module is being modified in an incompatible manner (like adding generics to class). My request is for development within typeshed itself in mind.

If it's really not possible that's fine. I was looking for improvements of the dev experience in typeshed (which I know is a special case when it comes to stub repos)

@github-actions github-actions bot added user responded Was "waiting for user response" and they responded and removed waiting for user response Requires more information from user labels Dec 4, 2023
@erictraut
Copy link
Contributor

This is very specific to your particular use case. I don't think we should implement hacks or exceptions (violations of PEP 561) specifically for this. Perhaps you could use an editable install when developing the stub package?

@Avasam
Copy link
Author

Avasam commented Dec 4, 2023

For posterity, exploring a solution using PYTHONPATH:
python/typeshed#11102 (comment)

Ok so configuring python.autoComplete.extraPaths doesn't seem to do anything:

    "python.autoComplete.extraPaths": [
        "${workspaceFolder}/stubs/redis"
    ]

However, there's python.envFile which defaults to ${workspaceFolder}/.env. I tried with the following content which worked:

PYTHONPATH=stubs/redis

I could change the default workspace setting to ${workspaceFolder}/.vscode/.env and create a .env file in .vscode since it's specific to that editor (same as any other vscode config). Unless someone tells me this can be reused by other editors which have similar issues. But having to maintain the full list of stubs in there is... eh. And the path separator changes based on OS. At least it's progress and I can update it for myself.

@Avasam
Copy link
Author

Avasam commented Dec 4, 2023

If python.analysis.stubPath supported multiple paths and/or wildcard, that could also work.
image

Which may also be more sensible as that's exactly what's happening with typeshed: Multiple custom "typings" folder for local stubs.

@Avasam
Copy link
Author

Avasam commented Dec 7, 2023

After further exploration, my issue specific to redis might be due to the base library being marked as py.typed despite being very partially typed. Removing the py.typed marker in the source code fixes the linking issue to typeshed stubs.

For the new stubs networkx: It's still bundled from microsoft/python-type-stubs , deleting it from the bundled pylance stubs fixed the linking issue to local typeshed stubs.

identical openpyxl issues I had in the past: same story as networkx probably. Before it was removed from microsoft/python-type-stubs


  1. For redis: I can see why a library marked as typed should have its own annotations prioritized over the "bundled" stubs. It's an unfortunate edge case here. But can be worked around. (relates to discussions in Remove redis (not before February 2025) python/typeshed#10592 (comment))

  2. For stubs bundled from microsoft/python-type-stubs, I guess it'd be nice if typeshed from python.analysis.typeshedPaths had priority here? But I'm also probably the only one who's been hitting that constantly ^^"


If there is no action to be taken by pylance upon those two points. Please close this issue. I think I've just unfortunately been hitting all the specific edge cases leading me to believe this was a more widespread issue.

@debonte debonte closed this as completed Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
user responded Was "waiting for user response" and they responded
Projects
None yet
Development

No branches or pull requests

3 participants