Issues Following PyCharm Setup Instructions

[CF-FHB-X]

Description

Instructions for PyCharm Run/Debug Configuration no longer seem to be working when using PyCharm's Run button (throws ImportError in rich). Oddly enough, though, it works if you use PyCharm's Debug button.

Context

Setting up new kedro projects seems to be running into this error. The previous project we had set up a few weeks ago seems to be working fine.

Steps to Reproduce

1. Run kedro new (on Windows)
2. cd into new project directory and run py -m venv .venv (NB: we're not using conda - just vanilla venv from the std library)
3. pip install -s src\requirements.txt
4. Create a sample node and pipeline (for testing only)
5. Follow PyCharm Setup Instructions to set up kedro run
6. Try running it using Run button

Expected Result

It should run as per usual.

Actual Result

Instead we're getting ImportError from rich:

C:\Users\user01\PycharmProjects\test-kedro\.venv\Scripts\python.exe C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\__main__.py run 
Traceback (most recent call last):
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\__main__.py", line 6, in <module>
    from kedro.framework.cli import main
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\framework\cli\__init__.py", line 4, in <module>
    from .cli import main
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\framework\cli\cli.py", line 15, in <module>
    from kedro.framework.cli.catalog import catalog_cli
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\framework\cli\catalog.py", line 9, in <module>
    from kedro.framework.cli.utils import KedroCliError, env_option, split_string
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\framework\cli\utils.py", line 5, in <module>
    import logging
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\logging.py", line 11, in <module>
    import rich.logging
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\rich\logging.py", line 3, in <module>
    from logging import Handler, LogRecord
ImportError: cannot import name 'Handler' from partially initialized module 'logging' (most likely due to a circular import) (C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\logging.py)

Process finished with exit code 1

Your Environment

See previous note about using venv instead of conda (for a whole slew of reasons that I'll spare you from, conda isn't an option for us)

• Kedro version : 0.18.13
• Python version: 3.11.5
• Operating system and version: Windows 10

[noklam]

C:\Users\user01\PycharmProjects\test-kedro.venv\Scripts\python.exe C:\Users\user01\PycharmProjects\test-kedro.venv\Lib\site-packages\kedro_main_.py run

If you run this from your terminal does it fails too? how about kedro run in terminal?

You mentioned Debug button is working, could you also share what's the command that being run (should be simliar to above).

It would be helpful if you can provide rich version too.

[datajoely]

to @noklam 's point can you post a screenshot of your run config?

[CF-FHB-X]

If I run kedro run from the terminal, we're all good to go (everything's working).
If I run python .\.venv\Lib\site-packages\kedro\__main__.py run from the terminal, we're back to that ImportError again.
In terms to using the Debug vs. Run button in PyCharm, it's all from the same run configuration:

In terms of rich, I'm running 13.5.3, and our "last working kedro project" is running 13.5.2. I even tried downgrading a new test project to that same version, but still running into the same issues.

[noklam]

Not sure if this is the correct configuration, AFAIK it should be pointing to venv/bin/kedro or venv/bin/kedro.py (if Windows).

See this:

https://docs.kedro.org/en/stable/development/set_up_pycharm.html

[CF-FHB-X]

I was going off the same page, but it instructs us to replace that script path with the output from the following:

# macOS / Linux
which kedro

# Windows (in `cmd`)
python -c "import sys, os.path; print(os.path.join(os.path.dirname(sys.executable), 'Lib\site-packages\kedro\__main__.py'))"

[CF-FHB-X]

And to confirm, I can point the script path to .venv/Scripts/kedro.exe and Run works fine, but now Debug fails with:

SyntaxError: invalid or missing encoding declaration for 'C:\Users\user01\PycharmProjects\test-kedro\.venv\Scripts\kedro.exe'

[astrojuanlu]

File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\framework\cli\utils.py", line 5, in <module>
    import logging
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\logging.py", line 11, in <module>
    import rich.logging
  File "C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\rich\logging.py", line 3, in <module>
    from logging import Handler, LogRecord
ImportError: cannot import name 'Handler' from partially initialized module 'logging' (most likely due to a circular import) (C:\Users\user01\PycharmProjects\test-kedro\.venv\Lib\site-packages\kedro\logging.py)

There's something wrong with the PYTHONPATH in this environment. When kedro.framework.cli.utils does import logging it should fetch the standard library module, but instead it's fetching kedro.logging. Then rich.logging does import logging too but again it tries to fetch kedro.logging and this produces a circular import error.

We have to determine if it's our instructions that create this problem or something in your environment.

If I run python ..venv\Lib\site-packages\kedro_main_.py run from the terminal, we're back to that ImportError again.

Yeah I don't think users should run Python scripts from the .venv like that. If anything, the script should end up doing py -m kedro run or at most C:\Users\user01\PycharmProjects\test-kedro\.venv\Scripts\python.exe -m kedro run.

[CF-FHB-X]

Bingo! That last comment of yours seems to have fixed the issue: changing the docs that point to using the following script:
Lib\site-packages\kedro\__main__.py
to instead just call the kedro module like this:

Now I can run the config in both Run and Debug modes as per how things used to work!

So seems this should just be a quick update to the PyCharm part of the docs...?

[astrojuanlu]

Glad this helped @CF-FHB-X ! 🙌🏽 Too many hours spent fighting Python path issues made me an expert... 😄

Turning this into a documentation issue.

[astrojuanlu]

Would you like to send a PR yourself?

[CF-FHB-X]

Sure thing! Prob won't be until later today or tomorrow, but I'll definitely make those changes!

Thanks for the help isolating them!

[stichbury]

This issue is as good as closed as we're just waiting for DCO signoff to merge. So I'm closing it