Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spyder 6.0a1 - Pandas error crashes console #21149

Closed
mg3146 opened this issue Jul 16, 2023 · 14 comments · Fixed by #21184
Closed

Spyder 6.0a1 - Pandas error crashes console #21149

mg3146 opened this issue Jul 16, 2023 · 14 comments · Fixed by #21184
Assignees
@ccordoba12
Labels
component:IPython Console severity:1-Critical type:Bug
Milestone
v6.0alpha2

Comments

@mg3146
Copy link

mg3146 commented Jul 16, 2023

Hi - quick bug. It seems like when you get a pandas error, like this one, it kills the Spyder iPython console:
image
@mg3146
Copy link
Author

mg3146 commented Jul 17, 2023 •
edited
Loading

Here is a reproducible example.

import pandas as pd

data = range(5)
index = range(3)
cols = range(4)

df = pd.DataFrame(data = data, columns = cols, index = index)

If there was a way to fix this one for the next release, I would really appreciate it. As its killing my productivity (I still seem to have issues with 5), and I love using Spyder

image

@ccordoba12
Copy link
Member

Hey @mg3146, thanks for reporting. Are you getting these errors when using Spyder 5?

@ccordoba12 ccordoba12 self-assigned this Jul 17, 2023
@mg3146
Copy link
Author

mg3146 commented Jul 17, 2023

I can't even get Spyder 5 to launch anymore, so I've just been using 6. This wasn't a problem for me in Spyder 5 though when I was using it. Are you not able to reproduce?

@ccordoba12
Copy link
Member

ccordoba12 commented Jul 17, 2023 •
edited
Loading

Not on Linux. @dalthviz, can you reproduce this problem on Windows?

@mg3146
Copy link
Author

mg3146 commented Jul 17, 2023 •
edited
Loading

maybe its something to do with this corrupted environment of mine that's causing issues with both this and Spyder 5. If it helps, when I try to launch Spyder 5 from the anaconda powershell prompt I get the following traceback before it crashes:
image

@ccordoba12
Copy link
Member

If you have Spyder 5 and 6 in the same env, I think that should be the cause. Please create a new env for Spyder 6 and report back if you're still having this problem.

@mg3146
Copy link
Author

mg3146 commented Jul 19, 2023 •
edited by ccordoba12
Loading

Took a look into that. Spyder 6 I used your standalone installer. For the console, I had originally been using an environment that Spyder 5 installed in it (via conda install spyder). I created a new environment without Spyder 5 installed, and I'm still getting the same crash / bug.

Here is a conda list of the environment I'm using for the console.

aiofiles                  22.1.0           py39haa95532_0
aiohttp                   3.8.4                    pypi_0    pypi
aiosignal                 1.2.0              pyhd3eb1b0_0
aiosqlite                 0.18.0           py39haa95532_0
anyio                     3.5.0            py39haa95532_0
appdirs                   1.4.4              pyhd3eb1b0_0
argon2-cffi               21.3.0             pyhd3eb1b0_0
argon2-cffi-bindings      21.2.0           py39h2bbff1b_0
arrow                     1.2.3                    pypi_0    pypi
astroid                   2.14.2           py39haa95532_0
asttokens                 2.0.5              pyhd3eb1b0_0
async-timeout             4.0.2            py39haa95532_0
attrs                     23.1.0                   pypi_0    pypi
autopep8                  1.6.0              pyhd3eb1b0_1
babel                     2.11.0           py39haa95532_0
backcall                  0.2.0              pyhd3eb1b0_0
bcrypt                    3.2.0            py39h2bbff1b_1
beanie                    1.18.0                   pypi_0    pypi
beautifulsoup4            4.12.0           py39haa95532_0
bidict                    0.21.2             pyhd3eb1b0_0
black                     23.3.0           py39haa95532_0
blas                      1.0                         mkl
bleach                    4.1.0              pyhd3eb1b0_0
blpapi                    3.19.3                   pypi_0    pypi
boost-cpp                 1.73.0              h2bbff1b_12
boto3                     1.26.137                 pypi_0    pypi
botocore                  1.29.137                 pypi_0    pypi
bottleneck                1.3.5            py39h080aedc_0
brotli                    1.0.9                h2bbff1b_7
brotli-bin                1.0.9                h2bbff1b_7
brotlipy                  0.7.0           py39h2bbff1b_1003
bs4                       4.12.0           py38hd3eb1b0_0
ca-certificates           2023.05.30           haa95532_0
cachetools                5.3.0                    pypi_0    pypi
cairo                     1.16.0               haedb8bc_5
certifi                   2023.5.7         py39haa95532_0
cffi                      1.15.1           py39h2bbff1b_3
charset-normalizer        3.1.0                    pypi_0    pypi
click                     8.1.3                    pypi_0    pypi
cloudpickle               2.2.1            py39haa95532_0
colorama                  0.4.6            py39haa95532_0
comm                      0.1.2            py39haa95532_0
contourpy                 1.0.7                    pypi_0    pypi
convertdate               2.3.2              pyhd3eb1b0_0
cryptography              41.0.1                   pypi_0    pypi
cycler                    0.11.0             pyhd3eb1b0_0
debugpy                   1.5.1            py39hd77b12b_0
decorator                 5.1.1              pyhd3eb1b0_0
defusedxml                0.7.1              pyhd3eb1b0_0
dill                      0.3.6            py39haa95532_0
docstring-to-markdown     0.11             py39haa95532_0
docutils                  0.18.1           py39haa95532_3
entrypoints               0.4              py39haa95532_0
exceptiongroup            1.1.1                    pypi_0    pypi
executing                 0.8.3              pyhd3eb1b0_0
expat                     2.4.9                h6c2663c_0
flake8                    6.0.0            py39haa95532_0
font-ttf-dejavu-sans-mono 2.37                 hd3eb1b0_0
font-ttf-inconsolata      2.001                hcb22688_0
font-ttf-source-code-pro  2.030                hd3eb1b0_0
font-ttf-ubuntu           0.83                 h8b1ccd4_0
fontconfig                2.14.1               h9c4af85_2
fonts-anaconda            1                    h8fa9717_0
fonts-conda-ecosystem     1                    hd3eb1b0_0
fonttools                 4.25.0             pyhd3eb1b0_0
fqdn                      1.5.1                    pypi_0    pypi
freetype                  2.12.1               ha860e81_0
fribidi                   1.0.10               h62dcd97_0
frozenlist                1.3.3            py39h2bbff1b_0
getopt-win32              0.1                  h2bbff1b_0
giflib                    5.2.1                h8cc25b3_3
gitdb                     4.0.7              pyhd3eb1b0_0
gitpython                 3.1.30           py39haa95532_0
glib                      2.69.1               h5dc1a3c_2
gprof2dot                 2022.7.29                pypi_0    pypi
graphite2                 1.3.14               hd77b12b_1
graphviz                  2.50.0               h7eca76f_1
greenlet                  2.0.1            py39hd77b12b_0
grpcio                    1.53.0                   pypi_0    pypi
gst-plugins-base          1.18.5               h9e645db_0
gstreamer                 1.18.5               hd78058f_0
gts                       0.7.6                h63ab5a1_3
harfbuzz                  4.3.0                hb646838_1
hijri-converter           2.2.2              pyhd3eb1b0_0
holidays                  0.18             py39haa95532_0
icc_rt                    2022.1.0             h6049295_2
icu                       58.2                 ha925a31_3
idna                      3.4              py39haa95532_0
importlib-metadata        6.0.0            py39haa95532_0
importlib_metadata        6.0.0                hd3eb1b0_0
importlib_resources       5.2.0              pyhd3eb1b0_1
iniconfig                 2.0.0                    pypi_0    pypi
intel-openmp              2021.4.0          haa95532_3556
ipykernel                 6.19.2           py39hd4e2768_0
ipynb-py-convert          0.4.6              pyh9f0ad1d_0    conda-forge
ipython                   8.14.0                   pypi_0    pypi
ipython_genutils          0.2.0              pyhd3eb1b0_1
ipywidgets                8.0.4            py39haa95532_0
isoduration               20.11.0                  pypi_0    pypi
isort                     5.9.3              pyhd3eb1b0_0
jedi                      0.18.1           py39haa95532_1
jinja2                    3.1.2            py39haa95532_0
jmespath                  0.10.0             pyhd3eb1b0_0
jpeg                      9e                   h2bbff1b_1
json5                     0.9.6              pyhd3eb1b0_0
jsonpointer               2.4                      pypi_0    pypi
jsonschema                4.18.2                   pypi_0    pypi
jsonschema-specifications 2023.6.1                 pypi_0    pypi
jupyter                   1.0.0            py39haa95532_8
jupyter_client            8.1.0            py39haa95532_0
jupyter_console           6.6.3            py39haa95532_0
jupyter_core              5.3.0            py39haa95532_0
jupyter_events            0.6.3            py39haa95532_0
jupyter_server            2.5.0            py39haa95532_0
jupyter_server_fileid     0.9.0            py39haa95532_0
jupyter_server_terminals  0.4.4            py39haa95532_1
jupyter_server_ydoc       0.8.0            py39haa95532_1
jupyter_ydoc              0.2.4            py39haa95532_0
jupyterlab                3.6.3            py39haa95532_0
jupyterlab_pygments       0.1.2                      py_0
jupyterlab_server         2.22.0           py39haa95532_0
jupyterlab_widgets        3.0.5            py39haa95532_0
kaleido                   0.2.1                    pypi_0    pypi
kiwisolver                1.4.4            py39hd77b12b_0
korean_lunar_calendar     0.2.1              pyhd3eb1b0_0
krb5                      1.19.4               h5b6d351_0
lazy-model                0.0.5                    pypi_0    pypi
lazy-object-proxy         1.6.0            py39h2bbff1b_0
lcms2                     2.12                 h83e58a3_0
lerc                      3.0                  hd77b12b_0
libboost                  1.73.0              h6c2663c_12
libbrotlicommon           1.0.9                h2bbff1b_7
libbrotlidec              1.0.9                h2bbff1b_7
libbrotlienc              1.0.9                h2bbff1b_7
libclang                  14.0.6          default_hb5a9fac_1
libclang13                14.0.6          default_h8e68704_1
libcurl                   8.1.1                h86230a5_1
libdeflate                1.17                 h2bbff1b_0
libffi                    3.4.4                hd77b12b_0
libgd                     2.3.3                hd77b12b_2
libiconv                  1.16                 h2bbff1b_2
libogg                    1.3.5                h2bbff1b_1
libpng                    1.6.39               h8cc25b3_0
libsodium                 1.0.18               h62dcd97_0
libssh2                   1.10.0               hcd4344a_2
libtiff                   4.5.0                h6c2663c_2
libvorbis                 1.3.7                he774522_0
libwebp                   1.2.4                hbc33d0d_1
libwebp-base              1.2.4                h2bbff1b_1
libxml2                   2.10.3               h0ad7f3c_0
libxslt                   1.1.37               h2bbff1b_0
lxml                      4.9.2            py39h2bbff1b_0
lz4-c                     1.9.4                h2bbff1b_0
mako                      1.2.3            py39haa95532_0
markupsafe                2.1.1            py39h2bbff1b_0
matplotlib                3.7.1            py39haa95532_1
matplotlib-base           3.7.1            py39hf11a4ad_1
matplotlib-inline         0.1.6            py39haa95532_0
mccabe                    0.7.0              pyhd3eb1b0_0
mistune                   0.8.4           py39h2bbff1b_1000
mkl                       2021.4.0           haa95532_640
mkl-service               2.4.0            py39h2bbff1b_0
mkl_fft                   1.3.1            py39h277e83a_0
mkl_random                1.2.2            py39hf11a4ad_0
motor                     3.1.2                    pypi_0    pypi
multidict                 6.0.2            py39h2bbff1b_0
munkres                   1.1.4                      py_0
mypy_extensions           0.4.3            py39haa95532_1
nbclassic                 0.5.5            py39haa95532_0
nbclient                  0.5.13           py39haa95532_0
nbconvert                 6.5.4            py39haa95532_0
nbformat                  5.7.0            py39haa95532_0
nest-asyncio              1.5.6            py39haa95532_0
networkx                  2.8.4            py39haa95532_1
notebook                  6.5.3            py39haa95532_0
notebook-shim             0.2.2            py39haa95532_0
numexpr                   2.8.4            py39h5b0cc5e_0
numpy                     1.23.5           py39h3b20f71_0
numpy-base                1.23.5           py39h4da318b_0
openai                    0.27.4                   pypi_0    pypi
openjpeg                  2.4.0                h4fc8c34_0
openssl                   1.1.1u               h2bbff1b_0
packaging                 23.0             py39haa95532_0
pandas                    1.5.3            py39hf11a4ad_0
pandocfilters             1.5.0              pyhd3eb1b0_0
pango                     1.50.7               h78c2152_0
paramiko                  2.11.0                   pypi_0    pypi
parso                     0.8.3              pyhd3eb1b0_0
pathspec                  0.10.3           py39haa95532_0
pcre                      8.45                 hd77b12b_0
pickleshare               0.7.5           pyhd3eb1b0_1003
pillow                    9.4.0            py39hd77b12b_0
pip                       23.1.2           py39haa95532_0
pixman                    0.40.0               h2bbff1b_1
platformdirs              2.5.2            py39haa95532_0
pluggy                    1.0.0            py39haa95532_1
ply                       3.11             py39haa95532_0
pooch                     1.4.0              pyhd3eb1b0_0
poppler                   22.12.0              h268424c_0
poppler-data              0.4.11               haa95532_1
powershell_shortcut       0.0.1                         3
prometheus_client         0.14.1           py39haa95532_0
prompt-toolkit            3.0.36           py39haa95532_0
prompt_toolkit            3.0.36               hd3eb1b0_0
protobuf                  4.22.1                   pypi_0    pypi
psutil                    5.9.0            py39h2bbff1b_0
pure_eval                 0.2.2              pyhd3eb1b0_0
pycodestyle               2.10.0           py39haa95532_0
pycountry                 22.3.5                   pypi_0    pypi
pycparser                 2.21               pyhd3eb1b0_0
pydantic                  1.10.7                   pypi_0    pypi
pydocstyle                6.3.0            py39haa95532_0
pydot                     1.4.1            py39haa95532_0
pyflakes                  3.0.1            py39haa95532_0
pygments                  2.15.1           py39haa95532_1
pylint                    2.16.2           py39haa95532_0
pymeeus                   0.5.11             pyhd3eb1b0_1
pynacl                    1.5.0            py39h8cc25b3_0
pyopenssl                 23.0.0           py39haa95532_0
pyparsing                 3.0.9            py39haa95532_0
pyqt                      5.15.7           py39hd77b12b_0
pyqt5-sip                 12.11.0          py39hd77b12b_0
pyrsistent                0.18.0           py39h196d8e1_0
pysftp                    0.2.9              pyhd3eb1b0_1
pyside6                   6.5.0                    pypi_0    pypi
pyside6-addons            6.5.0                    pypi_0    pypi
pyside6-essentials        6.5.0                    pypi_0    pypi
pysocks                   1.7.1            py39haa95532_0
pytest                    7.2.2                    pypi_0    pypi
python                    3.9.16               h6244533_2
python-dateutil           2.8.2              pyhd3eb1b0_0
python-fastjsonschema     2.16.2           py39haa95532_0
python-json-logger        2.0.7            py39haa95532_0
python-lsp-black          1.2.1            py39haa95532_0
python-lsp-jsonrpc        1.0.0              pyhd3eb1b0_0
python-lsp-server         1.7.1            py39haa95532_0
pytoolconfig              1.2.5            py39haa95532_1
pytz                      2022.7           py39haa95532_0
pywin32                   305              py39h2bbff1b_0
pywinpty                  2.0.10           py39h5da7b33_0
pyxll                     5.6.4                    pypi_0    pypi
pyxll-jupyter             0.4.1                    pypi_0    pypi
pyyaml                    6.0              py39h2bbff1b_1
pyzmq                     25.1.0           py39hd77b12b_0
qt-main                   5.15.2               he8e5bd7_8
qt-webengine              5.15.9               hb9a9bb5_5
qtconsole                 5.4.2            py39haa95532_0
qtpy                      2.2.0            py39haa95532_0
qtwebkit                  5.212                h2bbfb41_5
referencing               0.29.1                   pypi_0    pypi
requests                  2.29.0           py39haa95532_0
rfc3339-validator         0.1.4            py39haa95532_0
rfc3986-validator         0.1.1            py39haa95532_0
rope                      1.7.0            py39haa95532_0
rpds-py                   0.8.10                   pypi_0    pypi
s3transfer                0.6.0            py39haa95532_0
scipy                     1.10.1           py39h321e85e_0
send2trash                1.8.0              pyhd3eb1b0_1
setuptools                67.8.0           py39haa95532_0
shiboken6                 6.5.0                    pypi_0    pypi
sip                       6.6.2            py39hd77b12b_0
six                       1.16.0             pyhd3eb1b0_1
smmap                     4.0.0              pyhd3eb1b0_0
snakeviz                  2.1.1                    pypi_0    pypi
sniffio                   1.2.0            py39haa95532_1
snowballstemmer           2.2.0              pyhd3eb1b0_0
soupsieve                 2.4              py39haa95532_0
spyder-kernels            3.0.0b1                  pypi_0    pypi
sqlalchemy                1.4.39           py39h2bbff1b_0
sqlite                    3.41.2               h2bbff1b_0
stack_data                0.2.0              pyhd3eb1b0_0
tenacity                  8.0.1            py39haa95532_1
terminado                 0.17.1           py39haa95532_0
texttable                 1.6.4              pyhd3eb1b0_0
tinycss2                  1.2.1            py39haa95532_0
tk                        8.6.12               h2bbff1b_0
toml                      0.10.2             pyhd3eb1b0_0
tomli                     2.0.1            py39haa95532_0
tomlkit                   0.11.1           py39haa95532_0
tornado                   6.2              py39h2bbff1b_0
tqdm                      4.65.0                   pypi_0    pypi
traitlets                 5.7.1            py39haa95532_0
typing-extensions         4.6.3            py39haa95532_0
typing_extensions         4.6.3            py39haa95532_0
tzdata                    2023c                h04d1e81_0
ujson                     5.4.0            py39hd77b12b_0
uri-template              1.3.0                    pypi_0    pypi
urllib3                   1.26.15          py39haa95532_0
vc                        14.2                 h21ff451_1
vf-core                   2.5                      pypi_0    pypi
vs2015_runtime            14.27.29016          h5e58377_2
wcwidth                   0.2.5              pyhd3eb1b0_0
webcolors                 1.13                     pypi_0    pypi
webencodings              0.5.1            py39haa95532_1
websocket-client          0.58.0           py39haa95532_4
whatthepatch              1.0.2            py39haa95532_0
wheel                     0.38.4           py39haa95532_0
widgetsnbextension        4.0.5            py39haa95532_0
win_inet_pton             1.1.0            py39haa95532_0
winpty                    0.4.3                         4
wrapt                     1.14.1           py39h2bbff1b_0
xmltodict                 0.12.0             pyhd3eb1b0_0
xz                        5.4.2                h8cc25b3_0
y-py                      0.5.9            py39hb6bf4ef_0
yaml                      0.2.5                he774522_0
yapf                      0.31.0             pyhd3eb1b0_0
yarl                      1.8.1            py39h2bbff1b_0
ypy-websocket             0.8.2            py39haa95532_0
zeromq                    4.3.4                hd77b12b_0
zipp                      3.11.0           py39haa95532_0
zlib                      1.2.13               h8cc25b3_0
zstd                      1.5.5                hd43e919_0

@ccordoba12
Copy link
Member

No worries, I tested on Windows and I was able to reproduce this problem with the code you provided above. We'll try to fix it for alpha2.

@ccordoba12 ccordoba12 added this to the v6.0alpha2 milestone Jul 19, 2023
@ccordoba12
Copy link
Member

@impact27, this is a very serious bug and it doesn't happen on 5.x, which means it should be related to the many changes you did to the way we handle kernels.

Could you take a look at it? Thanks!

@ccordoba12 ccordoba12 changed the title Spyder 6.0.0.0a - Pandas error crashes console Spyder 6.0a1 - Pandas error crashes console Jul 19, 2023
@impact27
Copy link
Contributor

This is a tricky one. For some reason inspecting a frame while a pandas object is being created crashes python. This also means that debugging this would crash python. The problem here is that when a traceback is created in Spyder 6, the frames are inspected to provide helpful information about the crash. We could remove this but the underlying issue, which is probably a pandas issue, would still be there.

@impact27 impact27 mentioned this issue Jul 23, 2023
Merged
@impact27
Copy link
Contributor

Can you check if spyder-ide/spyder-kernels#464 fixes this issue?

@impact27
Copy link
Contributor

To reproduce, from a terminal:

% ipython
Python 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.14.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pandas as pd

In [2]: data = range(5)
   ...: index = range(3)
   ...: cols = range(4)

In [3]: %debug pd.DataFrame(data = data, columns = cols, index = index)
NOTE: Enter 'c' at the ipdb>  prompt to continue execution.
> <string>(1)<module>()

ipdb> s
--Call--
> /opt/homebrew/lib/python3.10/site-packages/pandas/core/frame.py(608)__init__()
    606     # Constructors
    607 
--> 608     def __init__(
    609         self,
    610         data=None,

ipdb> self.anythingatall
*** RecursionError: maximum recursion depth exceeded while calling a Python object
ipdb>

On macOS, I get a "RecursionError", on windows, python crashes.

@impact27
Copy link
Contributor

impact27 commented Jul 23, 2023 •
edited
Loading

Should I open a bug in pandas? It is not clear this is really an issue unless you are debugging, in which case you are interfering with the program flow. I think we should probably not get the traceback locals as this might get triggered.

@impact27 impact27 mentioned this issue Jul 24, 2023
Merged
@ccordoba12
Copy link
Member

Should I open a bug in pandas?

Yeah, please do.

It is not clear this is really an issue unless you are debugging, in which case you are interfering with the program flow. I think we should probably not get the traceback locals as this might get triggered.

I saw that you already did that, but I agree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ccordoba12 ccordoba12

Labels
component:IPython Console severity:1-Critical type:Bug
Projects
None yet
Milestone
v6.0alpha2
Development

Successfully merging a pull request may close this issue.

PR: Remove locals inspection from the kernel (IPython console) impact27/spyder
3 participants
@ccordoba12 @impact27 @mg3146