[pyqgis] RecursionError due to pandas.DataFrame or sklearn.metrics import

[vijdz]

Description
I am writing a standalone script with pyqgis and I noticed that pandas isn't working properly. Even if it is a pandas error we can see qgis.utils in the stack trace.

QGIS and OS versions
GNU Linux Debian 10 (buster)
QGIS 3.10.1 + Python3.7.3 + pandas0.23.3

Steps to reproduce

# init qgis
import sys
sys.path.append('/usr/share/qgis/python')                                                                                                                                                                                           
sys.path.append('/usr/share/qgis/python/plugins')

from qgis.core import (
		QgsApplication,
		QgsField,
		QgsRectangle,
		QgsVectorFileWriter,
		QgsVectorLayer,
		QgsProcessingFeedback,
		QgsProcessingException
	)
from qgis.analysis import QgsNativeAlgorithms
from PyQt5.QtCore import QVariant
import processing
from processing.core.Processing import Processing

qgs = QgsApplication([b''], False)
qgs.initQgis()
qgs.setPrefixPath('/usr')
Processing.initialize()
qgs.processingRegistry().addProvider(QgsNativeAlgorithms())
feedback = QgsProcessingFeedback()

# import pandas and try to create a DataFrame
import pandas as pd
pd.DataFrame({'test':[1,2,3]})

Stack trace

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pandas/core/base.py", line 82, in __repr__
    return str(self)
  File "/usr/lib/python3/dist-packages/pandas/core/base.py", line 61, in __str__
    return self.__unicode__()
  File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 663, in __unicode__
    line_width=width, show_dimensions=show_dimensions)
  File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 1970, in to_string
    show_dimensions=show_dimensions)
  File "/usr/lib/python3/dist-packages/pandas/io/formats/format.py", line 394, in __init__
    self.buf = _expand_user(_stringify_path(buf))
  File "/usr/lib/python3/dist-packages/pandas/io/common.py", line 150, in _stringify_path
    from py.path import local as LocalPath
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 746, in _import
    if mod and '__file__' in mod.__dict__:
  File "/usr/lib/python3/dist-packages/py/_vendored_packages/apipkg.py", line 168, in __dict__
    self.__makeattr(name)
  File "/usr/lib/python3/dist-packages/py/_vendored_packages/apipkg.py", line 148, in __makeattr
    result = importobj(modpath, attrname)
  File "/usr/lib/python3/dist-packages/py/_vendored_packages/apipkg.py", line 69, in importobj
    module = __import__(modpath, None, None, ['__doc__'])
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 744, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/lib/python3/dist-packages/py/_path/svnwc.py", line 9, in <module>
    import py
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 746, in _import
    if mod and '__file__' in mod.__dict__:
  File "/usr/lib/python3/dist-packages/py/_vendored_packages/apipkg.py", line 168, in __dict__
    self.__makeattr(name)
  File "/usr/lib/python3/dist-packages/py/_vendored_packages/apipkg.py", line 148, in __makeattr
    result = importobj(modpath, attrname)
  File "/usr/lib/python3/dist-packages/py/_vendored_packages/apipkg.py", line 69, in importobj
    module = __import__(modpath, None, None, ['__doc__'])
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 744, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/usr/lib/python3/dist-packages/py/__metainfo.py", line 1, in <module>
    import py
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 746, in _import
    if mod and '__file__' in mod.__dict__:

And it goes on... Any ideas ?

P.S. : This is my first issue ! I hope it is right.

[gioman]

P.S. : This is my first issue ! I hope it is right.

@vidlb if is a question rather than a bug report you must ask it in the users/developers mailing lists, not here. Cheers!

[vijdz]

P.S. : This is my first issue ! I hope it is right.

@vidlb if is a question rather than a bug report you must ask it in the users/developers mailing lists, not here. Cheers!

I don't think it comes from my script so yes it is a bug and not a question !

[roya0045]

possible dupllicate of #30446

[vijdz]

Same kind of traceback in #33191 . Closed but not explained. Seems like it also happens with any sklearn.metrics import.

[roya0045]

@vidlb I doubt that qgis must be friendly to external imports, maybe this should be more of a feature than a bug.

[vijdz]

@roya0045 you're right, but it is good to keep trace of those kind of issues, because many people may need external libs in their plugins. But yes, I guess it is not the best place to discuss it so I'm closing the issue.
I found out that a newer version of sklearn (0.22) seems to work fine. It may be the same with pandas.