BUG: assert_series_equal raises when series contains dict of np.arrays

[isVoid]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import numpy as np
s = pd.Series([{'a': np.array(['xx', 'yy'])}])
s2 = pd.Series([{'a': np.array(['xx', 'yy'])}])
pd.testing.assert_series_equal(s, s2)

Problem description

Raises:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-94-c9299c384117> in <module>
      2 s = pd.Series([{'a': np.array(['xx', 'yy'])}])
      3 s2 = pd.Series([{'a': np.array(['xx', 'yy'])}])
----> 4 pd.testing.assert_series_equal(s, s2)

~/compose/etc/conda/cuda_11.2/envs/notebooks/lib/python3.7/site-packages/pandas/_testing.py in assert_series_equal(left, right, check_dtype, check_index_type, check_series_type, check_less_precise, check_names, check_exact, check_datetimelike_compat, check_categorical, check_category_order, check_freq, check_flags, rtol, atol, obj)
   1482             check_dtype=check_dtype,
   1483             obj=str(obj),
-> 1484             index_values=np.asarray(left.index),
   1485         )
   1486 

pandas/_libs/testing.pyx in pandas._libs.testing.assert_almost_equal()

pandas/_libs/testing.pyx in pandas._libs.testing.assert_almost_equal()

~/compose/etc/conda/cuda_11.2/envs/notebooks/lib/python3.7/site-packages/pandas/core/dtypes/missing.py in array_equivalent(left, right, strict_nan, dtype_equal)
    427     # string dtypes must be come to this path for NumPy 1.7.1 compat
    428     if is_string_dtype(left.dtype) or is_string_dtype(right.dtype):
--> 429         return _array_equivalent_object(left, right, strict_nan)
    430 
    431     # NaNs can occur in float and complex arrays.

~/compose/etc/conda/cuda_11.2/envs/notebooks/lib/python3.7/site-packages/pandas/core/dtypes/missing.py in _array_equivalent_object(left, right, strict_nan)
    482         else:
    483             try:
--> 484                 if np.any(np.asarray(left_value != right_value)):
    485                     return False
    486             except TypeError as err:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Expected Output

True

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 2cb9652
python : 3.7.10.final.0
python-bits : 64
OS : Linux
OS-release : 4.15.0-76-generic
Version : #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : None
LOCALE : en_US.UTF-8

pandas : 1.2.4
numpy : 1.20.2
pytz : 2021.1
dateutil : 2.8.1
pip : 21.1.1
setuptools : 52.0.0.post20210125
Cython : 0.29.23
pytest : 6.2.4
hypothesis : 6.12.0
sphinx : 4.0.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.3.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : 2021.04.0
fastparquet : None
gcsfs : None
matplotlib : 3.3.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.1
pyxlsb : None
s3fs : None
scipy : 1.6.3
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.53.1

[mzeitlin11]

Thanks for the report @isVoid. Could you please elaborate on why you think this is a bug? {'a': np.array(['xx', 'yy'])} is just treated as an arbitrary object, so assert_series_equal ends up trying this

{'a': np.array(['xx', 'yy'])} == {'a': np.array(['xx', 'yy'])}

which raises the error you see independently of any pandas behavior.

[isVoid]

@mzeitlin11 Sorry about the late reply. Thanks for clarification. It makes sense it's about the equality of two arbitrary objects. In this case, instead of considering this as a bug, would it makes sense to provide a custom lambda to determine equality of row values?

E.g.:

pd.testing.assert_series_equal(s, s2, eq_func=lambda x, y: all(x['a'] == y['a']))

[mzeitlin11]

I think that would be out of scope for assert_series_equal. Something like you describe above with applying an arbitrary function item by item sounds better suited as a custom function (that could be defined relatively cleanly with existing apply functionality)

[mroeschke]

Thanks for the report, but agreed for arbitrary comparisons I think this would be no different that writing a custom function to determine equality. Closing since this enhancement request hasn't gotten support from the core team but happy to reopen if there's revived interest.