Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: DecimalArray and JSONArray that are empty return incorrect results for isna() #21189

Closed
Dr-Irv opened this issue May 24, 2018 · 0 comments · Fixed by #21190
Closed

BUG: DecimalArray and JSONArray that are empty return incorrect results for isna() #21189

Dr-Irv opened this issue May 24, 2018 · 0 comments · Fixed by #21190
Labels
Bug ExtensionArray Extending pandas with custom dtypes or arrays.
Milestone

Comments

@Dr-Irv
Copy link
Contributor

Dr-Irv commented May 24, 2018

Code Sample, a copy-pastable example if possible

In [1]: import pandas as pd

In [2]: from pandas.tests.extension.decimal.array import DecimalArray

In [3]: from pandas.tests.extension.json.array import JSONArray

In [4]: DecimalArray([]).isna()
Out[4]: array([], dtype=float64)

In [5]: JSONArray([]).isna()
Out[5]: array([], dtype=float64)

In [6]: import decimal

In [7]: DecimalArray([decimal.Decimal(1.0)]).isna()
Out[7]: array([False])

In [8]: DecimalArray([decimal.Decimal(1.0)]).isna().dtype
Out[8]: dtype('bool')

Problem description

This relates to a discussion with @TomAugspurger in #21183 when I made (since redacted) changes to pandas/util/testing.py in assert_extension_array_equal . It turns out that the implementations of isna() in DecimalArray and JSONArray return an ndarray of the wrong dtype if the arrays are empty.

So we need to force the dtype of the result of isna() to be bool in those implementations

Expected Output

In [4]: DecimalArray([]).isna()
Out[4]: array([], dtype=bool)

In [5]: JSONArray([]).isna()
Out[5]: array([], dtype=bool)

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.0
pytest: 3.3.2
pip: 9.0.1
setuptools: 38.4.0
Cython: 0.27.3
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.6.6
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: None
matplotlib: 2.1.2
openpyxl: 2.4.10
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.1
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: 1.2.1
pymysql: 0.7.11.None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug ExtensionArray Extending pandas with custom dtypes or arrays.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants