.astype() and Decimal

[shadchin]

Begin from 0.24.0 next code raise exception

import decimal
import pandas as pd
ser = pd.Series([1, 2])
ser.astype(decimal.Decimal)

Traceback

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/shadchin/t_d/venv/local/lib/python2.7/site-packages/pandas/core/generic.py", line 5691, in astype
    **kwargs)
  File "/home/shadchin/t_d/venv/local/lib/python2.7/site-packages/pandas/core/internals/managers.py", line 531, in astype
    return self.apply('astype', dtype=dtype, **kwargs)
  File "/home/shadchin/t_d/venv/local/lib/python2.7/site-packages/pandas/core/internals/managers.py", line 395, in apply
    applied = getattr(b, f)(**kwargs)
  File "/home/shadchin/t_d/venv/local/lib/python2.7/site-packages/pandas/core/internals/blocks.py", line 534, in astype
    **kwargs)
  File "/home/shadchin/t_d/venv/local/lib/python2.7/site-packages/pandas/core/internals/blocks.py", line 595, in _astype
    dtype = pandas_dtype(dtype)
  File "/home/shadchin/t_d/venv/local/lib/python2.7/site-packages/pandas/core/dtypes/common.py", line 2029, in pandas_dtype
    raise TypeError("dtype '{}' not understood".format(dtype))
TypeError: dtype '<class 'decimal.Decimal'>' not understood

Problem description

This begin from #21160

After was

pandas/pandas/core/dtypes/common.py

Line 2013 in 0409521

npdtype = np.dtype(dtype)

Now

pandas/pandas/core/internals/blocks.py

Line 595 in cb00deb

dtype = pandas_dtype(dtype)

pandas_dtype() does not understand decimal.Decimal

Is it expected behavior or regress?

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.101-26
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.24.2
pytest: None
pip: 19.1.1
setuptools: 41.0.1
Cython: None
numpy: 1.16.4
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.8.0
pytz: 2019.1
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[WillAyd]

There is no first class support in pandas for Decimal objects. You would need to instantiate the Decimal objects within your container and use an an object dtype

[WillAyd]

There is however a DecimalArray extension type in our test suite. @TomAugspurger is there a plan to move this into the core at some point?

[TomAugspurger]

Nope. Arrow has a decimal type, that would be the recommended way to store them.

…

On Jun 8, 2019, at 06:36, William Ayd ***@***.***> wrote: There is however a DecimalArray extension type in our test suite. @TomAugspurger is there a plan to move this into the core at some point? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[sid-kap]

@TomAugspurger Is storing decimals via the Arrow backend possible today with Pandas? Or do you mean that we will be able to do this once we hit Pandas 1.0 and Pandas is using Arrow as a backend?

[TomAugspurger]

No it's not possible today, at least not without writing your own extension type.

Pandas 1.0 won't be using Arrow. That will come after.

[avandenberghe]

is this in your pipeline / realised now ?

[gosuto-inzasheru]

curious about this :)

[colrobloxkid69420]

There is no first class support in pandas for Decimal objects. You would need to instantiate the Decimal objects within your container and use an an object dtype

Could you perhaps give me an example? As a python/pandas beginner and an "Intermediate" english speaker, I couldn't understand what "instantiate the Decimal objects within your container and use an an object dtype" means. Sorry for my lack of understanding. Thanks in advance.

[qgbcs]

pandas 中没有对 Decimal 对象的一流支持。您需要在容器中实例化 Decimal 对象并使用对象 dtype

你能给我举个例子吗？作为 python/pandas 初学者和“中级”英语使用者，我无法理解“在容器中实例化 Decimal 对象并使用对象 dtype”是什么意思。抱歉我的理解不够。提前致谢。

same