Skip to content

Commit

Permalink
ARROW-2273: [Python] Raise NotImplementedError when pandas Sparse typ…
Browse files Browse the repository at this point in the history
…es serializing

This fixes [ARROW-2273](https://issues.apache.org/jira/browse/ARROW-2273).

`pandas` Sparse types are planned to be deprecated in pandas future releases (pandas-dev/pandas#19239).
`SparseDataFrame` and `SparseSeries` are naive implementation and have many bugs. IMO, this is not the right time to support these in `pyarrow`.

Author: Licht-T <[email protected]>

Closes #1997 from Licht-T/add-pandas-sparse-unsupported-msg and squashes the following commits:

64e24ce <Licht-T> ENH: Raise NotImplementedError when pandas Sparse types serializing pandas Sparse types are planned to be deprecated in pandas future releases. pandas-dev/pandas#19239
  • Loading branch information
Licht-T authored and xhochy committed May 5, 2018
1 parent 5bdfff8 commit 3d594bc
Showing 1 changed file with 18 additions and 0 deletions.
18 changes: 18 additions & 0 deletions python/pyarrow/serialization.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,31 @@ def _register_custom_pandas_handlers(context):

import pyarrow.pandas_compat as pdcompat

sparse_type_error_msg = (
'{0} serialization is not supported.\n'
'Note that {0} is planned to be deprecated '
'in pandas future releases.\n'
'See https://github.com/pandas-dev/pandas/issues/19239 '
'for more information.'
)

def _serialize_pandas_dataframe(obj):
if isinstance(obj, pd.SparseDataFrame):
raise NotImplementedError(
sparse_type_error_msg.format('SparseDataFrame')
)

return pdcompat.dataframe_to_serialized_dict(obj)

def _deserialize_pandas_dataframe(data):
return pdcompat.serialized_dict_to_dataframe(data)

def _serialize_pandas_series(obj):
if isinstance(obj, pd.SparseSeries):
raise NotImplementedError(
sparse_type_error_msg.format('SparseSeries')
)

return _serialize_pandas_dataframe(pd.DataFrame({obj.name: obj}))

def _deserialize_pandas_series(data):
Expand Down

0 comments on commit 3d594bc

Please sign in to comment.