Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: fix Series(extension array) + extension array values addition #22479

Merged
merged 6 commits into from
Oct 3, 2018
Merged

BUG: fix Series(extension array) + extension array values addition #22479

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ca8f101
BUG: series of extension array add extension array values cause runti…
peterpanmj Aug 23, 2018
9736f38
clear some style error
peterpanmj Sep 18, 2018
4f7ada3
update whatsnew and add issue number
peterpanmj Sep 19, 2018
215d7f7
Update v0.24.0.txt
TomAugspurger Sep 30, 2018
31d8f02
Merge remote-tracking branch 'upstream/master' into peterpanmj-ops_ne…
TomAugspurger Oct 3, 2018
7efeed8
Move test to base, implement
TomAugspurger Oct 3, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pandas/core/ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1218,7 +1218,7 @@ def dispatch_to_extension_op(op, left, right):
new_right = [new_right]
new_right = list(new_right)
elif is_extension_array_dtype(right) and type(left) != type(right):
new_right = list(new_right)
new_right = list(right)
else:
new_right = right

Expand Down
6 changes: 6 additions & 0 deletions pandas/tests/extension/base/ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ def test_divmod(self, data):
self._check_divmod_op(s, divmod, 1, exc=TypeError)
self._check_divmod_op(1, ops.rdivmod, s, exc=TypeError)

def test_add_series_with_extension_array(self, data):
s = pd.Series(data)
result = s + data
expected = pd.Series(data + data)
self.assert_series_equal(result, expected)

def test_error(self, data, all_arithmetic_operators):
# invalid ops
op_name = all_arithmetic_operators
Expand Down
5 changes: 5 additions & 0 deletions pandas/tests/extension/json/test_json.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,11 @@ class TestArithmeticOps(BaseJSON, base.BaseArithmeticOpsTests):
def test_error(self, data, all_arithmetic_operators):
pass

def test_add_series_with_extension_array(self, data):
ser = pd.Series(data)
with tm.assert_raises_regex(TypeError, "unsupported"):
ser + data


class TestComparisonOps(BaseJSON, base.BaseComparisonOpsTests):
pass
6 changes: 6 additions & 0 deletions pandas/tests/extension/test_categorical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
from pandas.api.types import CategoricalDtype
from pandas import Categorical
from pandas.tests.extension import base
import pandas.util.testing as tm


def make_data():
Expand Down Expand Up @@ -202,6 +203,11 @@ def test_arith_series_with_scalar(self, data, all_arithmetic_operators):
else:
pytest.skip('rmod never called when string is first argument')

def test_add_series_with_extension_array(self, data):
ser = pd.Series(data)
with tm.assert_raises_regex(TypeError, "cannot perform"):
ser + data


class TestComparisonOps(base.BaseComparisonOpsTests):

Expand Down
6 changes: 6 additions & 0 deletions pandas/tests/extension/test_integer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,12 @@ def test_error(self, data, all_arithmetic_operators):
# other specific errors tested in the integer array specific tests
pass

@pytest.mark.xfail(reason="EA is listified. GH-22922", strict=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, here's what's happening for IntegerArray with missing values,

Because of #22922, the ExtensionArray is listified, so we do IntegerArray + List. Then, in IntegerArray.__add__ we do np.asarray(other). Since we have missing values, we naturally end up with a float ndarray.

Fixing #22922 will fix this specific issue, but doesn't handle the general problem of IntegerArray + List. i.e. is this a bug?

In [3]: arr = pd.Series([1, None, 3], dtype='Int8').values

In [4]: arr
Out[4]: IntegerArray([1, nan, 3], dtype='Int8')

In [5]: arr + list(arr)
Out[5]: array([ 2., nan,  6.])

Should Out[5] be an IntegerArray([2, nan, 6], dtype='Int8')? If so, I'll open a new issue.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes i think you are right

not sure this is hitting the right path

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#22962

def test_add_series_with_extension_array(self, data):
super(TestArithmeticOps, self).test_add_series_with_extension_array(
data
)


class TestComparisonOps(base.BaseComparisonOpsTests):

Expand Down