Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix for pandas column object bug #3176

Merged
merged 3 commits into from
Oct 2, 2018
Merged

fix for pandas column object bug #3176

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b4aa0fe
turn columns object into pandas array, plus test
corinnebosley Sep 26, 2018
e0f1782
made things the way pandas like it; bambooish
corinnebosley Oct 1, 2018
10305e4
regenerated cml to match output
corinnebosley Oct 1, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion lib/iris/pandas.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,8 @@ def as_cube(pandas_array, copy=True, calendars=None):
_add_iris_coord(cube, "index", pandas_array.index, 0,
calendars.get(0, None))
if pandas_array.ndim == 2:
_add_iris_coord(cube, "columns", pandas_array.columns, 1,
_add_iris_coord(cube, "columns", np.array([k for k in
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be better done as pandas_array.columns.values, which returns you a NumPy array of the column headers in one shot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good shout @dkillick, I'll get on it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm....

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkillick The problem here is that this method turns all the column headers into objects. Now that seems fine to me, but @DavidSimonin seemed to think that they all should have been strings. I'm not convinced that the headers should be strings at all, I think you can probably extract the string from the object, but I might have to dig a little deeper to determine the appropriate behaviour.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dkillick I reckon that should be it. I removed and regenerated the cml file (self-serving as that seems...), and I'm banking on the tests passing and the fix being good now.

pandas_array.columns]), 1,
calendars.get(1, None))
return cube

Expand Down
15 changes: 15 additions & 0 deletions lib/iris/tests/results/pandas/as_cube/data_frame_multidim.cml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
<?xml version="1.0" ?>
<cubes xmlns="urn:x-iris:cubeml-0.2">
<cube dtype="int64" units="unknown">
<coords>
<coord datadims="[1]">
<auxCoord id="d00b57c8" long_name="columns" points="[col_1, col_2, col_3, col_4, col_5]" shape="(5,)" units="Unit('unknown')" value_type="string"/>
</coord>
<coord datadims="[0]">
<dimCoord id="343cbc01" long_name="index" points="[0, 1]" shape="(2,)" units="Unit('unknown')" value_type="int64"/>
</coord>
</coords>
<cellMethods/>
<data checksum="0x5e85e4a4" dtype="int64" mask_checksum="no-masked-elements" shape="(2, 5)"/>
</cube>
</cubes>
11 changes: 11 additions & 0 deletions lib/iris/tests/test_pandas.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,17 @@ def test_data_frame_masked(self):
tests.get_result_path(('pandas', 'as_cube',
'data_frame_masked.cml')))

def test_data_frame_multidim(self):
data_frame = pandas.DataFrame([[0, 1, 2, 3, 4],
[5, 6, 7, 8, 9]],
index=[0, 1],
columns=['col_1', 'col_2', 'col_3',
'col_4', 'col_5'])
self.assertCML(
iris.pandas.as_cube(data_frame),
tests.get_result_path(('pandas', 'as_cube',
'data_frame_multidim.cml')))

def test_data_frame_cftime_360(self):
data_frame = pandas.DataFrame(
[[0, 1, 2, 3, 4],
Expand Down