BUG: comparing 'NaT' in series with series returns True

[jorisvandenbossche]

xref #12877 (test on Series & DataFrame)

When having a 'NaT' in a datetime series and comparing this to a datetime series, the comparison with NaT returns True (and shouldn't comparisons with NA not always return False?) Comparing it with a scalar does return False:

In [54]: s = pd.Series([pd.Timestamp('2012-01-01'), pd.Timestamp('NaT')])
In [55]: s2 = pd.Series([pd.Timestamp('2013-01-01'), pd.Timestamp('2014-01-01')])

In [57]: s
Out[57]:
0   2012-01-01
1          NaT
dtype: datetime64[ns]

In [58]: s2
Out[58]:
0   2013-01-01
1   2014-01-01
dtype: datetime64[ns]

In [59]: s < s2
Out[59]:
0    True
1    True
dtype: bool

In [60]: s < pd.Timestamp('2013-01-01')
Out[60]:
0     True
1    False
dtype: bool

Original from http://stackoverflow.com/questions/27305324/pandas-comparison-with-variable-number-of-columns/27305661#27305661

[jreback]

hmm that looks wrong for series; yep should compare as False

[jorisvandenbossche]

Greater than does return False:

In [68]: s > s2
Out[68]:
0    False
1    False
dtype: bool

[jreback]

yep because its comparing the actual value (and not using `_maybe_mask_results()`` (method) after

[cfperez]

Hey @jreback, this bug just bit me in 0.18.0. Just pinging to put back on the radar.

[jreback]

@cfperez pull-requests are welcome here.

[jreback]

This is in 0.18.0; all of which looks correct, e.g. comparison with NaT all return False

In [1]:  s = pd.Series([pd.Timestamp('2012-01-01'), pd.Timestamp('NaT')])

In [2]:  s2 = pd.Series([pd.Timestamp('2013-01-01'), pd.Timestamp('2014-01-01')])

In [3]: s == s2
Out[3]: 
0    False
1    False
dtype: bool

In [4]: s < s2
Out[4]: 
0     True
1    False
dtype: bool

In [5]: s < pd.Timestamp('2014-01-01')
Out[5]: 
0     True
1    False
dtype: bool

In [6]: s2 < s
Out[6]: 
0    False
1    False
dtype: bool

In [7]: pd.Timestamp('2014-01-01') < s
Out[7]: 
0    False
1    False
dtype: bool

@cfperez are [4] and [5] not returning like the above?

[jreback]

@sinhrks has been fixing lots of these NaT issues. so this looks fixed. just need confirming tests.

[cfperez]

I'll see what I can do! Working on #10668 first.

On Wed, 6 Apr 2016, 9:29 AM Jeff Reback [email protected] wrote:

@cfperez https://github.com/cfperez pull-requests are welcome here.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#9005 (comment)

[jreback]

@cfperez gr8!.