feat: is_nan

[MarcoGorelli]

Looks like this would be useful to Tubular: azukds/tubular#349 (comment)

There would need to be a caveat in the docstring about how pandas doesn't distinguish between nan and null but other libraries typically do

This requires some attention to check that we get such nan vs null details correct:

• for Polars, there is already is_nan
• for PyArrow, there is https://arrow.apache.org/docs/python/generated/pyarrow.compute.is_nan.html
• for pandas, we need to be careful. We don't want is_nan to pick up NaT / None / anything else which is used for missing values, but instead, specifically, we only want nan. So, we should check
  • that the dtype is Float
  • that s != s

[camriddell]

For a specific nan do you want to disambiguate between Float64 & float64 in pandas? Where the latter would have the traditional nan, but the former would coerce all present nan to NA?

from pandas import NA, Series
from numpy import nan

s = Series([0, NA, nan, 3], dtype='object')

print(
    s,                                     # both (object dtype)
    s.astype('Float64'),                   # pd.NA
    s.astype('Int64'),                     # pd.NA
    s.astype('Float64').astype('float64'), # np.nan
    sep=f'\n{"-" * 40}\n'
)

[MarcoGorelli]

Float64 only coerces to NA on IO, but nan can still arise:

In [4]: s = pd.Series([1, 0, None], dtype='Float64')

In [5]: s / s
Out[5]:
0     1.0
1     NaN
2    <NA>
dtype: Float64

So here the expected output would be [False, True, NA]

[camriddell]

Last question- any reason to prefer value != value over using numpy.isnan for this operation?