Fix offset inits, apply_index dtypes #19142

jbrockmendel · 2018-01-08T22:22:25Z

This PR does two main things:

Change the default kwargs for offset __init__ methods so that the defaults are valid; ATM WeekOfMonth() and LastWeekOfMonth() will both raise ValueError.
ATM Week.apply_index adds an int64 array to a DatetimeIndex. This happens to have the desired output because the int64 array represents nanoseconds. This explicitly casts the array to timedelta64[ns] to make this explicit. This is a necessary step towards fixing DatetimeIndex + ndarray[int] wrong, reverse op errors #19123.

In addition there are two small pieces of refactoring:

Remove liboffsets.EndMixin since it is only mixed in to Week.
Use liboffsets.roll_convention to de-duplicate some code in liboffsets.shift_months.

closes #xxxx
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

…days add timedelta64[ns] array instead of int64 array

jbrockmendel · 2018-01-09T02:20:14Z

Travis looks like a moto problem.

jreback · 2018-01-09T12:33:48Z

pandas/tseries/offsets.py

@@ -1226,7 +1225,8 @@ def _get_roll(self, i, before_day_of_month, after_day_of_month):
        return roll

    def _apply_index_days(self, i, roll):
-        i += (roll % 2) * Timedelta(days=self.day_of_month).value
+        nanos = (roll % 2) * Timedelta(days=self.day_of_month).value


I would rather

n = Timedelta((roll % 2) * Timedelta(days=self.day_of_month).value) return i + n + Timedelta(days-1)

roll is an array

then pls add a doc-string (same below)

jreback · 2018-01-09T12:34:01Z

pandas/tseries/offsets.py

@@ -1271,13 +1271,14 @@ def _get_roll(self, i, before_day_of_month, after_day_of_month):
        return roll

    def _apply_index_days(self, i, roll):
-        return i + (roll % 2) * Timedelta(days=self.day_of_month - 1).value
+        nanos = (roll % 2) * Timedelta(days=self.day_of_month - 1).value


jreback · 2018-01-09T12:35:09Z

pandas/tseries/offsets.py

@@ -1394,7 +1411,7 @@ class WeekOfMonth(_WeekOfMonthMixin, DateOffset):
    _prefix = 'WOM'
    _adjust_dst = True

-    def __init__(self, n=1, normalize=False, week=None, weekday=None):


this doesn't break anything???

The default kwargs raise ATM.

where is the test for this? why isn't it raising currently?

…fsets-intarray

codecov · 2018-01-10T00:50:46Z

Codecov Report

❗ No coverage uploaded for pull request base (master@27a5039). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master   #19142   +/-   ##
=========================================
  Coverage          ?   91.52%           
=========================================
  Files             ?      147           
  Lines             ?    48785           
  Branches          ?        0           
=========================================
  Hits              ?    44651           
  Misses            ?     4134           
  Partials          ?        0

Flag	Coverage Δ
#multiple	`89.89% <100%> (?)`
#single	`41.61% <26.31%> (?)`

Impacted Files	Coverage Δ
pandas/tseries/offsets.py	`97.09% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27a5039...4ea4e1e. Read the comment docs.

…fsets-intarray

jreback · 2018-01-11T00:12:10Z

pandas/tseries/offsets.py

@@ -1226,7 +1225,8 @@ def _get_roll(self, i, before_day_of_month, after_day_of_month):
        return roll

    def _apply_index_days(self, i, roll):
-        i += (roll % 2) * Timedelta(days=self.day_of_month).value
+        nanos = (roll % 2) * Timedelta(days=self.day_of_month).value


then pls add a doc-string (same below)

jreback · 2018-01-11T00:13:01Z

pandas/tseries/offsets.py

@@ -1394,7 +1411,7 @@ class WeekOfMonth(_WeekOfMonthMixin, DateOffset):
    _prefix = 'WOM'
    _adjust_dst = True

-    def __init__(self, n=1, normalize=False, week=None, weekday=None):


where is the test for this? why isn't it raising currently?

jreback · 2018-01-11T00:13:06Z

pandas/tseries/offsets.py

+            return self._end_apply_index(i)
+
+    def _end_apply_index(self, dtindex):
+        """Offsets index to end of Period frequency"""


jbrockmendel · 2018-01-11T00:23:24Z

where is the test for this? why isn't it raising currently?

Until a few dozen PRs ago all that was passed to __init__ was n=1, normalize=False, **kwargs. The tests had to explicitly pass the relevant kwargs or else you'd get a KeyError. So there aren't any tests that use the defaults.

jreback · 2018-01-11T00:38:31Z

Until a few dozen PRs ago all that was passed to init was n=1, normalize=False, **kwargs. The tests had to explicitly pass the relevant kwargs or else you'd get a KeyError. So there aren't any tests that use the defaults.

ok can you add some pls.

…fsets-intarray

jreback · 2018-01-11T11:26:24Z

since pd.offsets.WeekOfMonth() now works, pls add a note in bug fixes for whatsnew. otherwise lgtm. ping on green.

…fsets-intarray

jbrockmendel · 2018-01-11T21:48:28Z

ping

jreback · 2018-01-12T00:47:25Z

doc/source/whatsnew/v0.23.0.txt

@@ -474,3 +474,4 @@ Other

 - Improved error message when attempting to use a Python keyword as an identifier in a ``numexpr`` backed query (:issue:`18221`)
 - :func:`Timestamp.replace` will now handle Daylight Savings transitions gracefully (:issue:`18319`)
+- Bug in :class:`WeekOfMonth` and :class:`LastWeekOfMonth` where default keyword arguments for constructor raised ``ValueError`` (:issue:`19142`)


pls move this and the one above to COnversion, don't put things Other unless they obviously don't fit elsewhere. You could make a TimesSeries section if you want (and move appropriate things), but move this for now.

OK. For future reference, what heuristic should I be using that maps WeekOfMonth.__init__ ValueError to "conversion"?

…fsets-intarray

jreback · 2018-01-12T11:45:23Z

thanks!

jbrockmendel added 4 commits January 8, 2018 13:55

remove EndMixin since it is only used in one place; make apply_index_…

891374a

…days add timedelta64[ns] array instead of int64 array

make default kwargs valid

1b7d11a

use roll_convention where possible

cd08026

make roll_convention nogil

38415c4

jreback requested changes Jan 9, 2018

View reviewed changes

jreback added Frequency DateOffsets Clean labels Jan 9, 2018

jbrockmendel added 2 commits January 9, 2018 09:02

Merge branch 'master' of https://github.com/pandas-dev/pandas into of…

0cf28ba

…fsets-intarray

Merge branch 'master' of https://github.com/pandas-dev/pandas into of…

c851879

…fsets-intarray

jbrockmendel added 2 commits January 10, 2018 10:13

Merge branch 'master' of https://github.com/pandas-dev/pandas into of…

8bcf15c

…fsets-intarray

fix unhelpful pytest info, arg for to_period

11aa109

jreback requested changes Jan 11, 2018

View reviewed changes

jbrockmendel added 4 commits January 10, 2018 18:00

docstrings

39e0090

docstring

baa07b1

smoke test that default kwargs are valid

4a1c28c

Merge branch 'master' of https://github.com/pandas-dev/pandas into of…

2a516e2

…fsets-intarray

jreback added this to the 0.23.0 milestone Jan 11, 2018

jbrockmendel added 2 commits January 11, 2018 10:35

whatsnew note

6658679

Merge branch 'master' of https://github.com/pandas-dev/pandas into of…

162504b

…fsets-intarray

jreback requested changes Jan 12, 2018

View reviewed changes

jbrockmendel and others added 3 commits January 11, 2018 17:18

move whatsnew to conversion

39b66b5

Merge branch 'master' of https://github.com/pandas-dev/pandas into of…

dbd40ef

…fsets-intarray

Merge branch 'master' into PR_TOOL_MERGE_PR_19142

4ea4e1e

jreback approved these changes Jan 12, 2018

View reviewed changes

jreback merged commit 8912efc into pandas-dev:master Jan 12, 2018

jbrockmendel deleted the offsets-intarray branch February 11, 2018 22:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix offset inits, apply_index dtypes #19142

Fix offset inits, apply_index dtypes #19142

jbrockmendel commented Jan 8, 2018

jbrockmendel commented Jan 9, 2018

jreback Jan 9, 2018

jbrockmendel Jan 9, 2018 •

edited

Loading

jreback Jan 11, 2018

jreback Jan 9, 2018

jreback Jan 9, 2018

jbrockmendel Jan 9, 2018 •

edited

Loading

jreback Jan 11, 2018

codecov bot commented Jan 10, 2018 •

edited

Loading

jreback Jan 11, 2018

jreback Jan 11, 2018

jreback Jan 11, 2018

jbrockmendel Jan 11, 2018

jbrockmendel commented Jan 11, 2018

jreback commented Jan 11, 2018

jreback commented Jan 11, 2018

jbrockmendel commented Jan 11, 2018

jreback Jan 12, 2018

jbrockmendel Jan 12, 2018

jreback commented Jan 12, 2018

Fix offset __inits__, apply_index dtypes #19142

Fix offset __inits__, apply_index dtypes #19142

Conversation

jbrockmendel commented Jan 8, 2018

jbrockmendel commented Jan 9, 2018

Choose a reason for hiding this comment

jbrockmendel Jan 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbrockmendel Jan 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jan 10, 2018 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbrockmendel commented Jan 11, 2018

jreback commented Jan 11, 2018

jreback commented Jan 11, 2018

jbrockmendel commented Jan 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jan 12, 2018

Fix offset inits, apply_index dtypes #19142

Fix offset inits, apply_index dtypes #19142

jbrockmendel Jan 9, 2018 •

edited

Loading

jbrockmendel Jan 9, 2018 •

edited

Loading

codecov bot commented Jan 10, 2018 •

edited

Loading