[migration] Fix migration 3dda56f1c

[john-bodley]

This PR fixes a number of issues with the 3dda56f1c migration which we experienced whilst trying to upgrade our production database. Part of the issues are due to the lack of validation/sanitizing in the form-data and thus erroneous fields may be present.

Specifically this PR:

• Adds the missing time grains. Note it's unfortunate that we have both week_start_ and week_starting_. Ideally there should only be week_starting_ which is consistent with week_ending_.
• Fixes the granularity field lookup. Due to erroneous fields we've witnesses form-data with both granularity (Druid) and time_grain_sqla (SQLA) defined (possibly due to switching of datasources). Rather than using the first truthy value it uses the the appropriate field associated with the datasource type.
• Fixes the period ratio type logic. The period_ratio_type field could be None and thus params.get('period_ratio_type', 'growth') would return None which would result in an exception when trying to call lower(). The fix is simply params.get('period_ratio_type') or 'growth' to ensure that growth is the fallback value.

to: @betodealmeida @graceguo-supercat @mistercrunch @Michelle-Thomas @timifasubaa @williaster

[codecov-io]

Codecov Report

Merging #5471 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #5471   +/-   ##
=======================================
  Coverage   59.11%   59.11%           
=======================================
  Files         372      372           
  Lines       23751    23751           
  Branches     2758     2758           
=======================================
  Hits        14041    14041           
  Misses       9695     9695           
  Partials       15       15

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bea0a0a...ae5ad56. Read the comment docs.

[betodealmeida]

@john-bodley, if I understand correctly you're removing isodate because it's a dependency of another package? But what if we remove that other package in the future? The migration script would still require isodate. What's the benefit of removing it?

[betodealmeida]

@john-bodley, I think the db_engine_specs_map is wrong.

[betodealmeida]

This should be the opposite, since it maps the old format to the new one:

'week_ending_saturday': 'P1W/1970-01-03T00:00:00Z',
'week_start_sunday': '1969-12-28T00:00:00Z/P1W',
'week_start_monday': '1969-12-29T00:00:00Z/P1W',
'week_starting_sunday': '1969-12-28T00:00:00Z/P1W',

[john-bodley]

@betodealmeida the isodate package wasn't added in your PR but rather a follow up PR. I can retract that portion of the change if required.