Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow adding a day to streams' file name date specifier #224

Merged
merged 3 commits into from
Apr 17, 2023
Merged

Allow adding a day to streams' file name date specifier #224

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f8afd75
Allow adding a day to streams' file name date specifier
billsacks Apr 12, 2023
4b784dc
Add doctests of _add_day function
billsacks Apr 12, 2023
d4d0809
Rename filename_add_days to filename_advance_days
billsacks Apr 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 51 additions & 3 deletions cime_config/stream_cdeps.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,14 @@ def create_stream_xml(self, stream_names, case, streams_xml_file, data_list_file
stream_year_last = int(value)
year_first = max(stream_year_first, data_year_first)
year_last = min(stream_year_last, data_year_last)
stream_datafiles = self._sub_paths(stream_datafiles, year_first, year_last)
if 'filename_advance_days' in child.xml_element.attrib:
filename_advance_days = int(child.xml_element.get('filename_advance_days'))
else:
filename_advance_days = 0
stream_datafiles = self._sub_paths(stream_name,
stream_datafiles,
year_first, year_last,
filename_advance_days)
stream_datafiles = stream_datafiles.strip()
#endif
if stream_vars[node_name]:
Expand Down Expand Up @@ -427,7 +434,33 @@ def _days_in_month(month, year=1):
next_month_start = datetime.date(next_year, next_month, 1)
return (next_month_start - month_start).days

def _sub_paths(self, filenames, year_start, year_end):
@classmethod
def _add_day(cls, year, month, day):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of notes about this function:

(1) I considered using python's datetime module to do the addition of a day, but it seems like this would assume the use of a Gregorian (leap year) calendar, which we don't want here. Working around that felt like it would be at least as complicated as just writing our own function.

(2) In principle this could be extended to add N days instead of just one. But I'm having trouble envisioning a scenario where we'd want to add more than one day, so for now decided to keep it simple, following the YAGNI (You Aren't Gonna Need It) principle. We can always extend it later.

"""Given a year, month and day, add 1 day

Returns a new tuple, (adjusted_year, adjusted_month, adjusted_day)

Assumes a no-leap calendar

>>> StreamCDEPS._add_day(1999, 1, 1)
(1999, 1, 2)
>>> StreamCDEPS._add_day(1999, 1, 31)
(1999, 2, 1)
>>> StreamCDEPS._add_day(1999, 12, 31)
(2000, 1, 1)
Comment on lines +445 to +450
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doctests here can be run manually with:

PYTHONPATH=$PYTHONPATH:../../../cime/CIME/Tools python -m doctest stream_cdeps.py

"""
adjusted_year = year
adjusted_month = month
adjusted_day = day + 1
if adjusted_day > cls._days_in_month(month):
adjusted_day = 1
adjusted_month = adjusted_month + 1
if adjusted_month > 12:
adjusted_month = 1
adjusted_year = adjusted_year + 1
return (adjusted_year, adjusted_month, adjusted_day)

def _sub_paths(self, stream_name, filenames, year_start, year_end, filename_advance_days):
"""Substitute indicators with given values in a list of filenames.

Replace any instance of the following substring indicators with the
Expand All @@ -447,13 +480,24 @@ def _sub_paths(self, filenames, year_start, year_end):
also that we use a no-leap calendar, i.e. every month has the same
number of days every year.

filename_advance_days is an integer specifying the number of days to add to the date
portion of the file name when using %ymd. Currently only values of 0 or 1 are
supported. This is typically 0 but can be 1 to indicate that the dates have a
one-day offset, starting with day 2 in the first year and ending with day 1 in the
year following the last year. This can be the case, for example, for daily coupler
history files.

The difference between this function and `_sub_fields` is that this
function is intended to be used for file names (especially from the
`strm_datfil` defaults), whereas `_sub_fields` is intended for use on
variable names.

Returns a string (filenames separated by newlines).
"""
expect(filename_advance_days == 0 or filename_advance_days == 1,
"Bad filename_advance_days attribute ({}) for {}: must be 0 or 1".format(
filename_advance_days, stream_name))

lines = [line for line in filenames.split("\n") if line]
new_lines = []
for line in lines:
Expand All @@ -470,7 +514,11 @@ def _sub_paths(self, filenames, year_start, year_end):
for month in range(1, 13):
days = self._days_in_month(month)
for day in range(1, days+1):
date_string = (year_format + "-{:02d}-{:02d}").format(year, month, day)
if filename_advance_days == 1:
(adjusted_year, adjusted_month, adjusted_day) = self._add_day(year, month, day)
else:
(adjusted_year, adjusted_month, adjusted_day) = (year, month, day)
date_string = (year_format + "-{:02d}-{:02d}").format(adjusted_year, adjusted_month, adjusted_day)
new_line = line.replace(match.group(0), date_string)
new_lines.append(new_line)
elif match.group('month'):
Expand Down
17 changes: 17 additions & 0 deletions cime_config/stream_definition_v2.0.xsd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,22 @@
<xs:attribute name="compset" type="xs:string"/>
<xs:attribute name="version" type="xs:string"/>

<!-- For file names with the %ymd specifier, this attribute can be set
to 1 to indicate that the dates have a one-day offset, starting
with day 2 in the first year and ending with day 1 in the year
following the last year. This can be the case, for example, for
daily coupler history files. (The absence of this attribute is
equivalent to a value of 0. Currently only values of 0 and 1 are
supported, though we may extend that in the future if needed.)
-->
<xs:attribute name="filename_advance_days">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:pattern value="0|1"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>

<!-- ======================== -->
<!-- simple elements -->
<!-- ======================== -->
Expand Down Expand Up @@ -166,6 +182,7 @@
<xs:attribute ref="first_year"/>
<xs:attribute ref="last_year"/>
<xs:attribute ref="model_grid"/>
<xs:attribute ref="filename_advance_days"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
Expand Down