Use of cycle point format

[hjoliver]

In cylc-6, setting the cycle point format affects everything: task ID, task labels in the GUI, log directory naming, the format of $CYLC_TASK_CYCLE_POINT in the task execution environment, etc.

Firstly, @benfitzpatrick - can you confirm that it's not possible for users to specify a cycle point format that violates the ISO 8601 standard (aside from the cylc legacy format that we still support)? If that was possible, tasks could not use cylc cycle-point or similar to do date time calculations.

Secondly, this new format flexibility probably makes it more difficult for third-party file housekeeping utilities and the like - they can no longer assume a fixed format date-time in suite output directory names.

This may in fact be the best way to do it, but I just want to check...

Another approach could be to use the user-specified cycle point format only for display purposes, and stick with the most basic format for everything else (the format string could be passed to task too). Then third-party utilities can assume the form of date-time strings without having to be fully ISO 8601 aware. But maybe this is not practical now that such a massive range of date-times is possible (climate modellers won't want to specify cycle points to the minute...).

Any thoughts on this Ben?

[benfitzpatrick]

Firstly, @benfitzpatrick - can you confirm that it's not possible for users to specify a cycle point format that violates the ISO 8601 standard (aside from the cylc legacy format that we still support)?

That's true at the moment, although technically it could easily be done -
the isodatetime library does support reading in arbitrary date time
formats given a format string (see e.g. these tests).

If cylc cycle-point could take an input format string, there wouldn't be
a barrier (rose date will already do that, although potentially these should
just merge...). Maybe we should export the cycle point format as an
environment variable for this kind of use. Both cylc cycle-point and
rose date will preserve the input cycle point formatting when applying
offsets, anyway.

There is a good point about expanded year digit information... maybe we
need an environment variable for that as well, if it's not just 0.

Third party stuff:

third-party file housekeeping utilities ... can no longer assume a fixed format date-time in suite output directory names

vs

third-party utilities can assume the form of date-time strings without having to be fully ISO 8601 aware.

I think third party housekeeping tasks should always make use of
cylc cycle-point or rose date to manipulate or parse date-times. Making
a fixed-format assumption is, I guess, in order to hardcode the parsing by
using magic numbers with e.g. sed or cut, which is a hacky way to do
things. This is especially true when you're a housekeeping task whose
primary purpose is to do some date-time calculation and then fire off some
rms. I think it's better to say that this flexibility exposes a bad assumption
rather than say that it makes it more difficult.

Even the date command will read (some) ISO 8601, in more recent
versions of GNU coreutils. That wouldn't be a bad way to do it.

[matthewrmshin]

See also metomi/isodatetime#49.

[hjoliver]

[meeting] - too difficult; third party tools will just have to deal with it.