Allow suites to override the run directory location?

[hjoliver]

I have one user who wants to be able to put all suite and job logs, and share and work directories etc. inside his suite definition directories. This used to be possible pre site/user config file when all these directories were individually configurable in runtime namespaces. Personally I'm not fond of the idea, but I'm posting this to flag it for consideration... It could be achieved in several ways, e.g.:

• a "run directory" item in the suite.rc which if used would override the site/user item.
• a single "put suite output in suite definition directory" option in the site/user file.
• suite-specific variables in site/user "run directory" (requiring the suite definition to be parsed before site/user settings)

Implications for other task hosts would have to be thought through. And: non-suite access to the suite would require parsing the suite definition again.

Low priority at best as users can use symbolic links in the suite definition directory to get essentially the same effect now.

[hjoliver]

The more I think about this the (even) less it seems worth doing. Having to parse a suite definition to find the output location is not good. For example, dependence between suites: if the upstream suite is being edited it may not parse, thus breaking the downstream suite even though the upstream suite is still running fine.

One way around this would be to parse the suite definition at registration time, and to store certain parameters like output locations along with the suite name registration. But this is not ideal, the registration db could get out sync with the suite if not updated quickly after suite changes. There is no server process watching for changes in registered suites, but I suppose we could have the registration db access code do a time stamp comparison every time a suite is accessed by a cylc command, and reparse if necessary to update itself (but again, same problem if the suite is currently broken due to editing in progress).

[m214089]

Hi Hilary,
as we do not have operational setups, we hope to get properly encapsulated 'working' units.

Means something like (just a sketch):

/etc
/bin
/lib
/share/restart
/share/output
/var

where the suite.def can be as well in etc/. The suite itself does not need to know much of this. That it does for the time being is just the try to transfer the existing setup one-to-one, which isn't a good idea anyway.
Hope that helps in your decision. I think you are right and the stuff should be adapted. There is only one thing:
it would be good to be able to determine a 'task run directory' so that a job can have a separate preparation/cleanup step.
Cheerio,
Luis

[hjoliver]

@m214089 - question sent via email regarding your comment above.

[hjoliver]

Closing this. No one can think of a good reason to not use the standard locations, especially after #1885, when data directories can be redirected with symlinks as required.