Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conf: change suite.rc to flow.cylc #3755

Merged
merged 27 commits into from
Aug 18, 2020
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
b116843
Deprecate suite.rc in favour of flow.cylc
MetRonnie Aug 5, 2020
11d3c95
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 5, 2020
690234f
Update syntax highlighters with flow.cylc filetype
MetRonnie Aug 7, 2020
dc158e8
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 5, 2020
9a5260b
On run, re-register previously-run workflows that use deprecated suit…
MetRonnie Aug 5, 2020
a759049
Rename suiterc -> flow_file
MetRonnie Aug 6, 2020
b4e2847
Rename log/suiterc/ dir to log/flow-config/
MetRonnie Aug 11, 2020
c5fdfc2
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 6, 2020
db2faec
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 6, 2020
60401af
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 6, 2020
4fff54b
Change flow.rc -> global.cylc
MetRonnie Aug 7, 2020
16b5230
Change global.rc -> global.cylc
MetRonnie Aug 10, 2020
e5b2c6c
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 10, 2020
adfb8bc
Change flow-tests.rc -> global-tests.cylc
MetRonnie Aug 10, 2020
28a3c08
Merge branch 'master' into config-changes
MetRonnie Aug 11, 2020
87cdd00
Wider rename of suite.rc -> flow.cylc
MetRonnie Aug 10, 2020
3295a64
Change all other *.rc -> *.cylc
MetRonnie Aug 11, 2020
2d9415a
Address code review
MetRonnie Aug 12, 2020
399cc47
Fix `cylc validate .` when using deprecated suite.rc
MetRonnie Aug 12, 2020
ee666f8
Functional test for upgrading suite.rc -> flow.cylc on registration
MetRonnie Aug 13, 2020
e7e3756
Merge branch 'master' into config-changes
MetRonnie Aug 13, 2020
fe497cb
Update changelog
MetRonnie Aug 13, 2020
29c61b4
Wider rename of .rc -> .cylc
MetRonnie Aug 13, 2020
b470c5d
Improve parse_suite_arg
MetRonnie Aug 13, 2020
d1d567c
Merge commit '1895c13bf8852591d0935d935d16e0b0ee363dac' into config-c…
MetRonnie Aug 17, 2020
a757f04
Fix code style
MetRonnie Aug 17, 2020
ee20de9
Address code review
MetRonnie Aug 18, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ __pycache__/

# processed suite definitions
*.rc.processed
*.cylc.processed

# profiling
.profiling
Expand Down
43 changes: 24 additions & 19 deletions cylc/flow/cfgspec/globalcfg.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
# - value_type: value type (compulsory).
# - default: the default value (optional).
# - allowed_2, ...: the only other allowed values of this setting (optional).
with Conf('flow.rc', desc='''
with Conf('global.cylc', desc='''
The global configuration which defines default Cylc Flow settings
for a user or site.

Expand All @@ -43,11 +43,11 @@
$ cylc get-global-config --sparse


Cylc will attempt to load the global configuration (flow.rc) from two
Cylc will attempt to load the global configuration (global.cylc) from two
locations:

* ``/etc/cylc/flow/<CYLC_VERSION>/flow.rc``
* ``~/.cylc/flow/<CYLC_VERSION>/flow.rc``
* ``/etc/cylc/flow/<CYLC_VERSION>/global.cylc``
* ``~/.cylc/flow/<CYLC_VERSION>/global.cylc``

If both files are present files will be loaded in this order so those
lower down the list may override settings from those higher up.
Expand All @@ -56,8 +56,13 @@

.. note::

The ``flow.rc`` file can be templated using Jinja2 variables.
The ``global.cylc`` file can be templated using Jinja2 variables.
See :ref:`Jinja`.

.. note::

In earlier versions of Cylc, ``global.cylc`` was named ``global.rc`` or
``flow.rc``, but those names are no longer supported.
''') as SPEC:

# suite
Expand Down Expand Up @@ -87,21 +92,21 @@

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You will need to rename the configuration file at line 37.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean flow.rc? Would it make sense to split that off into another PR?

Copy link
Member

@oliver-sanders oliver-sanders Aug 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, just read the PR description, I would have thought it would be easier to do both at the same time, but if you want to do it in two PRs that's ok.

# suite
with Conf('cylc', desc='''
Default values for entries in the suite.rc ``[cylc]`` section.
Default values for entries in flow.cylc ``[cylc]`` section.
'''):
Conf('UTC mode', VDR.V_BOOLEAN, False, desc='''
Default for :cylc:conf:`suite.rc[cylc]UTC mode`.
Default for :cylc:conf:`flow.cylc[cylc]UTC mode`.
''')
Conf('task event mail interval', VDR.V_INTERVAL, DurationFloat(300),
desc='''
Default for
:cylc:conf:`suite.rc[cylc]task event mail interval`.
:cylc:conf:`flow.cylc[cylc]task event mail interval`.
''')

with Conf('events', desc='''
You can define site defaults for each of the following options,
details of which can be found under
:cylc:conf:`suite.rc[cylc][events]`.
:cylc:conf:`flow.cylc[cylc][events]`.
'''):
Conf('handlers', VDR.V_STRING_LIST)
Conf('handler events', VDR.V_STRING_LIST)
Expand Down Expand Up @@ -380,7 +385,7 @@
''')
Conf('retrieve job logs', VDR.V_BOOLEAN, desc='''
Global default for
:cylc:conf:`suite.rc[runtime][<namespace>][remote]retrieve job
:cylc:conf:`flow.cylc[runtime][<namespace>][remote]retrieve job
logs`.
''')
Conf('retrieve job logs command', VDR.V_STRING, 'rsync -a',
Expand All @@ -391,22 +396,22 @@
''')
Conf('retrieve job logs max size', VDR.V_STRING, desc='''
Global default for the
:cylc:conf:`suite.rc[runtime][<namespace>][remote]retrieve job
:cylc:conf:`flow.cylc[runtime][<namespace>][remote]retrieve job
logs max size`.
the specified host.
''')
Conf('retrieve job logs retry delays', VDR.V_INTERVAL_LIST,
desc='''
Global default for the
:cylc:conf:`suite.rc[runtime][<namespace>][remote]retrieve job
:cylc:conf:`flow.cylc[runtime][<namespace>][remote]retrieve job
logs retry delays`.
setting for the specified host.
''')
Conf('task event handler retry delays', VDR.V_INTERVAL_LIST,
desc='''
Host specific default for
:cylc:conf:`suite.rc[runtime][<namespace>][events]handler retry
delays`.
:cylc:conf:`flow.cylc[runtime][<namespace>][events]handler
retry delays`.
''')
Conf('tail command template',
VDR.V_STRING, 'tail -n +1 -F %(filename)s', desc='''
Expand Down Expand Up @@ -540,7 +545,7 @@
# task
with Conf('task events', desc='''
Global site/user defaults for
:cylc:conf:`suite.rc[runtime][<namespace>][events]`.
:cylc:conf:`flow.cylc[runtime][<namespace>][events]`.
'''):
Conf('execution timeout', VDR.V_INTERVAL)
Conf('handlers', VDR.V_STRING_LIST)
Expand All @@ -558,7 +563,7 @@
Settings for the automated development tests.

.. note::
The test battery reads ``flow-tests.rc`` instead of the normal
The test battery reads ``global-tests.cylc`` instead of the normal
site/user global config files (from the same locations, however).
'''):
Conf('remote host with shared fs', VDR.V_STRING, desc='''
Expand Down Expand Up @@ -659,7 +664,7 @@
with Conf('authentication', desc='''
Authentication of client programs with suite server programs can be
configured here, and overridden in suites if necessary with
:cylc:conf:`suite.rc[cylc][authentication]`.
:cylc:conf:`flow.cylc[cylc][authentication]`.

The suite-specific passphrase must be installed on a user's account to
authorize full control privileges (see
Expand Down Expand Up @@ -760,7 +765,7 @@ class GlobalConfig(ParsecConfig):

_DEFAULT = None
_HOME = os.getenv('HOME') or get_user_home()
CONF_BASENAME = "flow.rc"
CONF_BASENAME = "global.cylc"
SITE_CONF_DIR = os.path.join(os.sep, 'etc', 'cylc', 'flow', CYLC_VERSION)
USER_CONF_DIR = os.path.join(_HOME, '.cylc', 'flow', CYLC_VERSION)

Expand Down Expand Up @@ -814,7 +819,7 @@ def load(self):
# Abort on bad user file (users can fix it).
LOG.error('bad %s %s', conf_type, fname)
raise
# (OK if no flow.rc is found, just use system defaults).
# (OK if no global.cylc is found, just use system defaults).
self._transform()

def get_host_item(self, item, host=None, owner=None, replace_home=False,
Expand Down
36 changes: 22 additions & 14 deletions cylc/flow/cfgspec/suite.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,20 @@
REC_COMMAND = re.compile(r'(`|\$\()\s*(.*)\s*([`)])$')

with Conf(
'suite.rc',
'flow.cylc',
desc='''
Defines a cylc suite configuration.

Embedded Jinja2 code (see :ref:`Jinja`) must process to a valid
raw suite.rc file. See also :ref:`SuiteRCFile` for a descriptive
overview of suite.rc files, including syntax (:ref:`Syntax`).
raw flow.cylc file. See also :ref:`FlowConfigFile` for a descriptive
overview of flow.cylc files, including syntax (:ref:`Syntax`).

.. note::

In earlier versions of Cylc, this was named ``suite.rc``, but that
name is now deprecated. If you ``cylc run`` a directory that
contains a ``suite.rc`` file instead of a ``flow.cylc`` file, Cylc
will automatically create the latter, symlinked to the former.
'''
) as SPEC:

Expand All @@ -66,7 +73,7 @@
A web URL to suite documentation. If present it can be browsed
with the ``cylc doc`` command. The string template
``%(suite_name)s`` will be replaced with the actual suite name.
See also :cylc:conf:`suite.rc[runtime][<namespace>][meta]URL`.
See also :cylc:conf:`flow.cylc[runtime][<namespace>][meta]URL`.

Example:

Expand Down Expand Up @@ -201,14 +208,14 @@

In date-time cycling, if you do not provide time zone information
for this, it will be assumed to be local time, or in UTC if
:cylc:conf:`suite.rc[cylc]UTC mode` is set, or in the time zone
determined by :cylc:conf`suite.rc[cylc][cycle point time zone]`.
:cylc:conf:`flow.cylc[cylc]UTC mode` is set, or in the time zone
determined by :cylc:conf`flow.cylc[cylc][cycle point time zone]`.

The string ``now`` converts to the current date-time on the suite
host (adjusted to UTC if the suite is in UTC mode but the host is
not) to minute resolution. Minutes (or hours, etc.) may be
ignored depending on the value of
:cylc:conf:`suite.rc[cylc]cycle point format`.
:cylc:conf:`flow.cylc[cylc]cycle point format`.
''')
Conf('final cycle point', VDR.V_STRING, desc='''
Cycling tasks are held once they pass the final cycle point, if
Expand All @@ -218,9 +225,9 @@

In date-time cycling, if you do not provide time zone information
for this, it will be assumed to be local time, or in UTC if
:cylc:conf:`suite.rc[cylc]UTC mode`
:cylc:conf:`flow.cylc[cylc]UTC mode`
is set, or in the time zone determined by
:cylc:conf`suite.rc[cylc][cycle point time zone]`.
:cylc:conf`flow.cylc[cylc][cycle point time zone]`.
''')
Conf('initial cycle point constraints', VDR.V_STRING_LIST, desc='''
in a cycling suite it is possible to restrict the initial cycle
Expand Down Expand Up @@ -675,7 +682,8 @@

The top level share and work directory location can be changed
(e.g. to a large data area) by a global config setting (see
:cylc:conf:`flow.rc[hosts][<hostname glob>]work directory`).
:cylc:conf:`global.cylc[hosts][<hostname glob>]
work directory`).

.. note::

Expand Down Expand Up @@ -732,7 +740,7 @@
``%(task_name)s`` will be replaced with the actual
suite and task names.

See also :cylc:conf:`[meta]URL <suite.rc[meta]URL>`.
See also :cylc:conf:`[meta]URL <flow.cylc[meta]URL>`.

Example:

Expand Down Expand Up @@ -855,7 +863,7 @@
with Conf('events', desc='''
Cylc can call nominated event handlers when certain task
events occur. This section configures specific task event
handlers; see :cylc:conf:`suite.rc[cylc][events]` for
handlers; see :cylc:conf:`flow.cylc[cylc][events]` for
suite event handlers.

Event handlers can be located in the suite ``bin/`` directory,
Expand Down Expand Up @@ -1078,7 +1086,7 @@
script without evaluation or manipulation by cylc, so any
variable assignment expression that is legal in the job
submission shell can be used. White space around the
``=`` is allowed (as far as cylc's suite.rc parser is
``=`` is allowed (as far as cylc's flow.cylc parser is
concerned these are just normal configuration items).

Examples::
Expand All @@ -1089,7 +1097,7 @@
WAZ = ${FOO%.jpg}.png
NEXT_CYCLE = $( cylc cycle-point --offset=PT6H )
ZAZ = "${FOO#bar}"
# ^ quoted to escape the suite.rc comment character
# ^ quoted to escape the flow.cylc comment character
''')

with Conf('directives', desc='''
Expand Down
22 changes: 11 additions & 11 deletions cylc/flow/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -483,9 +483,9 @@ def __init__(

self.cfg['scheduling']['special tasks'][s_type] = result

self.collapsed_families_rc = (
self.collapsed_families_config = (
self.cfg['visualization']['collapsed families'])
for fam in self.collapsed_families_rc:
for fam in self.collapsed_families_config:
if fam not in self.runtime['first-parent descendants']:
raise SuiteConfigError(
'[visualization]collapsed families: '
Expand All @@ -497,7 +497,7 @@ def __init__(
elif is_reload:
self.closed_families = []
else:
self.closed_families = self.collapsed_families_rc
self.closed_families = self.collapsed_families_config
for cfam in self.closed_families:
if cfam not in self.runtime['descendants']:
self.closed_families.remove(cfam)
Expand Down Expand Up @@ -620,7 +620,7 @@ def __init__(

# (Note that we're retaining 'default node attributes' even
# though this could now be achieved by styling the root family,
# because putting default attributes for root in the suite.rc spec
# because putting default attributes for root in the flow.cylc spec
# results in root appearing last in the ordered dict of node
# names, so it overrides the styling for lesser groups and
# nodes, whereas the reverse is needed - fixing this would
Expand Down Expand Up @@ -698,7 +698,7 @@ def __init__(
self.mem_log("config.py: end init config")

def process_initial_cycle_point(self):
"""Validate and set initial cycle point from suiterc.
"""Validate and set initial cycle point from flow.cylc.

Sets:
self.initial_point
Expand Down Expand Up @@ -1448,7 +1448,7 @@ def check_tasks(self):
huge suites (several thousand tasks).
Note:
(a) self.cfg['runtime'][name]
contains the task definition sections of the suite.rc file.
contains the task definition sections of the flow.cylc file.
(b) self.taskdefs[name]
contains tasks that will be used, defined by the graph.
Tasks (a) may be defined but not used (e.g. commented out of the
Expand Down Expand Up @@ -1566,10 +1566,10 @@ def generate_taskdefs(self, orig_expr, left_nodes, right, seq, suicide):
replicate(self.cfg['runtime'][name],
self.cfg['runtime']['root'])
if 'root' not in self.runtime['descendants']:
# (happens when no runtimes are defined in the suite.rc)
# (happens when no runtimes are defined in flow.cylc)
self.runtime['descendants']['root'] = []
if 'root' not in self.runtime['first-parent descendants']:
# (happens when no runtimes are defined in the suite.rc)
# (happens when no runtimes are defined in flow.cylc)
self.runtime['first-parent descendants']['root'] = []
self.runtime['parents'][name] = ['root']
self.runtime['linearized ancestors'][name] = [name, 'root']
Expand Down Expand Up @@ -1743,16 +1743,16 @@ def get_graph_raw(self, start_point_string, stop_point_string,

if self.first_graph:
self.first_graph = False
if not self.collapsed_families_rc and not ungroup_all:
if not self.collapsed_families_config and not ungroup_all:
# initially default to collapsing all families if
# "[visualization]collapsed families" not defined
group_all = True

first_parent_descendants = self.runtime['first-parent descendants']
if group_all:
# Group all family nodes
if self.collapsed_families_rc:
self.closed_families = copy(self.collapsed_families_rc)
if self.collapsed_families_config:
self.closed_families = copy(self.collapsed_families_config)
else:
for fam in first_parent_descendants:
if fam != 'root':
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,17 @@
#------------------------------------------------------------------------------
# Some functional tests need to modify or ignore installed global configuration
# values; they therefore ignore the standard site and user global config files
# and start from a special flow-tests.rc file instead. These should contain the
# minimal essential settings (e.g. for host names) needed to run at your site.
# and start from a special global-tests.cylc file instead. These should contain
# the minimal essential settings (e.g. for host names) needed to run at your
# site.
#
# See documentation in flow.rc.eg
# See documentation in global.cylc.eg
#
#------------------------------------------------------------------------------
## GLOBAL flow-tests.rc FILE LOCATIONS:
## GLOBAL global-tests.cylc FILE LOCATIONS:
#----------------
# SITE:
# /etc/cylc/flow/<CYLC-VERSION>/flow-tests.rc
# /etc/cylc/flow/<CYLC-VERSION>/global-tests.cylc
# USER:
# $HOME/.cylc/flow/<CYLC-VERSION>/flow-tests.rc
# $HOME/.cylc/flow/<CYLC-VERSION>/global-tests.cylc
#------------------------------------------------------------------------------
12 changes: 6 additions & 6 deletions cylc/flow/etc/flow.rc.eg → cylc/flow/etc/global.cylc.eg
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
#------------------------------------------------------------------------------
# How to create a site or user flow.rc global config file.
# How to create a site or user global.cylc global config file.
#------------------------------------------------------------------------------
# The "cylc get-global-config" command prints global config defaults,
# overridden by site global settings (if any), overridden by user global
# settings (if any).
#
# To generate a new global config file:
# % cylc get-global-config > flow.rc
# % cylc get-global-config > global.cylc
# Edit any settings that you need to modify.
# Delete or comment out any seeting you do not need (to avoid inadvertently
# overriding defaults or site settings that may change in the future).
#
# For available configuration items, see the global flow.rc reference in the
# For available configuration items, see the global global.cylc reference in the
# User Guide.
#
# GLOBAL flow.rc FILE LOCATIONS:
# GLOBAL global.cylc FILE LOCATIONS:
#----------------
# SITE:
# /etc/cylc/flow/<CYLC-VERSION>/flow.rc
# /etc/cylc/flow/<CYLC-VERSION>/global.cylc
# USER:
# $HOME/.cylc/flow/<CYLC-VERSION>/flow.rc
# $HOME/.cylc/flow/<CYLC-VERSION>/global.cylc
#------------------------------------------------------------------------------
Loading