diff --git a/src/7-to-8/major-changes/scheduling.rst b/src/7-to-8/major-changes/scheduling.rst index 1da463951d..b597f37abd 100644 --- a/src/7-to-8/major-changes/scheduling.rst +++ b/src/7-to-8/major-changes/scheduling.rst @@ -23,10 +23,12 @@ Cylc can manage infinite workflows of repeating tasks: Cylc 8 has a new scheduling algorithm that: -- Is much more efficient because it only has to manage active tasks +- Is much more efficient because it doesn't need to track nearly as + many waiting and succeeded tasks in addition to the active + (submitted and running) ones. - waiting tasks are not pre-spawned before they are needed - - succeeded tasks are not kept across the active task window + - succeeded tasks are not retained across the :term:`active window` - no costly indiscriminate dependency matching is done - Distinguishes between :term:`optional ` and :term:`required ` task outputs, to support: @@ -37,7 +39,7 @@ Cylc 8 has a new scheduling algorithm that: - instances of same task can run out of cycle point order - the workflow will not unnecessarily stall downstream of failed tasks -- Provides a sensible active-task based window on the evolving workflow +- Provides a sensible activity-based window on the evolving workflow - (to fully understand which tasks appeared in the Cylc 7 GUI you had to understand the scheduling algorithm) diff --git a/src/glossary.rst b/src/glossary.rst index 2821d8c6b6..bee907262e 100644 --- a/src/glossary.rst +++ b/src/glossary.rst @@ -67,50 +67,39 @@ Glossary Submit number also appears in the job log path so that job log files don't get overwritten. - - active active task - An active task is a task which is near ready to run, in the process of - running, or which requires user intervention. These are all the tasks - being actively managed by the scheduler at this point in the run. + Tasks that are in the ``submitted`` and ``running`` states, i.e., those + with active jobs. Be aware that the :term:`active window` of the workflow + may also contain some tasks that are not active in this sense. - Active tasks are: + window + active window + The active window of a workflow contains only the tasks that Cylc needs + to actively manage in order to perform its job of scheduling efficiently: - - Tasks which have some, but not all of their - :term:`prerequisites ` satisfied. - - ``waiting`` tasks, that are actively waiting on: + - :term:`active tasks ` (``submitted`` and ``running``, with active jobs) + - ``preparing`` tasks - i.e. tasks in the process of submitting jobs + - ``waiting`` tasks that are nearly ready to run but: - - :term:`xtriggers `. - - :ref:`internal queues ` - - :ref:`runahead limit ` + - still have some (but not all) unsatisfied :term:`prerequisites ` + - are waiting on :term:`xtriggers `, + :ref:`internal queues `, or the :ref:`runahead limit ` - - ``preparing`` tasks - i.e. tasks in the process of submitting jobs - - ``submitted`` and ``running`` tasks - i.e. those with active jobs - tasks that reached a :term:`final status` without completing their :term:`required outputs ` (e.g. a task failed where success was required). - Active tasks are in the ``n=0`` :term:`window ` which means they - will always be displayed in the GUI and Tui. - - The distinction between active and non-active tasks is important for - the computing of the :term:`runahead limit`. - + n-window + The GUI provides a view extending ``n`` (default ``n=1``) + graph edges out from the :term:`active window` of the workflow. + Thus in the context of the GUI the active window may also be + referred to as the `n=0` window. active cycle - A cycle point is active if it contains any :term:`active tasks `. - - Active cycles are counted towards the :term:`runahead limit`. - - - window - n-window - active window - The GUI provides a :term:`graph`-based window or view of the workflow at - runtime, including tasks out to ``n`` graph edges from current - :term:`active tasks `. + A cycle point is active if any of its tasks appear in the :term:`active window` + of the workflow. - Active tasks form the ``n=0`` window. + Active cycles are counted towards the :term:`runahead limit`. .. seealso:: @@ -1215,9 +1204,10 @@ Glossary workflow :term:`source directory` before reload, rather than made by editing the installed files directly. - :ref:`RemoteInit` will be redone for each job platform, when the first job is submitted there after a reload. + :ref:`RemoteInit` will be redone for each job platform, when the first + job is submitted there after a reload. - Any :term:`task` that is :term:`active ` at reload + Any :term:`task` that is already present in the :term:`active window` at reload will continue with its pre-reload configuration. The next instance of the task (at the next cycle point) will adopt the new configuration. @@ -1613,7 +1603,7 @@ Glossary suicide trigger Suicide triggers remove :term:`tasks ` from the - :term:`active window ` at runtime. + :term:`active window` at runtime. They are denoted by exclamation marks, and are triggered like normal dependencies. For instance, the following suicide trigger will remove the @@ -1701,9 +1691,8 @@ Glossary flow front - :term:`Active tasks `, i.e. tasks in the - :term:`n=0 window `, with a common :term:`flow number` - comprise the active front of the flow. + Tasks in the :term:`active window` of the workflow with a common + :term:`flow number` comprise the active front of the flow. flow merge @@ -1750,8 +1739,8 @@ Glossary ahead of the oldest :term:`active cycle` the workflow is permitted to run. - The "oldest active cycle point" is the first cycle in the workflow to contain - any :term:`active tasks ` (e.g. running tasks). + The "oldest active cycle point" is the first cycle in the workflow to + contain any submitted or running tasks. .. seealso:: @@ -1765,11 +1754,11 @@ Glossary :term:`shut down `, if no tasks remain in the :term:`n=0 `. - That is, all active tasks have finished, and no tasks remain waiting on + That is, all :term:`active tasks ` have finished, and no tasks remain waiting on :term:`prerequisites ` or "external" constraints (such as :term:`xtriggers ` or task :term:`hold`). - If no active tasks remain and all external constraints are satisfied, + If no :term:`active tasks ` remain and all external constraints are satisfied, but the n=0 window contains tasks waiting with partially satisfied :term:`prerequisites `, or tasks with :term:`final status` and :term:`incomplete outputs `, then the workflow is diff --git a/src/reference/changes.rst b/src/reference/changes.rst index c2a6b25cb9..04b174fa3e 100644 --- a/src/reference/changes.rst +++ b/src/reference/changes.rst @@ -196,17 +196,14 @@ N-Window selector in the GUI The :term:`n-window` determines how much of a workflow is visible in the GUI / Tui. -The ``n=0`` window contains only the active tasks -(i.e. queued, preparing, submitted or running tasks). +The ``n=0`` window displays only the scheduler's :term:`active window`. -The ``n=1`` window, also contains tasks one "edge" out from active tasks -(i.e. the tasks immediately upstream or downstream of active tasks). +The ``n=1`` window also contains tasks one graph "edge", i.e. immediately +upstream or downstream from, those in the :term:`active window`. And so on, for +``n=2`` and ``n=3``. -The ``n=2`` window, also contains tasks two "edges" out from active tasks, -and so on. - -It is now possible to change the window extent in the GUI via a button in the -toolbar allowing you to see tasks further back in the workflow's history. +You can change the n-window extent in the GUI with a toolbar button, to display +fewer or more tasks around the current ``n=0`` :term:`active window`. .. image:: changes/gui-n-window-selector.gif :width: 100% diff --git a/src/user-guide/interventions/index.rst b/src/user-guide/interventions/index.rst index 202a586473..d2880f6622 100644 --- a/src/user-guide/interventions/index.rst +++ b/src/user-guide/interventions/index.rst @@ -558,7 +558,8 @@ Remove Tasks The removed task will be greyed out but it might not disappear from view because the GUI displays all tasks - in a graph-based window around current active tasks. + in a graph-based :term:`n-window` surrounding the + current :term:`active window`. .. tab-item:: CLI diff --git a/src/user-guide/running-workflows/reflow.rst b/src/user-guide/running-workflows/reflow.rst index 0f7dc7b122..3f8502cea9 100644 --- a/src/user-guide/running-workflows/reflow.rst +++ b/src/user-guide/running-workflows/reflow.rst @@ -117,19 +117,18 @@ Triggering with No Active Flows ``cylc trigger [--wait] ID`` By default, triggered tasks will be given the flow numbers of the most - recent active task. This can happen, for instance, if you restart a + recent :term:`active tasks `. This can happen, for instance, if you restart a completed workflow and then trigger a task in it. The result will be the same as if you had triggered the task just before the workflow completed. -Special Case: Triggering ``n=0`` Tasks - Tasks in the ``n=0`` window are :term:`active tasks `. - Their flow membership is already determined - that of - the parent tasks that spawned them. +Special Case: Triggering ``n=0`` (:term:`active window`) Tasks + Tasks in the :term:`active window` already have flow membership assigned + (namely that of the parent tasks that spawned them). - Triggering a task with a submitted or running job has no effect (it is already triggered). - - Triggering other :term:`n=0 tasks `, including tasks with - :term:`incomplete outputs ` queues them to run in + - Triggering other :term:`active window` tasks, including tasks with + :term:`incomplete outputs `, queues them to run in the flow that they already belong to. @@ -142,8 +141,9 @@ If a task spawning into the :term:`n=0 window ` finds another instance of itself (same task ID) already there, the two will merge and carry both (sets of) flow numbers forward. Downstream tasks will belong to both flows. -Flow merging is necessary because :term:`active ` task IDs -must be unique. +Flow merging is necessary because tasks in the :term:`active window` must have +unique task IDs. However, it is also useful: it allows a single flow to carry +on downstream of multi-flow interventions. If the original task instance has a :term:`final status` (and has been retained in the :term:`n=0 window ` with @@ -157,7 +157,7 @@ Stopping Flows By default, ``cylc stop`` halts the workflow and shuts the scheduler down. It can also stop specific flows: ``cylc stop --flow=N`` removes the flow number -``N`` from :term:`active tasks `. Tasks that have no flow +``N`` from tasks in the :term:`active window`. Tasks that have no flow numbers left as a result do not spawn children at all. If there are no active flows left, the scheduler shuts down. diff --git a/src/user-guide/running-workflows/tasks-jobs-ui.rst b/src/user-guide/running-workflows/tasks-jobs-ui.rst index 813c16bfd6..ed26ce3c00 100644 --- a/src/user-guide/running-workflows/tasks-jobs-ui.rst +++ b/src/user-guide/running-workflows/tasks-jobs-ui.rst @@ -102,31 +102,32 @@ The "n" Window .. versionchanged:: 8.0.0 -Cylc workflow :term:`graphs ` can be very large, even infinite for -:term:`cycling workflows ` with no :term:`final cycle point`. +Cylc workflow :term:`graphs ` can be very large, or infinite in +extent for :term:`cycling workflows ` with no +:term:`final cycle point`. Consequently the GUI often can't display "all of the tasks" at once. Instead -it displays all :term:`active tasks ` (e.g. running tasks) -as well as any tasks which are a configurable number of tasks away from -them in the task dependency :term:`graph`. +it displays all tasks in the :term:`active window` of the workflow, as well +as any tasks out to a configurable number of graph edges away from them in +the task dependency :term:`graph`. .. image:: ../../img/n-window.png :align: center n=0: - The ``n=0`` window contains :term:`active tasks `. An active - task is a task which is near ready to run, in the process of running, or - which requires user intervention (see the :term:`glossary ` - for a more detailed description). + The ``n=0`` window corresponds to the scheduler's active window: + tasks that are near ready to run, in the process of running, or + which require user intervention. See the :term:`glossary ` + for a more detailed description. n=1: - The ``n=1`` window contains all "active tasks" as well as any tasks one - "edge" out from them, i.e. their dependencies (the tasks that come immediately - before them in the graph) and their descendants (the tasks that come - immediately after them in the graph). + The ``n=1`` window contains the ``n=0`` window plus all tasks out to one + graph edge around them, i.e. their dependencies (the tasks that come immediately + before them in the graph) and their descendants (immediately after them in the + graph). n=2: - The ``n=2`` window contains all "active tasks" as well as any tasks *two* - edges out from them, and so on. + The ``n=2`` window extends out to *two* graph edges around + the :term:`active window`, and so on. This animation shows how the n-window advances as a workflow runs, tasks are colour coded according to their n-window value with the colours changing from diff --git a/src/user-guide/running-workflows/workflow-completion.rst b/src/user-guide/running-workflows/workflow-completion.rst index 147dca239c..29ee4a8001 100644 --- a/src/user-guide/running-workflows/workflow-completion.rst +++ b/src/user-guide/running-workflows/workflow-completion.rst @@ -7,11 +7,11 @@ A workflow is complete, and the scheduler will automatically :term:`shut down `, if no tasks remain in the :term:`n=0 `. -That is, all active tasks have finished, and no tasks remain waiting on +That is, all :term:`active tasks ` have finished, and no tasks remain waiting on :term:`prerequisites ` or "external" constraints (such as :term:`xtriggers ` or task :term:`hold`). -If no active tasks remain and all external constraints are satisfied, +If no :term:`active tasks ` remain and all external constraints are satisfied, but the n=0 window contains tasks waiting with partially satisfied :term:`prerequisites `, or tasks with :term:`final status` and :term:`incomplete outputs `, then the workflow is diff --git a/src/user-guide/task-implementation/skip-mode.rst b/src/user-guide/task-implementation/skip-mode.rst index 35fb615162..3da2f4254d 100644 --- a/src/user-guide/task-implementation/skip-mode.rst +++ b/src/user-guide/task-implementation/skip-mode.rst @@ -14,8 +14,8 @@ Skip mode is designed as an aid to workflow control: * It allows skipping of tasks in a running workflow using either: * ``cylc broadcast -s 'run mode = skip'`` (for when it is ready to run). This will work with any future task or family. - * ``cylc set --out skip`` (to immediately skip). This only works with - globs for active tasks. Otherwise task names must be explicit. + * ``cylc set --out skip`` (to immediately skip). Note that globs only match + tasks in the :term:`active window` of the workflow. Otherwise task names must be explicit. .. note:: diff --git a/src/user-guide/troubleshooting.rst b/src/user-guide/troubleshooting.rst index b77d5f8881..1f5b1b2af2 100644 --- a/src/user-guide/troubleshooting.rst +++ b/src/user-guide/troubleshooting.rst @@ -293,9 +293,9 @@ Why isn't my task running? To find out why a task is not being run, use the ``cylc show`` command. This will list the task's prerequisites and xtriggers. -Note, at present ``cylc show`` can only display -:term:`active tasks `. Waiting tasks beyond the -:term:`n=0 window ` have no satisfied prerequisites. +Note, at present ``cylc show`` can only display tasks in the +:term:`active window` (however, waiting tasks beyond that, by +definition, have no satisfied prerequisites). Note, tasks which are held |task-held| will not be run, use ``cylc release`` to release a held task. diff --git a/src/user-guide/writing-workflows/runtime.rst b/src/user-guide/writing-workflows/runtime.rst index 2dab27a0e6..e70846c56b 100644 --- a/src/user-guide/writing-workflows/runtime.rst +++ b/src/user-guide/writing-workflows/runtime.rst @@ -760,7 +760,7 @@ Late Events .. warning:: The scheduler can only check for lateness once a task has appeared in its - active task window. In Cylc 8 this is usually when the task is actually + :term:`active window`. In Cylc 8 this is usually when the task is actually ready to run, which severely limits the usefulness of late events as currently implemented. diff --git a/src/user-guide/writing-workflows/scheduling.rst b/src/user-guide/writing-workflows/scheduling.rst index df61a140ef..d698cf0f66 100644 --- a/src/user-guide/writing-workflows/scheduling.rst +++ b/src/user-guide/writing-workflows/scheduling.rst @@ -2160,10 +2160,10 @@ Runahead Limiting ----------------- Runahead limiting prevents a workflow from getting too far ahead of the oldest -active cycle point by holding back tasks in cycles beyond a specified limit. +:term:`active cycle point` by holding back tasks in cycles beyond a specified limit. The runahead limit is defined as an interval measured from the oldest active cycle. -A cycle is considered to be "active" if it contains any :term:`active` tasks +A cycle is considered to be "active" if it contains any :term:`active window` tasks (e.g. running tasks). Tasks in cycles which are beyond the limit are called :term:`runahead` tasks