Skip to content

Commit

Permalink
Disambiguate 'active tasks'.
Browse files Browse the repository at this point in the history
  • Loading branch information
hjoliver committed Feb 18, 2025
1 parent e1ae078 commit 7af814f
Show file tree
Hide file tree
Showing 11 changed files with 80 additions and 90 deletions.
8 changes: 5 additions & 3 deletions src/7-to-8/major-changes/scheduling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@ Cylc can manage infinite workflows of repeating tasks:

Cylc 8 has a new scheduling algorithm that:

- Is much more efficient because it only has to manage active tasks
- Is much more efficient because it doesn't need to track nearly as
many waiting and succeeded tasks in addition to the active
(submitted and running) ones.

- waiting tasks are not pre-spawned before they are needed
- succeeded tasks are not kept across the active task window
- succeeded tasks are not retained across the :term:`active window`
- no costly indiscriminate dependency matching is done
- Distinguishes between :term:`optional <optional output>` and
:term:`required <required output>` task outputs, to support:
Expand All @@ -37,7 +39,7 @@ Cylc 8 has a new scheduling algorithm that:

- instances of same task can run out of cycle point order
- the workflow will not unnecessarily stall downstream of failed tasks
- Provides a sensible active-task based window on the evolving workflow
- Provides a sensible activity-based window on the evolving workflow

- (to fully understand which tasks appeared in the Cylc 7 GUI you had to
understand the scheduling algorithm)
Expand Down
73 changes: 31 additions & 42 deletions src/glossary.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,50 +67,39 @@ Glossary
Submit number also appears in the job log path so that job log files
don't get overwritten.


active
active task
An active task is a task which is near ready to run, in the process of
running, or which requires user intervention. These are all the tasks
being actively managed by the scheduler at this point in the run.
Tasks that are in the ``submitted`` and ``running`` states, i.e., those
with active jobs. Be aware that the :term:`active window` of the workflow
may also contain some tasks that are not active in this sense.

Active tasks are:
window
active window
The active window of a workflow contains only the tasks that Cylc needs
to actively manage in order to perform its job of scheduling efficiently:

- Tasks which have some, but not all of their
:term:`prerequisites <prerequisite>` satisfied.
- ``waiting`` tasks, that are actively waiting on:
- :term:`active tasks <active task>` (``submitted`` and ``running``, with active jobs)
- ``preparing`` tasks - i.e. tasks in the process of submitting jobs
- ``waiting`` tasks that are nearly ready to run but:

- :term:`xtriggers <xtrigger>`.
- :ref:`internal queues <InternalQueues>`
- :ref:`runahead limit <RunaheadLimit>`
- still have some (but not all) unsatisfied :term:`prerequisites <prerequisite>`
- are waiting on :term:`xtriggers <xtrigger>`,
:ref:`internal queues <InternalQueues>`, or the :ref:`runahead limit <RunaheadLimit>`

- ``preparing`` tasks - i.e. tasks in the process of submitting jobs
- ``submitted`` and ``running`` tasks - i.e. those with active jobs
- tasks that reached a :term:`final status` without completing their
:term:`required outputs <required output>`
(e.g. a task failed where success was required).

Active tasks are in the ``n=0`` :term:`window <n-window>` which means they
will always be displayed in the GUI and Tui.

The distinction between active and non-active tasks is important for
the computing of the :term:`runahead limit`.

n-window
The GUI provides a view extending ``n`` (default ``n=1``)
graph edges out from the :term:`active window` of the workflow.
Thus in the context of the GUI the active window may also be
referred to as the `n=0` window.

active cycle
A cycle point is active if it contains any :term:`active tasks <active task>`.

Active cycles are counted towards the :term:`runahead limit`.


window
n-window
active window
The GUI provides a :term:`graph`-based window or view of the workflow at
runtime, including tasks out to ``n`` graph edges from current
:term:`active tasks <active task>`.
A cycle point is active if any of its tasks appear in the :term:`active window`
of the workflow.

Active tasks form the ``n=0`` window.
Active cycles are counted towards the :term:`runahead limit`.

.. seealso::

Expand Down Expand Up @@ -1215,9 +1204,10 @@ Glossary
workflow :term:`source directory` before reload, rather than made by
editing the installed files directly.

:ref:`RemoteInit` will be redone for each job platform, when the first job is submitted there after a reload.
:ref:`RemoteInit` will be redone for each job platform, when the first
job is submitted there after a reload.

Any :term:`task` that is :term:`active <active task>` at reload
Any :term:`task` that is already present in the :term:`active window` at reload
will continue with its pre-reload configuration.
The next instance of the task (at the next cycle point)
will adopt the new configuration.
Expand Down Expand Up @@ -1613,7 +1603,7 @@ Glossary

suicide trigger
Suicide triggers remove :term:`tasks <task>` from the
:term:`active window <n-window>` at runtime.
:term:`active window` at runtime.

They are denoted by exclamation marks, and are triggered like normal
dependencies. For instance, the following suicide trigger will remove the
Expand Down Expand Up @@ -1701,9 +1691,8 @@ Glossary


flow front
:term:`Active tasks <active task>`, i.e. tasks in the
:term:`n=0 window <n-window>`, with a common :term:`flow number`
comprise the active front of the flow.
Tasks in the :term:`active window` of the workflow with a common
:term:`flow number` comprise the active front of the flow.


flow merge
Expand Down Expand Up @@ -1750,8 +1739,8 @@ Glossary
ahead of the oldest :term:`active cycle` the workflow is permitted
to run.

The "oldest active cycle point" is the first cycle in the workflow to contain
any :term:`active tasks <active task>` (e.g. running tasks).
The "oldest active cycle point" is the first cycle in the workflow to
contain any submitted or running tasks.

.. seealso::

Expand All @@ -1765,11 +1754,11 @@ Glossary
:term:`shut down <shutdown>`, if no tasks remain in the
:term:`n=0 <n-window>`.

That is, all active tasks have finished, and no tasks remain waiting on
That is, all :term:`active tasks <active task>` have finished, and no tasks remain waiting on
:term:`prerequisites <prerequisite>` or "external" constraints (such as
:term:`xtriggers <xtrigger>` or task :term:`hold`).

If no active tasks remain and all external constraints are satisfied,
If no :term:`active tasks <active task>` remain and all external constraints are satisfied,
but the n=0 window contains tasks waiting with partially satisfied
:term:`prerequisites <prerequisite>`, or tasks with :term:`final status` and
:term:`incomplete outputs <output completion>`, then the workflow is
Expand Down
15 changes: 6 additions & 9 deletions src/reference/changes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -196,17 +196,14 @@ N-Window selector in the GUI

The :term:`n-window` determines how much of a workflow is visible in the GUI / Tui.

The ``n=0`` window contains only the active tasks
(i.e. queued, preparing, submitted or running tasks).
The ``n=0`` window displays only the scheduler's :term:`active window`.

The ``n=1`` window, also contains tasks one "edge" out from active tasks
(i.e. the tasks immediately upstream or downstream of active tasks).
The ``n=1`` window also contains tasks one graph "edge", i.e. immediately
upstream or downstream from, those in the :term:`active window`. And so on, for
``n=2`` and ``n=3``.

The ``n=2`` window, also contains tasks two "edges" out from active tasks,
and so on.

It is now possible to change the window extent in the GUI via a button in the
toolbar allowing you to see tasks further back in the workflow's history.
You can change the n-window extent in the GUI with a toolbar button, to display
fewer or more tasks around the current ``n=0`` :term:`active window`.

.. image:: changes/gui-n-window-selector.gif
:width: 100%
Expand Down
3 changes: 2 additions & 1 deletion src/user-guide/interventions/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -558,7 +558,8 @@ Remove Tasks

The removed task will be greyed out but it might not
disappear from view because the GUI displays all tasks
in a graph-based window around current active tasks.
in a graph-based :term:`n-window` surrounding the
current :term:`active window`.


.. tab-item:: CLI
Expand Down
20 changes: 10 additions & 10 deletions src/user-guide/running-workflows/reflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -117,19 +117,18 @@ Triggering with No Active Flows
``cylc trigger [--wait] ID``

By default, triggered tasks will be given the flow numbers of the most
recent active task. This can happen, for instance, if you restart a
recent :term:`active tasks <active task>`. This can happen, for instance, if you restart a
completed workflow and then trigger a task in it. The result will be the
same as if you had triggered the task just before the workflow completed.

Special Case: Triggering ``n=0`` Tasks
Tasks in the ``n=0`` window are :term:`active tasks <active task>`.
Their flow membership is already determined - that of
the parent tasks that spawned them.
Special Case: Triggering ``n=0`` (:term:`active window`) Tasks
Tasks in the :term:`active window` already have flow membership assigned
(namely that of the parent tasks that spawned them).

- Triggering a task with a submitted or running job has no effect
(it is already triggered).
- Triggering other :term:`n=0 tasks <n-window>`, including tasks with
:term:`incomplete outputs <output completion>` queues them to run in
- Triggering other :term:`active window` tasks, including tasks with
:term:`incomplete outputs <output completion>`, queues them to run in
the flow that they already belong to.


Expand All @@ -142,8 +141,9 @@ If a task spawning into the :term:`n=0 window <n-window>` finds another instance
of itself (same task ID) already there, the two will merge and carry both
(sets of) flow numbers forward. Downstream tasks will belong to both flows.

Flow merging is necessary because :term:`active <active task>` task IDs
must be unique.
Flow merging is necessary because tasks in the :term:`active window` must have
unique task IDs. However, it is also useful: it allows a single flow to carry
on downstream of multi-flow interventions.

If the original task instance has a :term:`final status` (and has been retained
in the :term:`n=0 window <n-window>` with
Expand All @@ -157,7 +157,7 @@ Stopping Flows
By default, ``cylc stop`` halts the workflow and shuts the scheduler down.

It can also stop specific flows: ``cylc stop --flow=N`` removes the flow number
``N`` from :term:`active tasks <active task>`. Tasks that have no flow
``N`` from tasks in the :term:`active window`. Tasks that have no flow
numbers left as a result do not spawn children at all. If there are no active
flows left, the scheduler shuts down.

Expand Down
31 changes: 16 additions & 15 deletions src/user-guide/running-workflows/tasks-jobs-ui.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,31 +102,32 @@ The "n" Window

.. versionchanged:: 8.0.0

Cylc workflow :term:`graphs <graph>` can be very large, even infinite for
:term:`cycling workflows <cycling workflow>` with no :term:`final cycle point`.
Cylc workflow :term:`graphs <graph>` can be very large, or infinite in
extent for :term:`cycling workflows <cycling workflow>` with no
:term:`final cycle point`.

Consequently the GUI often can't display "all of the tasks" at once. Instead
it displays all :term:`active tasks <active task>` (e.g. running tasks)
as well as any tasks which are a configurable number of tasks away from
them in the task dependency :term:`graph`.
it displays all tasks in the :term:`active window` of the workflow, as well
as any tasks out to a configurable number of graph edges away from them in
the task dependency :term:`graph`.

.. image:: ../../img/n-window.png
:align: center


n=0:
The ``n=0`` window contains :term:`active tasks <active task>`. An active
task is a task which is near ready to run, in the process of running, or
which requires user intervention (see the :term:`glossary <active task>`
for a more detailed description).
The ``n=0`` window corresponds to the scheduler's active window:
tasks that are near ready to run, in the process of running, or
which require user intervention. See the :term:`glossary <active window>`
for a more detailed description.
n=1:
The ``n=1`` window contains all "active tasks" as well as any tasks one
"edge" out from them, i.e. their dependencies (the tasks that come immediately
before them in the graph) and their descendants (the tasks that come
immediately after them in the graph).
The ``n=1`` window contains the ``n=0`` window plus all tasks out to one
graph edge around them, i.e. their dependencies (the tasks that come immediately
before them in the graph) and their descendants (immediately after them in the
graph).
n=2:
The ``n=2`` window contains all "active tasks" as well as any tasks *two*
edges out from them, and so on.
The ``n=2`` window extends out to *two* graph edges around
the :term:`active window`, and so on.

This animation shows how the n-window advances as a workflow runs, tasks are
colour coded according to their n-window value with the colours changing from
Expand Down
4 changes: 2 additions & 2 deletions src/user-guide/running-workflows/workflow-completion.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ A workflow is complete, and the scheduler will automatically
:term:`shut down <shutdown>`, if no tasks remain in the
:term:`n=0 <n-window>`.

That is, all active tasks have finished, and no tasks remain waiting on
That is, all :term:`active tasks <active task>` have finished, and no tasks remain waiting on
:term:`prerequisites <prerequisite>` or "external" constraints (such as
:term:`xtriggers <xtrigger>` or task :term:`hold`).

If no active tasks remain and all external constraints are satisfied,
If no :term:`active tasks <active task>` remain and all external constraints are satisfied,
but the n=0 window contains tasks waiting with partially satisfied
:term:`prerequisites <prerequisite>`, or tasks with :term:`final status` and
:term:`incomplete outputs <output completion>`, then the workflow is
Expand Down
4 changes: 2 additions & 2 deletions src/user-guide/task-implementation/skip-mode.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ Skip mode is designed as an aid to workflow control:
* It allows skipping of tasks in a running workflow using either:
* ``cylc broadcast -s 'run mode = skip'`` (for when it is ready to run).
This will work with any future task or family.
* ``cylc set --out skip`` (to immediately skip). This only works with
globs for active tasks. Otherwise task names must be explicit.
* ``cylc set --out skip`` (to immediately skip). Note that globs only match
tasks in the :term:`active window` of the workflow. Otherwise task names must be explicit.

.. note::

Expand Down
6 changes: 3 additions & 3 deletions src/user-guide/troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -293,9 +293,9 @@ Why isn't my task running?
To find out why a task is not being run, use the ``cylc show`` command.
This will list the task's prerequisites and xtriggers.

Note, at present ``cylc show`` can only display
:term:`active tasks <active task>`. Waiting tasks beyond the
:term:`n=0 window <n-window>` have no satisfied prerequisites.
Note, at present ``cylc show`` can only display tasks in the
:term:`active window` (however, waiting tasks beyond that, by
definition, have no satisfied prerequisites).

Note, tasks which are held |task-held| will not be run, use ``cylc release``
to release a held task.
Expand Down
2 changes: 1 addition & 1 deletion src/user-guide/writing-workflows/runtime.rst
Original file line number Diff line number Diff line change
Expand Up @@ -760,7 +760,7 @@ Late Events
.. warning::

The scheduler can only check for lateness once a task has appeared in its
active task window. In Cylc 8 this is usually when the task is actually
:term:`active window`. In Cylc 8 this is usually when the task is actually
ready to run, which severely limits the usefulness of late events as
currently implemented.

Expand Down
4 changes: 2 additions & 2 deletions src/user-guide/writing-workflows/scheduling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2160,10 +2160,10 @@ Runahead Limiting
-----------------

Runahead limiting prevents a workflow from getting too far ahead of the oldest
active cycle point by holding back tasks in cycles beyond a specified limit.
:term:`active cycle point` by holding back tasks in cycles beyond a specified limit.

The runahead limit is defined as an interval measured from the oldest active cycle.
A cycle is considered to be "active" if it contains any :term:`active` tasks
A cycle is considered to be "active" if it contains any :term:`active window` tasks
(e.g. running tasks).

Tasks in cycles which are beyond the limit are called :term:`runahead` tasks
Expand Down

0 comments on commit 7af814f

Please sign in to comment.