Skip to content

Commit

Permalink
Update the gel watch docs.
Browse files Browse the repository at this point in the history
Update the `gel.toml` description. Add `[hooks]` and `[[watch]]`
documentation.

Update the docs for the `gel watch` command.

Update the `gel watch` description in the migration docs.
  • Loading branch information
vpetrovykh committed Feb 28, 2025
1 parent 680cef1 commit 0946157
Show file tree
Hide file tree
Showing 6 changed files with 188 additions and 51 deletions.
14 changes: 11 additions & 3 deletions docs/intro/migrations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,21 @@ new instance and some empty schema files to get you started.
------------------------------

The easiest way to work with your schema in development is by running
:gelcmd:`watch`. This long-running task will monitor your schema files and
:gelcmd:`watch --migrate`. This long-running task will monitor your schema files and
automatically apply schema changes in your database as you work.

.. code-block:: bash
$ gel watch
Initialized. Monitoring "/projects/my-gel-project".
$ gel watch --migrate
Hint: --migrate will apply any changes from your schema files to the database.
When ready to commit your changes, use:
1) `gel migration create` to write those changes to a migration file,
2) `gel migrate --dev-mode` to replace all synced changes with the migration.
Monitoring /home/instancename for changes in:
--migrate: gel migration apply --dev-mode
If you get output similar to the output above, you're ready to get started!

Expand Down
4 changes: 2 additions & 2 deletions docs/reference/cli/gel_migration/gel_migration_apply.rst
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,5 @@ to. For specifying the connection target see :ref:`connection options
:cli:synopsis:`--dev-mode`
Apply the current schema changes on top of the current migration history,
without having created a new migration. This works the same way as
:ref:`ref_cli_gel_watch` but without starting a long-running watch
task.
:ref:`gel watch --migrate <ref_cli_gel_watch>` but without starting a
long-running watch task.
52 changes: 40 additions & 12 deletions docs/reference/cli/gel_watch.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,53 @@
gel watch
=========

Start a long-running process that watches for changes in schema files in your
project's ``dbschema`` directory and applies those changes to your current
|branch| in real time. Starting it is as simple as running this command:
Start a long-running process that watches for changes as specified in the
:ref:`gel.toml <ref_reference_gel_toml>` file. This process will monitor the
project for changes specified in the ``[[watch]]`` table array and run the
associated scripts in response to those changes.

When multiple changes target the same ``[[watch]]`` element, the corresponding
script will be triggered only once. All triggered watch scripts will be
executed in parallel. If the same script is triggered before it finishes
executing, the next execution will wait for the already running script to
terminate (i.e. only one instance of the same script will be runing at the
same time).

.. cli:synopsis::
.. note::

gel watch
Any output that the triggered scripts produce will be shown in the
:gelcmd:`watch` console. This includes any error messages. So if you're
not seeing a change you've expected, check on the watch process to make
sure there aren't any unexpected errors in the triggered scripts.

.. note::

Options
=======

.. warning::

This command changed in version 6. In older versions it only monitored the
schema file changes and it had no additional options.

.. versionadded:: 6.0

:cli:synopsis:`--migrate`
Watches for changes in schema files in your project's ``dbschema``
directory and applies those changes to your current |branch| in real time.

If a schema change cannot be applied, you will see an error in the
:gelcmd:`watch` console. You will also receive the error when you
try to run a query with any |Gel| client binding.

To learn about our recommended development migration workflow using
:gelcmd:`watch`, read our :ref:`intro to migrations <ref_intro_migrations>`.
.. note::

.. note::
If you want to apply a migration in the same manner as ``watch
--migrate`` but without the long-running process, use :gelcmd:`migrate
--dev-mode`. See :ref:`ref_cli_gel_migration_apply` for more details.

To learn about our recommended development migration workflow using
:gelcmd:`watch`, read our :ref:`intro to migrations
<ref_intro_migrations>`.

If you want to apply a migration in the same manner as ``watch`` but
without the long-running process, use :gelcmd:`migrate --dev-mode`. See
:ref:`ref_cli_gel_migration_apply` for more details.
:cli:synopsis:`-v, --verbose`
Verbose output.
4 changes: 2 additions & 2 deletions docs/reference/datamodel/migrations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ Sometimes when you're prototyping something new you don't want to spend
time worrying about migrations. There's no data to lose and not much code
that depends on the schema just yet.

For this use case you can use the :gelcmd:`watch` command, which will
monitor your |.gel| files and automatically create and apply migrations
For this use case you can use the :gelcmd:`watch --migrate` command, which
will monitor your |.gel| files and automatically create and apply migrations
for you in the background.

.. _ref_eql_ddl:
Expand Down
99 changes: 98 additions & 1 deletion docs/reference/reference/gel_toml.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ gel.toml
The |gel.toml| file is created in the project root after running
:ref:`ref_cli_gel_project_init`. If this file is present in a directory, it
signals to the CLI and client bindings that the directory is an instance-linked
|Gel| project. It supports two configuration settings across two tables:
|Gel| project. It supports the following configuration settings:

.. note::

Expand Down Expand Up @@ -45,6 +45,91 @@ signals to the CLI and client bindings that the directory is an instance-linked
Defaults to ``dbschema``.


[hooks] table
=============

.. versionadded:: 6

This table may contain the following keys, all of which are optional:

- ``project.init.before``
- ``project.init.after``
- ``branch.switch.before``
- ``branch.wipe.before``
- ``migration.apply.before``
- ``schema.update.before``
- ``branch.switch.after``
- ``branch.wipe.after``
- ``migration.apply.after``
- ``schema.update.after``

Each key represents a command hook that will be executed together with a CLI
command. All keys have a string value which is going to be executed as a shell
command when the corresponding hook is triggered.

Hooks are divided into two categories: ``before`` and ``after`` as indicated
by their names. All of the ``before`` hooks are executed prior to their
corresponding commands, so they happen before any changes are made. All of the
``after`` hooks run after the CLI command and thus the effects from the
command are already in place. Any error during the hook script execution will
terminate the CLI command (thus ``before`` hooks are able to prevent their
commands from executing if certain conditions are not met).

Overall, when multiple hooks are triggered they all execute sequentially in
the order they are listed above.

Here is a breakdown of which command trigger which hooks:

- :ref:`ref_cli_gel_project_init` command triggers the ``project.init.before``
and ``project.init.after`` hook. If the migrations are applied at the end of
the initialization, then the ``migration.apply.before``,
``schema.update.before``, ``migration.apply.after``, and
``schema.update.after`` hooks are also triggered.
- :ref:`ref_cli_gel_branch_switch` command triggers ``branch.switch.before``,
``schema.update.before``, ``branch.switch.after``, and ``schema.update.after``
hooks in that relative order.
- :ref:`ref_cli_gel_branch_wipe` command triggers the ``branch.wipe.before``,
``schema.update.before``, ``branch.wipe.after``, and ``schema.update.after``
hooks in that relative order.
- :ref:`ref_cli_gel_branch_rebase` and :ref:`ref_cli_gel_branch_merge`
commands trigger ``migration.apply.before``, ``schema.update.before``,
``migration.apply.after``, and ``schema.update.after`` hooks in that
relative order. Notice that although these are branch commands, but they do
not change the current branch, instead they modify and apply migrations.
That's why they trigger the ``migration.apply`` hooks.
- :ref:`ref_cli_gel_migration_apply` command triggers
``migration.apply.before``, ``schema.update.before``,
``migration.apply.after``, and ``schema.update.after`` hooks in that
relative order.

.. note::

All of these hooks are intended as project management tools. For this
reason they will only be triggered by the CLI commands that *don't
override* default project settings. Any CLI command that uses
:ref:`connection options <ref_cli_gel_connopts>` will not trigger any
hooks.

This is implementing `RFC 1028 <rfc1028_>`_.

[[watch]] table array
=====================

.. versionadded:: 6

Each element of this table array may contain the following required keys:

- ``files = ["<path-string>", ...]`` - specify file(s) being watched.

The paths must use ``/`` (\*nix-style) as path separators. They can also contain glob pattrens (``*``, ``**``, ``?``, etc.) in order to specify multiple files at one.

- ``script = "<command>"`` - command to be executed by the shell.

The watch mode can be activated by the :ref:`ref_cli_gel_watch` command.

This is implementing `RFC 1028 <rfc1028_>`_.


Example
=======

Expand All @@ -56,9 +141,21 @@ Example
[project]
schema-dir = "db/schema"
[hooks]
project.init.after = "setup_dsn.sh"
branch.switch.after = "setup_dsn.sh"
schema.update.after = "gel-orm sqlalchemy --mod compat --out compat"
[[watch]]
files = ["queries/*.edgeql"]
script = "npx @edgedb/generate queries"
.. lint-off
.. _all of the same version specifications as Cargo:
https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies

.. _rfc1028:
https://github.com/edgedb/rfcs/blob/master/text/1028-cli-hooks.rst

.. lint-on
66 changes: 35 additions & 31 deletions docs/resources/guides/migrations/guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -623,7 +623,7 @@ Squashing migrations

Users often end up making many changes to their schema because
of how effortless it is to do. (And in the next section we will learn
about :gelcmd:`watch`, which is even more effortless!) This leads to
about :gelcmd:`watch --migrate`, which is even more effortless!) This leads to
an interesting side effect: lots of ``.edgeql`` files, many of which
represent trials and approaches that don't end up making it to the
final schema.
Expand Down Expand Up @@ -807,25 +807,30 @@ We'll take its suggestion to apply the migration:
Gel Watch
=========
Another option when quickly iterating over schema changes is :gelcmd:`watch`.
Another option when quickly iterating over schema changes is :gelcmd:`watch --migrate`.
This will create a long-running process that keeps track of every time you
save an |.gel| file inside your ``/migrations`` folder, letting you know
if your changes have successfully compiled or not. The :gelcmd:`watch`
save a |.gel| file inside your ``/migrations`` folder, letting you know
if your changes have successfully compiled or not. The :gelcmd:`watch --migrate`
command itself will show the following input when the process starts up:
.. code-block::
Connecting to Gel instance 'anything' at localhost:10700...
Gel Watch initialized.
Hint: Use `gel migration create` and `gel migrate --dev-mode`
to apply changes once done.
Monitoring "/home/instancename".
Hint: --migrate will apply any changes from your schema files to the database.
When ready to commit your changes, use:
1) `gel migration create` to write those changes to a migration file,
2) `gel migrate --dev-mode` to replace all synced changes with the migration.
Unseen to the user, :gelcmd:`watch` will begin creating individual migration
scripts for every time you save a change to one of your files. These
Monitoring /home/instancename for changes in:
--migrate: gel migration apply --dev-mode
Unseen to the user, :gelcmd:`watch --migrate` will begin creating individual
migration scripts for every time you save a change to one of your files. These
are stored as separate "dev mode" migrations, which are sort of like
preliminary migrations that haven't been turned into a standalone
migration script yet.
preliminary migrations that haven't been turned into a standalone migration
script yet.
We can test this out by starting with this schema:
Expand All @@ -849,7 +854,7 @@ hit after making a change to the following schema:
}
}
You will see a quick "calculating diff" show up as :gelcmd:`watch` checks
You will see a quick "calculating diff" show up as :gelcmd:`watch --migrate` checks
to see that the change we made was a valid one. As the change we made was
to a valid schema, the "calculating diff" message will disappear pretty
quickly.
Expand All @@ -867,8 +872,8 @@ more verbose. Let's add some incorrect syntax to the existing schema:
}
}
Once you hit save, :gelcmd:`watch` will suddenly pipe up and inform you
that the schema can't be resolved:
Once you hit save, :gelcmd:`watch --migrate` will suddenly pipe up and inform
you that the schema can't be resolved:
.. code-block::
Expand All @@ -879,26 +884,27 @@ that the schema can't be resolved:
│ ^^^ error
Schema migration error:
cannot proceed until .gel files are fixed
cannot proceed until schema files are fixed
Once you correct the ``i32`` type to ``int32``, you will see a message
letting you know that things are okay now.
.. code-block::
Resolved. Schema is up to date now.
Schema is up to date.
The process will once again quieten down, but will continue to watch your
schema and apply migrations to any changes you make to your schema.
:gelcmd:`watch` is best run in a separate instance of your command line so
that you can take care of other tasks — including officially migrating
when you are satisfied with your current schema — without having to
stop the process.
:gelcmd:`watch --migrate` is best run in a separate instance of your command
line so that you can take care of other tasks — including officially migrating
when you are satisfied with your current schema — without having to stop the
process.
If you are curious what is happening as :gelcmd:`watch` does its thing,
try the following query after you have made some changes. It will return
a few lists of applied migrations, grouped by the way they were generated.
If you are curious what is happening as :gelcmd:`watch --migrate` does its
thing, try the following query after you have made some changes. It will
return a few lists of applied migrations, grouped by the way they were
generated.
.. code-block::
Expand All @@ -907,8 +913,8 @@ a few lists of applied migrations, grouped by the way they were generated.
script
} by .generated_by;
Some migrations will contain nothing in their ``generated_by`` property,
while those generated by :gelcmd:`watch` will have a
Some migrations will contain nothing in their ``generated_by`` property, while
those generated by :gelcmd:`watch --migrate` will have a
``MigrationGeneratedBy.DevMode``.
.. note::
Expand All @@ -918,11 +924,9 @@ while those generated by :gelcmd:`watch` will have a
show up if you directly change your schema by using DDL, which is
generally not recommended.
Once you are satisfied with your changes while running :gelcmd:`watch`,
just create the migration with :gelcmd:`migration create` and then
apply them with one small tweak to the ``migrate`` command:
:gelcmd:`migrate --dev-mode` to let the CLI know to apply the migrations
made during dev mode that were made by :gelcmd:`watch`.
Once you are satisfied with your changes while running :gelcmd:`watch
--migrate`, just create the migration with :gelcmd:`migration create` to
record the current changes to the file system.
Branches
========
Expand Down

0 comments on commit 0946157

Please sign in to comment.