From 0946157fdcc7b61067e5c5626893439cb052f277 Mon Sep 17 00:00:00 2001 From: Victor Petrovykh Date: Tue, 25 Feb 2025 05:35:23 -0500 Subject: [PATCH] Update the `gel watch` docs. Update the `gel.toml` description. Add `[hooks]` and `[[watch]]` documentation. Update the docs for the `gel watch` command. Update the `gel watch` description in the migration docs. --- docs/intro/migrations.rst | 14 ++- .../cli/gel_migration/gel_migration_apply.rst | 4 +- docs/reference/cli/gel_watch.rst | 52 +++++++--- docs/reference/datamodel/migrations.rst | 4 +- docs/reference/reference/gel_toml.rst | 99 ++++++++++++++++++- docs/resources/guides/migrations/guide.rst | 66 +++++++------ 6 files changed, 188 insertions(+), 51 deletions(-) diff --git a/docs/intro/migrations.rst b/docs/intro/migrations.rst index dc37a5e2680..ddbd5b0c736 100644 --- a/docs/intro/migrations.rst +++ b/docs/intro/migrations.rst @@ -16,13 +16,21 @@ new instance and some empty schema files to get you started. ------------------------------ The easiest way to work with your schema in development is by running -:gelcmd:`watch`. This long-running task will monitor your schema files and +:gelcmd:`watch --migrate`. This long-running task will monitor your schema files and automatically apply schema changes in your database as you work. .. code-block:: bash - $ gel watch - Initialized. Monitoring "/projects/my-gel-project". + $ gel watch --migrate + Hint: --migrate will apply any changes from your schema files to the database. + When ready to commit your changes, use: + 1) `gel migration create` to write those changes to a migration file, + 2) `gel migrate --dev-mode` to replace all synced changes with the migration. + + Monitoring /home/instancename for changes in: + + --migrate: gel migration apply --dev-mode + If you get output similar to the output above, you're ready to get started! diff --git a/docs/reference/cli/gel_migration/gel_migration_apply.rst b/docs/reference/cli/gel_migration/gel_migration_apply.rst index 8fa1e98fd68..480894a1910 100644 --- a/docs/reference/cli/gel_migration/gel_migration_apply.rst +++ b/docs/reference/cli/gel_migration/gel_migration_apply.rst @@ -52,5 +52,5 @@ to. For specifying the connection target see :ref:`connection options :cli:synopsis:`--dev-mode` Apply the current schema changes on top of the current migration history, without having created a new migration. This works the same way as - :ref:`ref_cli_gel_watch` but without starting a long-running watch - task. + :ref:`gel watch --migrate ` but without starting a + long-running watch task. diff --git a/docs/reference/cli/gel_watch.rst b/docs/reference/cli/gel_watch.rst index fd66845c901..878e0cfe8ec 100644 --- a/docs/reference/cli/gel_watch.rst +++ b/docs/reference/cli/gel_watch.rst @@ -5,25 +5,53 @@ gel watch ========= -Start a long-running process that watches for changes in schema files in your -project's ``dbschema`` directory and applies those changes to your current -|branch| in real time. Starting it is as simple as running this command: +Start a long-running process that watches for changes as specified in the +:ref:`gel.toml ` file. This process will monitor the +project for changes specified in the ``[[watch]]`` table array and run the +associated scripts in response to those changes. + +When multiple changes target the same ``[[watch]]`` element, the corresponding +script will be triggered only once. All triggered watch scripts will be +executed in parallel. If the same script is triggered before it finishes +executing, the next execution will wait for the already running script to +terminate (i.e. only one instance of the same script will be runing at the +same time). -.. cli:synopsis:: +.. note:: - gel watch + Any output that the triggered scripts produce will be shown in the + :gelcmd:`watch` console. This includes any error messages. So if you're + not seeing a change you've expected, check on the watch process to make + sure there aren't any unexpected errors in the triggered scripts. -.. note:: + +Options +======= + +.. warning:: + + This command changed in version 6. In older versions it only monitored the + schema file changes and it had no additional options. + +.. versionadded:: 6.0 + +:cli:synopsis:`--migrate` + Watches for changes in schema files in your project's ``dbschema`` + directory and applies those changes to your current |branch| in real time. If a schema change cannot be applied, you will see an error in the :gelcmd:`watch` console. You will also receive the error when you try to run a query with any |Gel| client binding. -To learn about our recommended development migration workflow using -:gelcmd:`watch`, read our :ref:`intro to migrations `. + .. note:: -.. note:: + If you want to apply a migration in the same manner as ``watch + --migrate`` but without the long-running process, use :gelcmd:`migrate + --dev-mode`. See :ref:`ref_cli_gel_migration_apply` for more details. + + To learn about our recommended development migration workflow using + :gelcmd:`watch`, read our :ref:`intro to migrations + `. - If you want to apply a migration in the same manner as ``watch`` but - without the long-running process, use :gelcmd:`migrate --dev-mode`. See - :ref:`ref_cli_gel_migration_apply` for more details. +:cli:synopsis:`-v, --verbose` + Verbose output. diff --git a/docs/reference/datamodel/migrations.rst b/docs/reference/datamodel/migrations.rst index 9b941e8a536..97cc334758d 100644 --- a/docs/reference/datamodel/migrations.rst +++ b/docs/reference/datamodel/migrations.rst @@ -73,8 +73,8 @@ Sometimes when you're prototyping something new you don't want to spend time worrying about migrations. There's no data to lose and not much code that depends on the schema just yet. -For this use case you can use the :gelcmd:`watch` command, which will -monitor your |.gel| files and automatically create and apply migrations +For this use case you can use the :gelcmd:`watch --migrate` command, which +will monitor your |.gel| files and automatically create and apply migrations for you in the background. .. _ref_eql_ddl: diff --git a/docs/reference/reference/gel_toml.rst b/docs/reference/reference/gel_toml.rst index bda4e47562c..80b2d1d4db0 100644 --- a/docs/reference/reference/gel_toml.rst +++ b/docs/reference/reference/gel_toml.rst @@ -7,7 +7,7 @@ gel.toml The |gel.toml| file is created in the project root after running :ref:`ref_cli_gel_project_init`. If this file is present in a directory, it signals to the CLI and client bindings that the directory is an instance-linked -|Gel| project. It supports two configuration settings across two tables: +|Gel| project. It supports the following configuration settings: .. note:: @@ -45,6 +45,91 @@ signals to the CLI and client bindings that the directory is an instance-linked Defaults to ``dbschema``. +[hooks] table +============= + +.. versionadded:: 6 + +This table may contain the following keys, all of which are optional: + +- ``project.init.before`` +- ``project.init.after`` +- ``branch.switch.before`` +- ``branch.wipe.before`` +- ``migration.apply.before`` +- ``schema.update.before`` +- ``branch.switch.after`` +- ``branch.wipe.after`` +- ``migration.apply.after`` +- ``schema.update.after`` + +Each key represents a command hook that will be executed together with a CLI +command. All keys have a string value which is going to be executed as a shell +command when the corresponding hook is triggered. + +Hooks are divided into two categories: ``before`` and ``after`` as indicated +by their names. All of the ``before`` hooks are executed prior to their +corresponding commands, so they happen before any changes are made. All of the +``after`` hooks run after the CLI command and thus the effects from the +command are already in place. Any error during the hook script execution will +terminate the CLI command (thus ``before`` hooks are able to prevent their +commands from executing if certain conditions are not met). + +Overall, when multiple hooks are triggered they all execute sequentially in +the order they are listed above. + +Here is a breakdown of which command trigger which hooks: + +- :ref:`ref_cli_gel_project_init` command triggers the ``project.init.before`` + and ``project.init.after`` hook. If the migrations are applied at the end of + the initialization, then the ``migration.apply.before``, + ``schema.update.before``, ``migration.apply.after``, and + ``schema.update.after`` hooks are also triggered. +- :ref:`ref_cli_gel_branch_switch` command triggers ``branch.switch.before``, + ``schema.update.before``, ``branch.switch.after``, and ``schema.update.after`` + hooks in that relative order. +- :ref:`ref_cli_gel_branch_wipe` command triggers the ``branch.wipe.before``, + ``schema.update.before``, ``branch.wipe.after``, and ``schema.update.after`` + hooks in that relative order. +- :ref:`ref_cli_gel_branch_rebase` and :ref:`ref_cli_gel_branch_merge` + commands trigger ``migration.apply.before``, ``schema.update.before``, + ``migration.apply.after``, and ``schema.update.after`` hooks in that + relative order. Notice that although these are branch commands, but they do + not change the current branch, instead they modify and apply migrations. + That's why they trigger the ``migration.apply`` hooks. +- :ref:`ref_cli_gel_migration_apply` command triggers + ``migration.apply.before``, ``schema.update.before``, + ``migration.apply.after``, and ``schema.update.after`` hooks in that + relative order. + + .. note:: + + All of these hooks are intended as project management tools. For this + reason they will only be triggered by the CLI commands that *don't + override* default project settings. Any CLI command that uses + :ref:`connection options ` will not trigger any + hooks. + +This is implementing `RFC 1028 `_. + +[[watch]] table array +===================== + +.. versionadded:: 6 + +Each element of this table array may contain the following required keys: + +- ``files = ["", ...]`` - specify file(s) being watched. + + The paths must use ``/`` (\*nix-style) as path separators. They can also contain glob pattrens (``*``, ``**``, ``?``, etc.) in order to specify multiple files at one. + +- ``script = ""`` - command to be executed by the shell. + +The watch mode can be activated by the :ref:`ref_cli_gel_watch` command. + +This is implementing `RFC 1028 `_. + + Example ======= @@ -56,9 +141,21 @@ Example [project] schema-dir = "db/schema" + [hooks] + project.init.after = "setup_dsn.sh" + branch.switch.after = "setup_dsn.sh" + schema.update.after = "gel-orm sqlalchemy --mod compat --out compat" + + [[watch]] + files = ["queries/*.edgeql"] + script = "npx @edgedb/generate queries" + .. lint-off .. _all of the same version specifications as Cargo: https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#specifying-dependencies +.. _rfc1028: + https://github.com/edgedb/rfcs/blob/master/text/1028-cli-hooks.rst + .. lint-on diff --git a/docs/resources/guides/migrations/guide.rst b/docs/resources/guides/migrations/guide.rst index ee835abd886..861ec6f583b 100644 --- a/docs/resources/guides/migrations/guide.rst +++ b/docs/resources/guides/migrations/guide.rst @@ -623,7 +623,7 @@ Squashing migrations Users often end up making many changes to their schema because of how effortless it is to do. (And in the next section we will learn -about :gelcmd:`watch`, which is even more effortless!) This leads to +about :gelcmd:`watch --migrate`, which is even more effortless!) This leads to an interesting side effect: lots of ``.edgeql`` files, many of which represent trials and approaches that don't end up making it to the final schema. @@ -807,25 +807,30 @@ We'll take its suggestion to apply the migration: Gel Watch ========= -Another option when quickly iterating over schema changes is :gelcmd:`watch`. +Another option when quickly iterating over schema changes is :gelcmd:`watch --migrate`. This will create a long-running process that keeps track of every time you -save an |.gel| file inside your ``/migrations`` folder, letting you know -if your changes have successfully compiled or not. The :gelcmd:`watch` +save a |.gel| file inside your ``/migrations`` folder, letting you know +if your changes have successfully compiled or not. The :gelcmd:`watch --migrate` command itself will show the following input when the process starts up: .. code-block:: Connecting to Gel instance 'anything' at localhost:10700... - Gel Watch initialized. - Hint: Use `gel migration create` and `gel migrate --dev-mode` - to apply changes once done. - Monitoring "/home/instancename". + Hint: --migrate will apply any changes from your schema files to the database. + When ready to commit your changes, use: + 1) `gel migration create` to write those changes to a migration file, + 2) `gel migrate --dev-mode` to replace all synced changes with the migration. -Unseen to the user, :gelcmd:`watch` will begin creating individual migration -scripts for every time you save a change to one of your files. These + Monitoring /home/instancename for changes in: + + --migrate: gel migration apply --dev-mode + + +Unseen to the user, :gelcmd:`watch --migrate` will begin creating individual +migration scripts for every time you save a change to one of your files. These are stored as separate "dev mode" migrations, which are sort of like -preliminary migrations that haven't been turned into a standalone -migration script yet. +preliminary migrations that haven't been turned into a standalone migration +script yet. We can test this out by starting with this schema: @@ -849,7 +854,7 @@ hit after making a change to the following schema: } } -You will see a quick "calculating diff" show up as :gelcmd:`watch` checks +You will see a quick "calculating diff" show up as :gelcmd:`watch --migrate` checks to see that the change we made was a valid one. As the change we made was to a valid schema, the "calculating diff" message will disappear pretty quickly. @@ -867,8 +872,8 @@ more verbose. Let's add some incorrect syntax to the existing schema: } } -Once you hit save, :gelcmd:`watch` will suddenly pipe up and inform you -that the schema can't be resolved: +Once you hit save, :gelcmd:`watch --migrate` will suddenly pipe up and inform +you that the schema can't be resolved: .. code-block:: @@ -879,26 +884,27 @@ that the schema can't be resolved: │ ^^^ error Schema migration error: - cannot proceed until .gel files are fixed + cannot proceed until schema files are fixed Once you correct the ``i32`` type to ``int32``, you will see a message letting you know that things are okay now. .. code-block:: - Resolved. Schema is up to date now. + Schema is up to date. The process will once again quieten down, but will continue to watch your schema and apply migrations to any changes you make to your schema. -:gelcmd:`watch` is best run in a separate instance of your command line so -that you can take care of other tasks — including officially migrating -when you are satisfied with your current schema — without having to -stop the process. +:gelcmd:`watch --migrate` is best run in a separate instance of your command +line so that you can take care of other tasks — including officially migrating +when you are satisfied with your current schema — without having to stop the +process. -If you are curious what is happening as :gelcmd:`watch` does its thing, -try the following query after you have made some changes. It will return -a few lists of applied migrations, grouped by the way they were generated. +If you are curious what is happening as :gelcmd:`watch --migrate` does its +thing, try the following query after you have made some changes. It will +return a few lists of applied migrations, grouped by the way they were +generated. .. code-block:: @@ -907,8 +913,8 @@ a few lists of applied migrations, grouped by the way they were generated. script } by .generated_by; -Some migrations will contain nothing in their ``generated_by`` property, -while those generated by :gelcmd:`watch` will have a +Some migrations will contain nothing in their ``generated_by`` property, while +those generated by :gelcmd:`watch --migrate` will have a ``MigrationGeneratedBy.DevMode``. .. note:: @@ -918,11 +924,9 @@ while those generated by :gelcmd:`watch` will have a show up if you directly change your schema by using DDL, which is generally not recommended. -Once you are satisfied with your changes while running :gelcmd:`watch`, -just create the migration with :gelcmd:`migration create` and then -apply them with one small tweak to the ``migrate`` command: -:gelcmd:`migrate --dev-mode` to let the CLI know to apply the migrations -made during dev mode that were made by :gelcmd:`watch`. +Once you are satisfied with your changes while running :gelcmd:`watch +--migrate`, just create the migration with :gelcmd:`migration create` to +record the current changes to the file system. Branches ========