CodeGen: more flexible output file locations

[fantix]

Background and Motivation

Now in most client bindings, the codegen have 2 modes for selecting the output path:

1. By default, each .edgeql file gets an individual generated file sitting side-by-side.
2. Optionally, the user could specify --file to get a single generated file for all .edgeql files inside the project directory. Some bindings allow specifying a path.

The default mode is not always the most suitable approach [1], the alternative single-file mode doesn't fulfill users' needs in particularly the Python binding ([2], [3]). The EdgeDB team is also discussing where to put the file by default [4].

Considering about the relative consistency of DX among client bindings, it might make sense to enhance the single-file mode to something more flexible.

Proposal

The idea is to merge the 2 modes, and use stub files to customize the codegen behavior. By default, we still generate separate files next to each .edgeql files, unless stub files are found under the project directory.

Stub files have a fixed file name .edgedb-codegen, can be placed multiple times in any sub-directory inside the project directory. The content of a stub file is a path to a file where generated code will be written. The path is relative to the stub file itself. The codegen will recursively use all .edgeql files under the directory enclosing the stub file to generate code. All .edgeql files will be included in at least 1 generated file - if no stub file is found in any parent folders, a separate file will be generated next to the .edgeql file - this is just the default mode.

Examples

(Borrowing @abbazs's sample from geldata/gel-python#411.)

1. Not using stub files (default mode, the same as the status quo):

queries
├── hero
│   ├── delete.edgeql
│   ├── delete_edgeql.py  # generated
│   ├── insert.edgeql
│   └── insert_edgeql.py  # generated
└── movies
    ├── delete.edgeql
    ├── delete_edgeql.py  # generated
    ├── insert.edgeql
    ├── insert_edgeql.py  # generated
    ├── select.edgeql
    ├── select_edgeql.py  # generated
    ├── update.edgeql
    └── update_edgeql.py  # generated

2. Using stub files:

queries
├── hero
│   ├── .edgedb-codegen  # content: "./edgeql.py"
│   ├── edgeql.py  # generated code of the following 2 files:
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── .edgedb-codegen  # content: "./edgeql.py"
    ├── edgeql.py  # generated code of the following 4 files:
    ├── delete.edgeql
    ├── insert.edgeql
    ├── select.edgeql
    └── update.edgeql

3. movies is not using a sub file:

queries
├── hero
│   ├── .edgedb-codegen  # content: "./edgeql.py"
│   ├── edgeql.py  # generated code of the following 2 files:
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── delete.edgeql
    ├── delete_edgeql.py  # generated
    ├── insert.edgeql
    ├── insert_edgeql.py  # generated
    ├── select.edgeql
    ├── select_edgeql.py  # generated
    ├── update.edgeql
    └── update_edgeql.py  # generated

4. Simulating legacy single-file mode:

queries
├── .edgedb-codegen  # content: "./queries_edgeql.py"
├── queries_edgeql.py  # generated from all .edgeql files below
├── hero
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── delete.edgeql
    ├── insert.edgeql
    ├── select.edgeql
    └── update.edgeql

5. Flexible path:

queries
├── hero_edgeql.py  # generated from hero/**/*.edgeql
├── hero
│   ├── .edgedb-codegen  # content: "../hero_edgeql.py"
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── .edgedb-codegen  # content: "/dev/null" - don't generate for movies/**/*.edgeql
    ├── delete.edgeql
    ├── insert.edgeql
    ├── select.edgeql
    └── update.edgeql

6. Nesting:

queries
├── .edgedb-codegen  # content: "./queries_edgeql.py"
├── queries_edgeql.py  # generated from all .edgeql files below
├── hero
│   ├── .edgedb-codegen  # content: "./edgeql.py"
│   ├── edgeql.py  # generated code of the following 2 files:
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── delete.edgeql
    ├── insert.edgeql
    ├── select.edgeql
    └── update.edgeql

7. Overwrite with default behavior in a sub-directory:

queries
├── .edgedb-codegen  # content: "./queries_edgeql.py"
├── queries_edgeql.py  # generated from all .edgeql files below
├── hero
│   ├── delete.edgeql
│   └── insert.edgeql
└── movies
    ├── .edgedb-codegen  # empty file: default behavior
    ├── delete.edgeql
    ├── delete_edgeql.py  # generated
    ├── insert.edgeql
    ├── insert_edgeql.py  # generated
    ├── select.edgeql
    ├── select_edgeql.py  # generated
    ├── update.edgeql
    └── update_edgeql.py  # generated

Potentially the stub file could also include other modifier instructions (like prefix), but I think we should start simple, especially when the upcoming type descriptor annotation may give codegen more information.

[tailhook]

The alternative is to have a [codegen] section in edgedb.toml with some rules, probably.

[Pentusha]

For me, the most convenient option would be for the codegen to create a repository class that would accept connection as init argument and with all queries from directory implemented as methods of this repo. So only one class and one file per entity/directory.

[tailhook]

@Pentusha, this is just one coding style. Usually classes-as-collections-of-functions are considered abusing OOP model. While I also understand the appeal of such pattern (i.e. not using connection argument). However, we can't support all styles that someone wants. So the goal is to provide the most common ones and then provide more tools for people to simplify their own codegens.

(@Pentusha, it's better to start another thread or probably discussion on this, because this thread should be discussing per-dir dotfiles vs whole project edgedb.toml)