New init-project command

[tailhook]

This is a follow-up on #1789

Project Initialization

Example 1:

$ edgedb init-project
There is no `dbschema` directory in `/home/user/work/myapp`.
Do you want to initialize a new project?
y
Noted EdgeDB version 1-beta1 in `dbschema/project.toml`
We will create new EdgeDB instance at localhost.
Enter instance name (type `cloud` to connect to EdgeDB cloud):
myapp
Creating EdgeDB instance `myapp`...
Saving `dbschema/local-credentials.json`
Adding local credentials to `.gitignore`
Creating `dbschema/default.esdl`
Done.

Example 2:

$ edgedb init-project
Found `dbschema/project.toml`
Using EdgeDB 1-alpha7
We will create new EdgeDB instance at localhost.
Enter instance name (type `cloud` to connect to EdgeDB cloud):
myapp
Creating EdgeDB instance `myapp`...
Saving `dbschema/local-credentials.json`
Applying migrations.

Done with 2 warnings:
1. To upgrade to latest stable (1-alpha7 -> 1-beta1) use:
     edgedb upgrade-project
2. `dbschema/local-credentials.json` is not ignored please add to `.gitignore`

Commands Introduced

• init-project -- bootstrap EdgeDB instance as configured in
  dbschema/project.toml, or initialize dbschema if no such directory.
• get-project-info {--edgedb-version|--default-instance-name} -- returns
  info from dbschema/project.toml
• upgrade-project -- runs edgedb server upgrade for the instance and
  updates project.toml

Note: edgedb init-project is also run as a part of curl | sh script.

project.toml Format

Two settings so far:

default-instance-name = "myapp"
edgedb-version = "1-beta1"  # or "nightly"

• edgedb-version is version used, not "minimal one", because users should
  make sure that their code will work on production, so installing newer than
  production one is not an option (Use edgedb upgrade-project at your own
  pace)
• default-instance-name is optional in this case instance name is guessed by
  directory name

Possible future extensions:

• prefered-cloud-account = "account1" for proprietary project that expect
  users to connect to their cloud even locally.
• prefer-docker (although it's unclear why do this in project config rather
  than in user's local config)

Connection Arguments

Bare edgedb looks for dbschema/local-credentials.json and
fails if there is no one.

edgedb -H or edgedb -P or edgedb -I or equivalent
environment variables EDGEDB_HOST, EDGEDB_PORT makes CLI ignore the file.

Note: EDGEDB_USER, EDGEDB_PASSWORD, EDGEDB_DATABASE do not ignore
local-credentials.json but override specific thing.

Language Bindings

Three forms:

# bare invocation
con = edgedb.connect()
# or equivalent construtor
con = edgedb.connect(Instance.from_env())
# or equivalent DSN
con = edgedb.connect("edgedb://")

... read dbschema/local-credentials.json

Notes for language bindings:

1. If any of EDGEDB_HOST and EDGEDB_PORT present, we don't read the file
2. File is read from workdir. It's possible that there is no workdir or no
   dbschema folder on production development. This is a good thing, as it
   will error out quickly.
3. There is a chance that dbschema/local-credentials.json leaks to git, this is only issue for cloud instances, and the risk is pretty minimal as we add the file to .gitignore automatically whenever possible.

Expansion of that is along the lines of:

if 'EDGEDB_HOST' in os.environ or 'EDGEDB_PORT' in os.environ:
    config = Instance(
        host=os.environ.get('EDGEDB_HOST', 'localhost'),
        port=int(os.environ.get('EDGEDB_PORT')),
    )
else:
    config = Instance.from_credentials_file(
        './dbschema/local-credentials.json'
    )
if 'EDGEDB_DATABASE' in os.environ:
    config.set_database(os.environ['EDGEDB_DATABASE'])

Alternatives

Using .env

Store credentials in .env file in the current directory instead of JSON
file.

Pros:

1. Don't need to read another file dbschema/local-credentials.json
2. Might be easier to use with docker-compose

Cons:

1. Format of .env files is a little bit different (e.g. variable expansion)
2. For some apps (e.g. next.js and create-react-app) our credentials should be
   written to .env.local or .env.development, but it may be hard to guess
3. We have to guess whether the .env file already applied (e.g. in
   docker-compose, or using next.js framework) or not (e.g. in bare python/js
   app)
4. .env file with credentials can be committed in the repo if .env.local layering is
   used
5. Other recommended workflow of committing .env.sample doesn't fit well
   with the tool either (i.e. the file must be copied strictly before the
   initialization of the database)

Use dbschema/instance_name.txt

Use a text file with instance name (that refers to
~/.edgedb/credentials/<instance_name>.json instead of full credentials file).

Or we could use dbschema/local-instance.json with contents of:

{
    "instance": "myapp",
    "database": "myapp_db",
}

(to allow overriding database)

Pros:

1. No duplication of credentials in home and project

Cons:

1. Using docker for app is harder, since home is usually not
   mounted inside the container
2. More files to read by each language binding (i.e. more failure modes)
3. Another file format and API to document, learn, implement in every binding
   (Current proposal uses
   Instance.from_credentials_file('dbschema/local-credentials.json') API)

init-project command

edgedb init

Pros:

1. Shorter

Cons:

1. Too much like git init, which initialize empty repository, our most
   common case is to initialize instance after git clone
2. May be messed up with edgedb server init (although latter requries
   instance name so this is not a big issue
3. Also other things like upgrade-project and get-project-info in unclear how to name

edgedb start-project

Cons:

1. Still not clear if thats for new project or existing one

edgedb init / edgedb start

Cons:

1. To hard to remember two commands
2. I have hard time to guess which one is start and which one is init
3. This initialisation is run by curl | sh script so must be a single command

Toml Alternatives

The issue with toml is that it doesn't scale very well with more complex
structures and long strings.

It's unlikely that we need these features in this specific config, but using
single format for all configs is preferred. For main edgedb cli's config we
may need complex stuff.

Alternatives (in the order of my personal preference):

• hjson -- superset of JSON, in particular with comments and multiline strings.
  Upside is that it's easy to generate by any script or tool. Although, not
  very popular so far.
• YAML -- probably viable, may be harder to update programmatically as there
  are much more ways to format files
• cjson -- underspecified JSON with comments
• JSON -- isn't great config format (particularly because no comments)

Note: we don't need to read project.toml by the application so are not very
concerned to have implementation in all languages.

Open Questions

Docker Support

The problem with docker support for apps is that while
dbschema/local-credentials.json can be read, the host name has to be
overriden. And when user supplies EDGEDB_HOSTNAME the hostname the files is
not read any more (unless you use --net=host which is uncommon).

So docker requires some special handling. See "Docker Compose" for more info.

This is not a concern if edgedb is used with --method=docker but app is
running locally.

Docker Compose

If docker-compose.yml is detected, it might be useful to do the following
instead of default behavior of edgedb init-project:

1. Generate credentials into .env

2. Suggest to add edgedb to docker-compose.yml or even modify the file.
   There are two options:

   a. [2a] Add image: edgedb/edgedb kind of service
   b. [2b] Start normal instance. And add
   external_links: [edgedb_instance_name:edgedb]

The consequence of [2b] is that instance name has to be updated in
docker-compose.yml file if user uses instance name other than default.
And also that docker-compose up doesn't do its job well (you have to start
edgedb before starting app)

Note there are two problems with not implementing docker-compose detection:

1. Local (not --method=docker) instances are unavailable for docker
2. For [2b] we have to detect and add --method=docker
3. For [2a] we have to supply EDGEDB_PASSWORD to the EdgeDB's docker
   container too so that the password is set for the role (it's unlikely that
   we want EdgeDB server container to detect local-credentials.json by
   itself)
4. For both options we can't use dbschema/local-credentials.json for the
   client as we have to override hostname (and that stops reading the JSON by
   default)

Pros of special-casing docker compose:

1. Helpful for docker compose users
2. The actual docker-compose.yml piece might be more standard if we provide
   it (especially in [2a])
3. edgedb upgrade-project might actually work for docker compose with this
   scenario (edgedb server upgrade won't for [2a]), but requires a special
   case in this specific command too

Cons:

1. Some users might have docker-compose.yml but prefer local installations anyway
2. More encouraging to edgedb in compose file (which we try to avoid)
3. Harder to modify docker-compose.yml automatically

[elprans]

Great start, @tailhook. Thanks!

Comments:

Note: edgedb init-project is also run as a part of curl | sh script.

Can you elaborate what this means, please? We probably wouldn't be running it unconditionally. Also, this implies that edgedb init-project runs edgedb server install, which isn't a bad idea, but we need to clarify that. Oh, and we should also make sure edgedb server init does that also. It's probably sufficient to just modify edgedb server init to download and install the necessary server version automatically.

Enter instance name (type cloud to connect to EdgeDB cloud):

This would be nicer as a choice option instead, e.g.:

1. Use a local instance
2. Use a cloud instance
Your choice> 2

Bare edgedb looks for dbschema/local-credentials.json and
fails if there is no one.

Hm. I feel uneasy about local-credentials.json. The proposal makes the case that this is necessary for Docker, but then there's a whole bunch of additional issues there as indicated later. TBH, I'd prefer the dbschema/local-instance.json approach.

Local (not --method=docker) instances are unavailable for docker

It depends. If local instances listen on 0.0.0.0 or docker bridge gateway, they would be available. Also, if you use host networking there's no problem. We can manipulate both the host instance and the credentials used in docker.

I think we should avoid trying to manipulate docker-compose.yml or do anything implicitly and instead rely on messages and documentation. We can still detect docker-compose and suggest what to do.

[tailhook]

Great start, @tailhook. Thanks!

Comments:

Note: edgedb init-project is also run as a part of curl | sh script.

Can you elaborate what this means, please? We probably wouldn't be running it unconditionally.

As you can see in examples, init-project is always interactive. So it either asks whether to initialize a new project, or asks instance name. So user has a chance to refuse anyway. We can add press Ctrl+C to cancel or something similar if it will be not clear enough.

Also, this implies that edgedb init-project runs edgedb server install, which isn't a bad idea, but we need to clarify that. Oh, and we should also make sure edgedb server init does that also. It's probably sufficient to just modify edgedb server init to download and install the necessary server version automatically.

init-project -- definitely yes. This is why there is a version in project.toml. I'm not sure about server init, for the same reason I've described elsewhere: server install needs sudo while on init it's a bit unexpected. So I'm not sure about this one.

Enter instance name (type cloud to connect to EdgeDB cloud):

This would be nicer as a choice option instead, e.g.:

1. Use a local instance
2. Use a cloud instance
Your choice> 2

Well, extra steps aren't great. And typing text is what every CLI tool user used to. So it looks like the matter of preference.

Bare edgedb looks for dbschema/local-credentials.json and
fails if there is no one.

Hm. I feel uneasy about local-credentials.json. The proposal makes the case that this is necessary for Docker, but then there's a whole bunch of additional issues there as indicated later. TBH, I'd prefer the dbschema/local-instance.json approach.

Well, yes. I've written that before diving too much into the docker issues. So on the surface it's only useful with --net=host, but I'll post another idea in the separate thread.

Local (not --method=docker) instances are unavailable for docker

It depends. If local instances listen on 0.0.0.0 or docker bridge gateway, they would be available. Also, if you use host networking there's no problem. We can manipulate both the host instance and the credentials used in docker.

We listen on 0.0.0.0 on local instances only in docker. The reason for that is that listening on 0.0.0.0 exposes your laptop to the local network. This is an issue in the normal setup. But is even bigger issue if someone would run edgedb server init on a server

I think we should avoid trying to manipulate docker-compose.yml or do anything implicitly and instead rely on messages and documentation. We can still detect docker-compose and suggest what to do.

We'll surely ask user before doing, and detect if we can't manipulate reliably (i.e. if yaml has weird formatting) and show example instead. But I think a lot of users like when things are added automatically (we try to do that for PATH for example, even though adding that is non-trivial with all different shells and turing-complete configs).

[elprans]

init-project -- definitely yes. This is why there is a version in project.toml. I'm not sure about server init, for the same reason I've described elsewhere: server install needs sudo while on init it's a bit unexpected. So I'm not sure about this one.

But then init-project and sh | curl would require sudo as well. Adding init-project to the initial installation script adds a whole lot more failure modes and it would be unclear for the users what succeeded and what didn't, at what point should they restart the process and how and so on. As a point, rustup doesn't run cargo new.

Well, extra steps aren't great. And typing text is what every CLI tool user used to. So it looks like the matter of preference.

Well, we already offer the multiple choice experience elsewhere consistently. Also, what if I want to call my local instance "cloud"?

The reason for that is that listening on 0.0.0.0 exposes your laptop to the local network.

You don't have to listen on 0.0.0.0 to make it available for docker, listening on the docker bridge address (172.17.0.1) is sufficient.

[tailhook]

init-project -- definitely yes. This is why there is a version in project.toml. I'm not sure about server init, for the same reason I've described elsewhere: server install needs sudo while on init it's a bit unexpected. So I'm not sure about this one.

But then init-project and sh | curl would require sudo as well. Adding init-project to the initial installation script adds a whole lot more failure modes and it would be unclear for the users what succeeded and what didn't, at what point should they restart the process and how and so on. As a point, rustup doesn't run cargo new.

Hm, I think you suggested to run server init on curl | sh. No?

For me part of init-project's motivation has always been running it as a part of curl | sh.

I don't think that "failure modes" are very big problem here. Generally repeating curl | sh is equivalent to edgedb self-upgrade which is a good thing to repeat, so it's a bit slower, but is not an issue. Also we may try to make UI clear enough for experienced users.

Well, extra steps aren't great. And typing text is what every CLI tool user used to. So it looks like the matter of preference.

Well, we already offer the multiple choice experience elsewhere consistently.

Well, we can't ask for a name with "multiple choice", so asking for name will be here at some stage anyway. So I don't think that adding extra steps is "consistent" in any way. Also currently we have no cloud, so we have to design this in a way that does allow us to add cloud later without rewriting all the tutorials (yes, detailed tutorial can mention each step, instead of writing bare "follow the prompts").

Also, what if I want to call my local instance "cloud"?

So we reserve one specific name. Choose another, what is the issue? If we don't do this right from the start it would be a problem. Therefore, we might also want to reserve for future something like remote which will ask for credentials. Other than that I don't think there is a shortage of names. We have much more reserved keywords in every programming language.

The reason for that is that listening on 0.0.0.0 exposes your laptop to the local network.

You don't have to listen on 0.0.0.0 to make it available for docker, listening on the docker bridge address (172.17.0.1) is sufficient.

We have to detect that. And we have to rewrite service file once bridge is up. And nobody guarantees that docker is up after reboot even if it is currently up. So edgedb unit having --listen-address 172.17.0.1 may not start after reboot.

So it either has to be done magically in EdgeDB (and I don't like magic). Or it has to be done in every (re)start , and that restart has to be run manually from time to time.

And then another issue if somebody made Wi-fi on 172.17.0.1 (Don't ask why).

[elprans]

Hm, I think you suggested to run server init on curl | sh. No?

I was suggesting server install, not init.

For me part of init-project's motivation has always been running it as a part of curl | sh.

What about other means of installing the CLI (homebrew, winget, apt)? Can you point to some examples where tools ask or attempt to "init-project" during self-installation?

[tailhook]

Follow-up on docker(compose) issue.

The problem is that if we write hostname dbschema/local-instance.json it's hard to override the hostname and the hostname is different if used from host system (in this case it's localhost) for CLI tool and if used in docker (in this case it's defined by docker link, e.g. edgedb).

But we can fix this by one of the two ways:

1. Omit host in credentials file (our schema allows that). And in this case, detect if client/tool is running within a docker container if it is, use edgedb hostname by default, otherwise use localhost hostname by default.

2. Use some special name like host: "local-instance.edgedb.com". This domain name will resolve to 127.0.0.1 and/or ::1, but in docker-compose.yml (or with --link of docker container) we can use links: [edgedb:local-instance.edgedb.com]. Docker will eagerly override the hostname resolution for you and connect to specified container.

The downside of (1) is that it might look a bit magical. And also detecting docker container has to be embedded in every client (currently it's just reading /proc/self/cgroup, but it may change with cgroup namespaces which are already supported in new kernels just not activated in docker by default I think). And then ocasionally this detection has to be overloaded for (our) tests or if app or cli runs in the same container.

Another note on (1) if that it might be tempting to just try hosts: ["localhost", "edgedb"]. But connecting to short names should be avoided if not explicitly asked for, because it may redirect to edgedb.some.corp.domain (e.g. if search some.corp.domain is set in /etc/resolv.conf) or even leak data to the guest wifi because search is populated from DHCP.

The downside of (2) is that to work on airplane you have to enable resolving local-instance.edgedb.com locally (e.g. by writing it into /etc/hosts file).

You can try (2) by using edgedb.devd.io domain for example (devd.io resolves all subdomains to localhost).

So for now (2) is my favorite. It seamlessly works with and without docker-compose without messing up with .env files. And we will be able to implement edgedb upgrade-project for docker compose (although latter would require modification of the docker-compose.yml to upgrade image version).

[elprans]

Perhaps we should avoid magic and be explicit: specify two hosts in the credentials file, e.g:

"host": {
   "local": "127.0.0.1",
   "docker": "edgedb"
}

[tailhook]

Looks like an option. Still not sure if we want to embed docker container detection code in all language bindings, though.

[tailhook]

Looking at geldata/gel-cli#173, we might want to add the following:

1. Support schema-dir = configuration parameter
2. To make (1) useful we have to read edgedb.toml file instead and fall back to dbschema/project.toml or dbschema/edgedb.toml (to have less clutter in the project root)
3. The (1) will also create an issue with local-credentials.json, so it also has to have a ./.local-credentials.json or .edgedb/local-credentials.json fallback.

These features can be added later backwards compatibly, but maybe we will need less fallback paths if we do that now.

[elprans]

Let's worry about schema-dir configurability later. It's not necessary for the initial version of init-project.

[tailhook]

And another interesting question: what if my project dir is ~/proj and current directory is in ~/proj/subdir/subdir2. Should we search ancestor directories?

• ~/proj/subdir/subdir2/dbschema/project.toml
• ~/proj/subdir/dbschema/project.toml
• ~/proj/dbschema/project.toml <-- found here
• ~/dbschema/project.toml

I think we should be fine without this. But it can potentially make some use cases easier.

[elprans]

I think we should do the traversal, at least in the CLI.

[jaclarke]

Pros:

1. No duplication of credentials in home and project

I'm assuming this means that a credentials file is always created in the .edgedb/credentials dir, and that the instance created for the project is known about 'globally'? (so edgedb studio would still see the instance even though it's not running the cli command from that project's directory)

[elprans]

That's right.

[tailhook]

Yes. local-credentials.json is a copy of that credentials by this spec.

[1st1]

Few comments:

• Note: edgedb init-project is also run as a part of curl | sh script.

I don't think this is a good idea, as the current working dir might not be a suitable one for the new project. I do think we need to mention it when the script is done.

project.toml

I think we should use JSON. We only have a couple of settings now, and we might have 2-3 more in the future. But even if we have 10 settings there, I don't think that a lot of people would need to put comments there; setting names are pretty self-explanatory.

If any of EDGEDB_HOST and EDGEDB_PORT present, we don't read the file

I think we should also add EDGEDB_INSTANCE and check for it first. IOW +1 to what @elprans said in #2396 (comment).

Saving dbschema/local-credentials.json

Lastly, I'm not sure if this file is absolutely necessary for the initial implementation. Is it?

[tailhook]

Few comments:

• Note: edgedb init-project is also run as a part of curl | sh script.

I don't think this is a good idea, as the current working dir might not be a suitable one for the new project. I do think we need to mention it when the script is done.

This was important part of the proposal, as you need source ../env to actually run edgedb init-project or whatever.

Also we explicitly ask if current whether you want to create a new project, if you run curl script in directory having no config.

The idea is that putting curl .. | sh in tutorial of your project is enough to get project up and running. Otherwise you have to instruct to do source ../env which doesn't work for some shells (fish? powershell?)

project.toml

I think we should use JSON. We only have a couple of settings now, and we might have 2-3 more in the future. But even if we have 10 settings there, I don't think that a lot of people would need to put comments there; setting names are pretty self-explanatory.

Comments aren't for explaining configuration variables, comments are like:

# Production is still beta.1
edgedb-version = "1-beta.1"
# Uncomment this to test on beta.2 locally, will go production at 15.04.2020
# edgedb-version = "1-beta.2"

Saving dbschema/local-credentials.json

Lastly, I'm not sure if this file is absolutely necessary for the initial implementation. Is it?

We either need credentials file or file stating the name of the instance (or env file if we go with that). Latter does not work with docker-compose. Although former needs some hostname quircks to work.

[tailhook]

Another question is, we likely want to record links from ~/.edgedb to projects. This should allow us to have "clean unused instances" command in future or at least show directory name in edgedb server status. And this should also help studio, e.g. if you connect to named instance, you can quickly find and edit project's schema.

Probably this should be ~/.edgedb/projects/<instance_name> which is a symlink to a project root.

What do you think?

[elprans]

This could be convenient, but this sort of linking tends to be fragile, since I can move/rename my project folders, which would make the "clean unused instances" thing especially dangerous. For schema editing/viewing in Studio we shall instruct users to open the project directory instead.

[tailhook]

I think move/rename is rare enough. And we're surely going to ask a confirmation. And maybe show warning and suggest updating the link on the next run of any edgedb command that reads project credentials.

And for studio, well, I think picking a name will always be a little bit faster than using directory browser. So users will select named instance first and then will probably see file browser when trying to actually open the schema file. So this will be a two step process in most cases.

So this is a time where we want to optimize for the common case instead of deoptimizing for the sake of general case (renaming the project directory) which almost never happen.

[elprans]

We had an offline discussion about this today, here's a summary:

1. We shouldn't concern ourselves about automatic support for docker-compose scenarios. Those are inherently messy and project-specific. Instead, we will document the best practices of using EdgeDB with docker-compose and, where possible, implement detection of incorrect use of edgedb server and edgedb project-init workflows in tandem with docker-compose.

2. There is a new implementation idea proposed by Yury: assign unique ids to every project and use them instead of an instance name. See below for details.

Approach 1: Unique Project ID

Scenarios:

1. Brand new project

$ edgedb init-project
There is no `dbschema` directory in `/home/user/work/myapp`.
Do you want to initialize a new project?
> y
Project name (<default to directory name>):
> myapp
Do you want to use a local EdgeDB instance or a cloud instance:
1. Local
2. Cloud
> 1
Creating EdgeDB instance `myapp`...

This will:

• create dbschema/project.json containing the project name and a generated project UUID
• generate an instance name based on project name; if instance name is conflicting, ask a user to provide an alternative
• create an instance with edgedb server init
• create an alias to an instance using project UUID: ln -s ~/.edgedb/credentials/myapp.json ~/.edgedb/credentials/<uuid>.json
• note the project directory in a well-known location: ln -s <project-dir> ~/.edgedb/projects/<uuid>

Language bindings will then look at dbschema/project.json, and use the UUID as the instance name.

2. Existing project, newly git-cloned

$ git clone https://github.com/myorg/myapp
$ cd myapp
$ edgedb init-project
There is no EdgeDB instance for project "myapp", do you want to create one?
> y
Do you want to use a local EdgeDB instance or a cloud instance:
1. Local
2. Cloud
> 1
Creating EdgeDB instance `myapp`...

This will:

• generate an instance name based on project name; if instance name is conflicting, ask a user to provide an alternative
• create an instance with edgedb server init
• create an alias to an instance using project UUID: ln -s ~/.edgedb/credentials/myapp.json ~/.edgedb/credentials/<uuid>.json
• note the project directory in a well-known location: ln -s <project-dir> ~/.edgedb/projects/<uuid>
• optionally run migrations from dbschema/migrations

3. A copy or a fork of an existing project:

$ git clone myapp myapp-copy
$ cd myapp-copy
$ edgedb init-project
An EdgeDB instance for project "myapp" already exists, but has been initialized for a different project directory: ~/.../myapp.
Do you want to create another instance or update the project directory to point to `myapp-copy`?
1. Create another instance
2. Repoint project directory
> 1
Specify instance name (<default to directory name>)
> myapp-copy
Creating EdgeDB instance `myapp-copy`...
Writing project ID alias to `dbschema/project.local.json`

This will:

• write a new project UUID to dbschema/project.local.json
• optionally add dbschema/project.local.json to .gitignore/.hgignore
• create an instance with edgedb server init
• create an alias to an instance using project UUID from project.local.json: ln -s ~/.edgedb/credentials/myapp-copy.json ~/.edgedb/credentials/<uuid>.json
• note the project directory in a well-known location: ln -s <project-dir> ~/.edgedb/projects/<uuid>
• optionally run migrations from dbschema/migrations

Advantages

1. No local VCS-ignored files are needed by default.
2. Unique project identification prevents accidental mixups of similarly-named projects.
3. Recording symlinks to projects makes it possible to easily distinguish between project copies.

Disadvantages

All language bindings need to learn to read and parse dbschema/project.json and dbschema/project.local.json. Since bindings are already required to read credentials.json, the dependency on local file access and JSON serialization already exists, so this isn't very burdensome.

Approach 2: Copy of credentials in dbschema/

This is discussed above, so I'm only summarizing the pros and cons.

Advantages

1. No extra indirection in bindings (although changes are obviously still necessary to recognize dbschema/local_credentials.json)

Disadvantages

1. Local credentials MUST NOT be committed, and even though we can generate ignore files for common VCS, the solution is not bullet-proof.
2. It is very easy to accidentally delete project configration which is ignored (e.g. by running git clean). Since dbschema/local_credentials.json is auto-generated, a user would not be immediately aware of the danger and will face confusing errors when attempting to use EdgeDB on the project post-clean. Admittedly, this issue is present with Approach 1 too, but only in presence of multiple project copies on the same machine, not in the general case.

[1st1]

To add to the project ID advantages: having a unique ID can later, potentially, enable us to use it in analytics or for future cloud ops etc.

[elprans]

When the project is forked

We need to distinguish between a) a fork, where you could just change the ID, and b) a temporary local copy of the same project, which still points to the same upstream git repo, where you can't change the ID without breaking everyone's checkouts.

[tailhook]

Just few nits for now:

We shouldn't concern ourselves about automatic support for docker-compose scenarios. Those are inherently messy

We decided a different approach for docker compose because there is a way to do that, not because it's inherently messy.

ln -s ~/.edgedb/credentials/myapp.json ~/.edgedb/credentials/.json

It is bad to keep uuids and names in the same directory.

To add to the project ID advantages: having a unique ID can later, potentially, enable us to use it in analytics or for future cloud ops etc.

This is a very thin ice

a) a fork, where you could just change the ID, and b) a temporary local copy of the same

Yes. So what you described in the start of this thread is a copy of the project. Not fork. To distinguish a fork we have to introduce another edgedb command.

[elprans]

It is bad to keep uuids and names in the same directory.

Perhaps this makes it less convenient to enumerate instances. I don't see other problems. Anyway, it's a minor point, we can have a by-id directory with symlinks instead.

To distinguish a fork we have to introduce another edgedb command.

Or rm dbschema/project.json && edgedb init-project

[tailhook]

To distinguish a fork we have to introduce another edgedb command.

Or rm dbschema/project.json && edgedb init-project

Until we have more things in the project.json, like a list of extensions or something else. But this is relatively minor.

The problem is that we can't teach it. It will never be in any tutorial (because it is too early). But nobody looks a the docs when forking the project.