Represent generated subtarget addresses as file names

[Eric-Arellano]

Before:

▶ ./pants dependencies src/python/pants/engine/internals:native
3rdparty/python:typing-extensions
src/python/pants/base:exiter.py
src/python/pants/engine/internals:__init__.py
src/python/pants/engine/internals:native_engine
src/python/pants/engine:fs.py
src/python/pants/engine:selectors.py
src/python/pants/engine:unions.py
src/python/pants/util:memo.py
src/python/pants/util:meta.py

After:

▶ ./pants dependencies src/python/pants/engine/internals:native
3rdparty/python:typing-extensions
src/python/pants/base/exiter.py
src/python/pants/engine/fs.py
src/python/pants/engine/internals/__init__.py
src/python/pants/engine/internals:native_engine
src/python/pants/engine/selectors.py
src/python/pants/engine/unions.py
src/python/pants/util/memo.py
src/python/pants/util/meta.py

This change solves several key questions with file args, as explained in https://docs.google.com/document/d/14MAst0Q0HpoLqysiN7KoKDkA6-Ga_KhjICTl0EC16uI/edit#heading=h.y495n7exlx1x:

1. Alignment with file arguments, which will use literal file names and will always result in generated subtargets being used.
2. A syntax for explicitly declaring generated subtarget dependencies in BUILD files.
   • Without this change, there is ambiguity between whether you intended :example.txt to mean a literal target or a generated subtarget.
3. Thanks to pants python handling moves to goals #2, a way to explicitly ignore false positives via ! in the dependencies field of BUILD files.

For alignment with target addresses and the sources field, we will plan to allow relative references to generated subtargets. foo.py will be relative to the BUILD file, and you need //foo.py to mean relative to the build root. This does not work with subdirectories. subdir/a.py is relative to the build root.

Multiple owners

If there are multiple owners of a target, and we generate a subtarget over the same file for each owner, then the resulting addresses would all "look" the same to the user because they would have the same Address.spec. This is confusing to the user.

For now, we defer this problem by never using generated subtargets when there are multiple owners. In the feature, we could add a syntax to refer to the original owner when there's ambiguity, e.g. helloworld/app.py@:original_target. This is safe because of the following semantics with multiple owners:

• dep inference -> no-op
• explicit BUILD files -> error
• CLI file args -> don't generate subtargets, and instead use all the original owners.

[ci skip-rust-tests]

[stuhood]

Thanks!

For alignment with target addresses and the sources field, we will plan to allow relative references to generated subtargets. foo.py will be relative to the BUILD file, and you need //foo.py to mean relative to the build root. This does not work with subdirectories. subdir/a.py is relative to the build root.

As it stands, the only way to get a relative address is by prefixing with :target_name, so depending on how the "ambiguous owners" syntax shakes out, it might be more consistent to have a relative file be :file.py. Unclear.

[stuhood]

We should maybe open a followup ticket with a short blurb describing the open question here (and referencing the doc for more info).

[Eric-Arellano]

I don't think this is necessary. I'm actively iterating on this part of the codebase the next day or two. It's possible that we adapt Address.parse() to be able to parse file names, given a hint for generated_base_target_name. But it's also possible that we want that logic to live somewhere else, like the dependency inference rule. Unclear, but will be answered very soon.

[stuhood]

Should probably remove the return value from this method too.

[Eric-Arellano]

Ah, I meant to do that. Thanks!

[stuhood]

Should reference the ticket for the ambiguity syntax here as well probably.

[Eric-Arellano]

it might be more consistent to have a relative file be :file.py

Agreed, but this would also introduce a new ambiguity between target addresses vs. generated subtargets. It's possible to have a traditional file named :foo.py.

I wonder if there's another symbol we could use? I was thinking ., but that doesn't work because you could have hidden files.

[stuhood]

I wonder if there's another symbol we could use? I was thinking ., but that doesn't work because you could have hidden files.

Generally that's addressed via ./$something. But yea, designing some syntax holistically is going to be important, although deferring that as we shake out the other issues makes sense.