[feat] Add microgenerator support to synthtool.

[lukesneeringer]

This commit adds microgenerator support to synthtool, and defines shortcut methods for the three languages where the ACT team currently publishes Docker images.

A couple of things were not entirely clear to me when I wrote this, so I am looking for some guidance:

• What is the appropriate testing regimen? I did not see any existing unit tests that seemed to exercise the GAPICGenerator or Artman classes, so I did not attempt to write any for mine either.
• It is not clear to me how you want to handle output directories. This might be because artman largely took the option out of your hands? It seems to me that the right thing to do is to make a temporary directory, generate code there, and copy out of it, but currently to do that correctly, it probably requires that the individual synth.py files handle the temporary directory context manager logic.

[lukesneeringer]

@crwilcox I think it is pretty likely that this is the wrong thing to do, but it is not obvious to me what a better option is.

Here are a couple potential possibilities:

• Return a TemporaryDirectory object instead of a Path.
  • Pros: Easy for synth.py files; they just use the context manager interface to copy out of the temporary directory and cleanup magically happens.
  • Cons: Violates / changes the interface observed by GAPICGenerator.
• Require an output directory.
  • Pros: Unambiguous. If you create a temporary directory to handle this, you clean it up.
  • Cons: Slightly more code than the option above (~1-2 more lines in each synth.py file). Slightly violates the interface by requiring output_dir.
• What is here now.
  • Pros: Holds to the existing interface.
  • Cons: Very easy to leave temporary directories lying around.
• Try to return a frankenstein object that subclasses both Path and TemporaryDirectory. (In this case, I would remove output_dir entirely and only write to temp directories.)
  • Pros: Holds to the existing interface.
  • Cons: Seems dirty. Edge cases intuitively seem likely.

[busunkim96]

I am OK with the approach here. Synthtool already leaves behind generated libraries in ~/.cache/synthtool/googleapis/artman-genfiles/.

In fact, we have a note in the README that recommends looking at that directory when you're doing a generation.

SynthTool will run Artman which will create generated code that can be found at ~/.cache/synthtool/googleapis<-private>/artman_genfiles. This is useful for figuring out what it is you need to copy for your specific library.

Would it be reasonable to have the microgenerator default to a similar location for predictability?

For reference:
Artman's output_dir is root_dir / "artman-genfiles"

synthtool/synthtool/gcp/artman.py

Lines 34 to 43 in 6f7311c

	def run(self, image, root_dir, config, *args):
	"""Executes artman command in the artman container.
	Args:
	root_dir: The input directory that will be mounted to artman docker
	container as local googleapis directory.
	Returns:
	The output directory with artman-generated files.
	"""
	container_name = "artman-docker"
	output_dir = root_dir / "artman-genfiles"

root_dir is the path for googleapis.

synthtool/synthtool/gcp/gapic_generator.py

Lines 108 to 112 in 6f7311c

	output_root = artman.Artman().run(
	f"googleapis/artman:{artman.ARTMAN_VERSION}",
	googleapis,
	config_path,
	gapic_language_arg,

And googleapis is cloned to a cache dir, by default.

synthtool/synthtool/sources/git.py

Lines 47 to 55 in 6f7311c

	def clone(
	url: str,
	dest: pathlib.Path = None,
	committish: str = "master",
	force: bool = False,
	depth: int = None,
	) -> pathlib.Path:
	if dest is None:
	dest = cache.get_cache_dir()

synthtool/synthtool/cache.py

Line 19 in 6f7311c

cache_dir = pathlib.Path.home() / ".cache" / "synthtool"

[lukesneeringer]

Thanks @busunkim96 for looking at this!

The new generators are protoc plugins, which require a specified directory to output to (that is the fundamental protoc plugin interface), so I can not have the generator itself default to any location.

However, I can have synthtool pick a default location (e.g. python-gapic-genfiles) and behave this way if that is your preference. That seems odd to me, but you are the end user, and I defer to your preference. :)

Should I make that change? Happy to do so.

[busunkim96]

@lukesneeringer Yes please! I think it would be the least surprising for the microgenerator to also put files in the synthtool cache dir. Thank-you. 😄

[busunkim96]

If folks want the library to be in some other directory, that can be changed in a future PR.