chore(internal/kokoro): port check_incompat_changes

[noahdietz]

Ports the apidiff logic of check_incompat_changes.sh into a Go program internal/apidiff.

This new version reads from the internal/.repo-metadata-full.json file and runs apidiff against all packages labeled as ga.

When it encounters a manual client or a generated subclient of a submodule, it attempts to find the submodule, change into that directory, and run apidiff from there.

It will collect all errors/diffs and report them at the end. The script will only fail if there are diffs. If there is an error during apidiff of one of the packages, it will be reported, but it will not fail the script.

Fixes #3764 by parsing the repo-metadata, not a regexp in a bash script.

[noahdietz]

Currently failing because apidiff is complaining about a constant that doesn't actually change in pubsub

cloud.google.com/go/pubsub:
- MaxPublishRequestBytes: value changed from 0.000582077 to 10000000

[codyoss]

Mostly nits, overall LGTM.

[noahdietz]

@codyoss I could move this check into shouldRun() if you want to get rid of this script altogether. WDYT?

[noahdietz]

Blocked by golang/go#44796

[noahdietz]

Oops, I messed up my git history some how and ive got waaaay too many commits that don't belong. Ignore this please!

[noahdietz]

@tbpg @codyoss can I get another pass on this? I am manually ignoring the incorrect pubsub error (if it is the only error there is for that pkg)

[tbpg]

Sounds good to move this to a Go program, but I'm having trouble following the flow of execution. I left some comments about the APIs. Overall, it feels more... complicated (?) than I expected. Hopefully the suggestions I gave help simplify the flow.

[noahdietz]

Overall, it feels more... complicated (?) than I expected

Yeah that's reasonable - it is more complicated. This now:

• supports diffing submodules, which the previous script didn't
• reads a config file instead of iterating over a hard coded list
• collects all diffs/errors before reporting instead of exiting at first issue

So it's not a direct port, it's a "port+" :) Your suggestions are totally valid though.

[tbpg]

Much more understandable -- thank you!

[noahdietz]

@codyoss I think you need to approve since you requested changes.

[codyoss]

@noahdietz Sorry about that, did not realize this was waiting on me.

[noahdietz]

Only waiting like 1 min :)