Fix API docs and benchmark dashboards

[echeran]

Main fixes to get benchmark dashboards to generate correctly and coexist API docs was tested in echeran#10.

There were a couple of problems:

• somehow, the path of the benchmark output files dropped a path segment compared to what I thought I had made it
• even though the benchmark action says to use a cache action, that doesn't work for us (I'm confused how that would make sense for anyone, really)
• we still have to store the commits in a local branch for storing history
• we should use the upload & download actions (instead of caching) to persist the benchmark dashboard output files across jobs within a workflow, collect them together with the API docs, and copy them at the same time to our docs repo

A second test PR was create to test the split nature behavior for PRs -- we want unmerged PRs to not overwrite our dashboards and API docs, and instead only allow PR merges to master to update those things: echeran#12. As verification:

• before the PR was merged, the correct jobs were run and skipped
• after the PR was merged, the correct jobs were run and skipped. The final step in the post-merge workflow instance only had an issue because I had already changed the configuration for the destination docs repo from echeran/icu4x-docs to unicode-org/icu4x-docs.

I have one benchmark dashboard and API docs. All the dashboards behave independently, now, so if one works, they all should work.

I looked at the link from @zbraniecki to the repo using Rust with benchmark dashboards. There were a couple of things that got in the way of adapting their setup for our repo:

1. they seem to run their benchmarks all at once. By contrast, we want to run our benchmarks on a per-component basis, so that we allow the possibility for meaningful alerts that are based on per-component comparisons, not global change comparisons.
2. they made their own GH action for doing performance comparisons, but it didn't work for me

[echeran]

Because we want to keep our canonical historical benchmark data (output from benchmark runs on commits in master) separate from other benchmark data (output from benchmark runs on unfinished/WIP PRs), the workflow uses the gh-pages and unmerged-pr-bench-data branches to store those 2 categories of benchmark data. It requires a one-time creation of those branches in the repo.

I forgot to create the latter branch in our upstream repo, but I just now did that using these commands and re-started the workflow:

git checkout --orphan unmerged-pr-bench-data
git commit --allow-empty -m "root commit"
git push upstream unmerged-pr-bench-data:unmerged-pr-bench-data

[filmil]

I for one welcome our new dashboards. Thank you!

…

On Fri, Oct 9, 2020 at 9:13 AM Elango ***@***.***> wrote: Because we want to keep our canonical historical benchmark data (output from benchmark runs on commits in master) separate from other benchmark data (output from benchmark runs on unfinished/WIP PRs), the workflow uses the gh-pages and unmerged-pr-bench-data branches to store those 2 categories of benchmark data. It requires a one-time creation of those branches in the repo. I forgot to create the latter branch in our upstream repo, but I just now did that using these commands and re-started the workflow: git checkout --orphan unmerged-pr-bench-data git commit --allow-empty -m "root commit" git push upstream unmerged-pr-bench-data:unmerged-pr-bench-data — You are receiving this because your review was requested. Reply to this email directly, view it on GitHub <#314 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB4GMBVI7XH6QMFTN3AGWLSJ4ZCNANCNFSM4SJXSW4A> .

[zbraniecki]

lgtm!