Reduce the amount of time it takes to analyze https://github.com/corgibytes/freshli-fixture-java-test

[edwinkortman]

In this PR: #276
Running the following test takes 1m52s to complete successfully.
Investigate why it's this slow.

Scenario: Run the analysis, start with 6 workers.
        Given the Freshli Web API is available
        And a directory named "~/.freshli" does not exist
        When I run `freshli --loglevel=Debug analyze --workers=6 https://github.com/corgibytes/freshli-fixture-java-test`
        Then it should pass with:
        """
        https://freshli.app/
        """
        And the directory named "~/.freshli" should exist
        And a file named "~/.freshli/freshli.db" should exist
        And we can open a SQLite connection to "~/.freshli/freshli.db"
        And a Git repository exists at "~/.freshli/repositories/08e8926bfb81cd10b2d0584f025da4f1b81788504c5f0ca0e1b8c9d0de7f26e5" with a Git SHA "7601fe07ea76d9ce8c9d5332db237d71e236ef4a" checked out
        And a directory named "~/.freshli/histories" exists
        And a directory named "~/.freshli/histories/08e8926bfb81cd10b2d0584f025da4f1b81788504c5f0ca0e1b8c9d0de7f26e5" is not empty
        And a file named "~/.freshli/histories/08e8926bfb81cd10b2d0584f025da4f1b81788504c5f0ca0e1b8c9d0de7f26e5/f58c3f8773da4ea4f01d819b842e384b3a343d40/archive.zip" does not exist

[mscottford]

This effort may be helped by getting the freshli-agent-java executable to cache http requests (per #340, corgibytes/freshli-agent-java#34, and corgibytes/dependency-history-maven#24).

[mscottford]

Changing this to an epic and then creating some specific issues that we can address. I decided to create a separate epic and then add this issue to that one.

[mscottford]

This effort may be helped by getting the freshli-agent-java executable to cache http requests (per #340, corgibytes/freshli-agent-java#34, and corgibytes/dependency-history-maven#24).

The http cache issue has been moved to a lower priority and other improvement mechanisms are being explored first.

[mscottford]

I've used hyperfine program to collect a performance baseline.

Arguments used:

hyperfine --prepare 'rm -rf ~/.freshli' --runs 5 'exe/freshli --loglevel=Debug --workers=6 analyze https://github.com/corgibytes/freshli-fixture-java-test' --show-output

Results:

  Time (mean ± σ):     37.170 s ±  0.628 s    [User: 265.770 s, System: 22.683 s]
  Range (min … max):   36.125 s … 37.684 s    5 runs

Git sha: eca78ceeffa34f46f266c23782229eb07850e151

[mscottford]

With both #363 and #364 things get a little bit worse:

  Time (mean ± σ):     71.392 s ±  1.805 s    [User: 213.920 s, System: 16.764 s]
  Range (min … max):   69.167 s … 73.208 s    5 runs

Git sha: f62a1b0aa2bf2aa921c87cfd5b4c58eaa2c8f071

It is worth noting that #363 fixes a bug, although it might make things take a little bit longer... it's possible there's a faster way to go about making that change, though.

[mscottford]

After making an adjustment to the locking mechanism in 92965c6 there is basically no change.

  Time (mean ± σ):     71.997 s ±  1.729 s    [User: 229.222 s, System: 18.075 s]
  Range (min … max):   70.455 s … 74.632 s    5 runs

Git sha: 379d08d7103a07c94f701b223f9eced934817b41

Note this run was while on battery power. It might affect the result. I'll re-run on wall power when I'm able to.