Use 'fd' rather than 'find' in makefiles if available

[tymcauley]

Type of change: other enhancement

Impact: other

Description:

I frequently run Chipyard on a server with NFS-mounted hard drives (in other words, accessing files on disk is very slow), and any given make command usually takes upwards of 3 seconds before it even starts running. I've done a bit of profiling with strace, and unsurprisingly most of make's time is spent on filesystem-related system calls (stat, close, etc). I didn't do extensive digging, but it looks to me like the lookup_srcs command in common.mk is the source of most of these system calls. Since that is based around find, I figured that trying out fd might present an opportunity for optimization. Here are some benchmarking results (done with hyperfine):

On that server with NFS-mounted spinning-platter hard drives:

# Using 'find'
$ hyperfine --ignore-failure --warmup 3 'make -q -C sims/vcs'
Benchmark #1: make -q -C sims/vcs
  Time (mean ± σ):      2.455 s ±  0.050 s    [User: 277.7 ms, System: 1037.2 ms]
  Range (min … max):    2.397 s …  2.528 s    10 runs
# Using 'fd'
$ hyperfine --ignore-failure --warmup 3 'make -q -C sims/vcs'
Benchmark #1: make -q -C sims/vcs
  Time (mean ± σ):      1.323 s ±  0.091 s    [User: 5.150 s, System: 12.332 s]
  Range (min … max):    1.276 s …  1.573 s    10 runs

On another server, but with an SSD:

# Using 'find'
$ hyperfine --ignore-failure --warmup 3 'make -q -C sims/vcs'
Benchmark #1: make -q -C sims/vcs
  Time (mean ± σ):     289.5 ms ±   1.2 ms    [User: 122.9 ms, System: 168.4 ms]
  Range (min … max):   288.1 ms … 291.8 ms    10 runs
# Using 'fd'
$ hyperfine --ignore-failure --warmup 3 'make -q -C sims/vcs'
Benchmark #1: make -q -C sims/vcs
  Time (mean ± σ):     240.8 ms ±   4.7 ms    [User: 629.7 ms, System: 278.9 ms]
  Range (min … max):   233.8 ms … 248.7 ms    12 runs

On a MacBook Pro (which has an SSD):

# Using 'find'
$ hyperfine --ignore-failure --warmup 3 'make -q -C sims/vcs'
Benchmark #1: make -q -C sims/vcs
  Time (mean ± σ):     362.3 ms ±   7.2 ms    [User: 112.8 ms, System: 243.2 ms]
  Range (min … max):   359.0 ms … 382.5 ms    10 runs
# Using 'fd'
$ hyperfine --ignore-failure --warmup 3 'make -q -C sims/vcs'
Benchmark #1: make -q -C sims/vcs
  Time (mean ± σ):     330.4 ms ±   3.1 ms    [User: 670.4 ms, System: 609.8 ms]
  Range (min … max):   323.8 ms … 334.8 ms    10 runs

So, all of these environments show improvements, but they're especially pronounced on a more disk-IO-limited system.

Considerations:

Since the find command being replaced prunes all paths containing target, I figured I would just use the fd default settings, which ignore hidden directories as well as paths in your gitignore settings, which includes the target folders. Not sure if this is concerning to anyone, but I'm guessing that the source file dependencies don't live in hidden folders and aren't targeted by gitignore files.

Release Notes
Improve make speed by using fd to find files, if that command is available.

[jerryz123]

Thank you for this. The slow performance of find here has been bothering me.

fd ignores the top-level .gitignore, but does it also ignore the .gitignores in the submodules?

[tymcauley]

Thank you for this. The slow performance of find here has been bothering me.

fd ignores the top-level .gitignore, but does it also ignore the .gitignores in the submodules?

As far as I know, fd uses gitignore files in the same way that git does. If you have a directory structure like this:

./
|-> .gitignore
|-> A/
|-> B/
    |-> .gitignore

Then fd will apply ./.gitignore to all files in ./ and ./A/. In ./B/, it will apply ./.gitignore as well as ./B/.gitignore.

I wasn't able to do a full comparison between the find results and the fd results (I couldn't print the SCALA_SOURCES, VLOG_SOURCES, or EXTRA_GENERATOR_REQS variables due to too-many-arguments errors), but I could print out SBT_SOURCES, and I noticed that one file was missing from the fd list: tools/chisel-testers/project/plugins.sbt. That's because tools/chisel-testers/.gitignore has a few lines which completely ignore the project directory.