fuzz: link libfuzzer engine to fuzz with bazel!

[asraa]

This makes a new config "asan-fuzzer" that links to the libfuzzer engine.

To run a fuzz target with asan and libfuzzer, do
bazel build --config=asan-fuzzer test/common/router:route_fuzz_test_with_libfuzzer --config=asan-fuzzer
bazel-bin/test/common/router/route_fuzz_test_driver test/common/router/route_corpus -runs=-1

(-runs=-1 for indefinite runs, -runs=100 for bounded).

Testing: works locally
Risk: Low

[asraa]

/retest

[repokitteh-read-only]

🔨 rebuilding ci/circleci: clang_tidy (failed build)

🐱

Caused by: a #7805 (comment) was created by @asraa.

see: more, trace.

[htuch]

This is super awesome, nice to see that it's relatively few lines of code and not too much hackery.

Can you update https://github.com/envoyproxy/envoy/blob/master/test/fuzz/README.md with a section on how to run this on CLI. I think the best guidance is to tell folks to do something like:

bazel run //test/common/common:base64_fuzz_test_driver --config asan-fuzzer

/wait

[asraa]

/retest

[repokitteh-read-only]

🔨 rebuilding ci/circleci: clang_tidy (failed build)

🐱

Caused by: a #7805 (comment) was created by @asraa.

see: more, trace.

[htuch]

This is huge improvement to fuzzing development productivity, LGTM. I like the ability to work directly with corpus.

@yevgenypats this is probably the right basis for doing fuzzing outside of oss-fuzz when working directly under CI.

[yevgenypats]

This is awesome, I'll rebase my PR on top of this one, once it is merged!.

[asraa]

/retest

[repokitteh-read-only]

🔨 rebuilding ci/circleci: clang_tidy (failed build)

🐱

Caused by: a #7805 (comment) was created by @asraa.

see: more, trace.

[yevgenypats]

looks like one of tests here fails. maybe it's out of memory?

[lizan]

Can you try applying this? I think it will fix clang_tidy:

diff --git a/tools/gen_compilation_database.py b/tools/gen_compilation_database.py
index 94869312b..4090b0505 100755
--- a/tools/gen_compilation_database.py
+++ b/tools/gen_compilation_database.py
@@ -8,7 +8,7 @@ import subprocess

 def generateCompilationDatabase(args):
   if args.run_bazel_build:
-    subprocess.check_call(["bazel", "build"] + args.bazel_targets)
+    subprocess.check_call(["bazel", "build", "--build_tag_filters=-manual"] + args.bazel_targets)

   gen_compilation_database_sh = os.path.join(
       os.path.realpath(os.path.dirname(__file__)), "../bazel/gen_compilation_database.sh")

Also merge master to pick up CI changes.

[yevgenypats]

This is still gets killed due to oom. I suspect that you need to add bazel build --jobs 8 or otherwise it will use 200 workers and will overuse the memory and will get killed by Circle.

[asraa]

Thank you! Yes, let's see. I'm not sure what the implications are later, but I did add the jobs flag to see what happens.

[yevgenypats]

horray! it worked:) #why_bazel_spins_up_200_workers_by_default:)

[asraa]

Yes!

@htuch, @lizan -- what do you think about this? (if this merges, probably everyone will OOM CI without the --jobs flag. how many jobs are appropriate, or is there a better way to mitigate this?)

[lizan]

Can you try to fix the jobs number to env ${NUM_CPUS} (which is set to 8 in CI)?

@yevgenypats bazel automatically spins up job to the number of CPU cores, which ignores cgroup, so in a docker environment in large host, it spins job more than cgroup restriction.

[asraa]

Thanks everyone. clang-tidy runs with NUM_CPUS jobs now.

[yevgenypats]

horray! lets merge!