Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark support #35

Open
Fuuzetsu opened this issue Dec 22, 2017 · 1 comment
Open

Benchmark support #35

Fuuzetsu opened this issue Dec 22, 2017 · 1 comment
Labels
P3 minor: not priorized type: feature request

Comments

@Fuuzetsu
Copy link
Collaborator

We may want some benchmark support. Some ideas:

  • benchmarks that produce HTML/json/whatever: think criterion outputs
  • it could just be a haskell_binary with some bazel tag we pick that identifies it as a benchmark: at the very least the user can ask bazel to list/run all rules with the tag then.

I think we definitely should just start with haskell_binary alias with a rule tag.

@mboes mboes added the P3 minor: not priorized label Jan 29, 2018
@thufschmitt
Copy link
Contributor

At novadiscovery, we use a haskell_test for that, with a special benchmark tag which is excluded by default, so that bazel test //... won't run the benchmarks and bazel test //... --test_tags_filter=benchmark will run all of them.

To keep the outputs, we use a custom criterion runner (i.e. a wrapper around Criterion.defaultMain) which puts all the outputs under $TEST_UNDECLARED_OUTPUTS_DIR, so that they are available after under bazel-testlogs/my/target/test.outputs/result.{html,csv,json}

That woks quite well for us, though that require some specific setup inside the repository (a test --test_tag_filters=-benchmark in .bazelrc and a library exporting the custom runner).

Profpatsch pushed a commit that referenced this issue Mar 7, 2019
The current rules_haskell setup is heavily Nix-based (including the Bazel
binary) and its usability relies on having Cachix or some other custom Nix
cache.

- Switch to an Ubuntu-based container
- Add and use cc_configure.bzl.
- Update the versions of nixpkgs for consistency
- Use the FormationAI/rules_haskell fork, temporarily
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P3 minor: not priorized type: feature request
Projects
None yet
Development

No branches or pull requests

3 participants