Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easy to detect regressions generated by nimma usage #1281

Closed
nulltoken opened this issue Jul 11, 2020 · 1 comment
Closed

Make it easy to detect regressions generated by nimma usage #1281

nulltoken opened this issue Jul 11, 2020 · 1 comment

Comments

@nulltoken
Copy link
Contributor

nulltoken commented Jul 11, 2020

Nimma has been introduced in #1136. It provides a more performant way to perform JsonPath based lookups.

However it may need some more refinements (cf. #1260 for instance).

In order to move forward and make nimma the default, we'd need more use cases and wider tests.

The proposal would be to add a switch to the CLI params (--nimma-double-run-opt-in for instance) and to request some help from Spectral users so that they run Spectral with this switch on.

When activated, this would run everything twice (with AND without nimma), fail the run and report a very explicit message if anything differs.

  • This would be easy to setup for the users
  • But may take up to twice the time to run (but mostly on CI)

The generated message should provide everything that's needed to help us fix the issue (mostly the JsonPath expressions that generated different outputs, I believe...) in a templated message the user could copy and paste while creating the issue in this tracker.

We may need an additional switch (eg do-not-fail-on-nimma-regressions) that users that have run into a nimma regression could temporarily activate while we're fixing it.

This switch would be dropped once nimma is considered stable and the default JsonPath lookup engine.

@P0lip
Copy link
Contributor

P0lip commented Oct 4, 2021

Closing this one out - nimma is no longer opt-in, it's enabled by default.
That said, I ensured we are backward compatible with jsonpath-plus, by:

@P0lip P0lip closed this as completed Oct 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants