.should('deep.equal') does not show a full diff comparison on failures

[testerez]

Current behavior:

Cypress does not show enough information to know what is the difference when deep comparing complex objects.

Example:

cy
  .wrap({ foo1: { bar: { foo1: { bar: 1 } } }, foo2: { bar: 1 } })
  .should('deep.equal', { foo1: { bar: { foo1: { bar: 1 } } }, foo2: { bar: 2 } })

Result:

Desired behavior:

Ideally, I'd like cypress to show a complete diff of the compared objects so we can easily spot the difference.

Versions

[email protected]

[jennifer-shehane]

Showing a more detailed diff of objects in the error message is part of this larger issue, that is currently in progress: #3762 You can see an comment on our planned designs there.

Will leave this open as part of that epic.

You can also click on the Assert command in the Command Log to see a fully diff within your DevTools console.

[testerez]

Thank you Jenniver, Happy to know this is a work in progress!
Thanks also for the console tips, very useful!

[ayushjain94]

Any estimation on when this'll be completed and released? Also, will appreciate any design to write this as a custom method to have this functionality ?

[william-wallace-simplisafe]

Thought I'd chime in on this one as the existence of this issue affects my decision to use cypress for REST API testing.

[tsvetan-ganev]

@jennifer-shehane, is there a technical reason why the two objects aren't diffed? It's a basic feature for most assertion libraries/test runners. Also I see that the related issue is closed, but this problem still exists in 2022. Are there any ways I can hook into the AssertionError and manually produce a diff from the two objects?

[gurudattgd04]

Here is the link for the issue which is still open for the diff feature #3129

Hope this will be addressed soon as its been years since this has been raised

[franzvezuli]

@Braveheart55 @tsvetan-ganev @gurudattgd04 If you are interested, here is a work-around that a friend of mine (@Bushwazi) came up with that we are using:

In our /support/index.js: we intercept and override our chai expect(expected).to.deep.equal(actual) lines across all of our tests

import { diff } from 'json-diff';

/*
 * Wraps/overwrites the "equal" method so that we can pretty print the diff
 * during errors only.
 *
 * The diffing library adds a "~" JSON key when looking at arrays, but it
 * doesn't affect anything else.
 */
chai.use(function (_chai, utils) {
  utils.overwriteMethod(chai.Assertion.prototype, 'equal', function (_super) {
    return function newEqual(str) {
      let obj = utils.flag(this, 'object');
      // "equal" has a "deep" flag it "deep" precedes it
      let deep = utils.flag(this, 'deep');

      try {
        _super.apply(this, arguments);
      } catch (error) {
        // If we do not have the "deep" flag, just throw the standard error
        if (deep !== true) {
          throw(error);
        }

        // If we do have the "deep" flag, use the diff module for highlight the diff
        // and then tidy it up a bit
        let diffObject = "expect objects to match:\n";
        diffObject += JSON.stringify(diff(str, obj), null, 4)
          .replace(/__old/g, "-")
          .replace(/__new/g, "+");
        throw(diffObject);
      }
    }
  });
});

Notes

• This only gets printed during an assertion failure.
• This uses this JSON diff library: https://www.npmjs.com/package/json-diff
• You can get a prettier diff (colorized) if you use console.log() and check the console instead of in-line printing it in the test runner as well.

It will look like this:

---

I've also been following this issue for 3 years+.

I understand Cypress is geared more towards UI-testing than API-testing (For example, I don't need a ViewPort if I'm just testing a JSON REST API, yet there is no flag to turn this off), but they really should start adding some quality of life changes for API-only testing.

It would be great if there was an API_ONLY_TESTING flag or something similar. I bet they can turn off a bunch of UI-only functionality -- and improve Cypress performance and responsiveness if you are just testing a REST API.

[hydrajump]

@franzvezuli thanks for sharing your workaround!

When I try to use it I'm seeing the following Type error:

Cannot create property 'isDefaultAssertionErr' on string 'expect objects to match:

{

"-": [
    "item1",
    "item2"
],
"+": [
    "item4",
    "item2"
]
}'

Any ideas what may be wrong?

[tsonnen-eventus]

You need to throw an error object, something like this should work.

var error = new Error(diffObject);
throw error;

[hydrajump]

@tsonnen-eventus I'm not sure where that code should go. Can you please elaborate on your solution?

As an aside without @franzvezuli workaround or any other changes I'm seeing a diff when using the Cypress Netlify plugin in the Netlify deployment logs:

       Timed out retrying after 4000ms
       + expected - actual
        [ 'a',
       -  'b',
          'c',
          'd',
          'e',
          'f',
 --
          'a',
          'c',
          'd',
          'e',
       +  'x',
          'f' ]

Is it the Cypress plugin that has this logic and if so couldn't it be extended to the web dashboard?

[tsonnen-eventus]

In the @franzvezuli solution, change

    let diffObject = "expect objects to match:\n";
    diffObject += JSON.stringify(diff(str, obj), null, 4)
        .replace(/__old/g, "-")
        .replace(/__new/g, "+");
    throw(diffObject);

to

    let diffObject = "expect objects to match:\n";
    diffObject += JSON.stringify(diff(str, obj), null, 4)
        .replace(/__old/g, "-")
        .replace(/__new/g, "+");
    let error = new Error(diffObject);
    throw error;

[gurudattgd04]

I had written down a post that chai assertion in cypress not showing the actual diff and also the solution. Link here https://www.linkedin.com/posts/gurudatt-s-a-5b892224_cypress-cypressio-testautomation-activity-6903244451108519936-4klR?utm_source=linkedin_share&utm_medium=ios_app See if this helps

…

On Thu, 7 Apr 2022 at 11:44 AM, Tim Sonnen ***@***.***> wrote: In the @franzvezuli <https://github.com/franzvezuli> solution, change let diffObject = "expect objects to match:\n"; diffObject += JSON.stringify(diff(str, obj), null, 4) .replace(/__old/g, "-") .replace(/__new/g, "+"); throw(diffObject); to let diffObject = "expect objects to match:\n"; diffObject += JSON.stringify(diff(str, obj), null, 4) .replace(/__old/g, "-") .replace(/__new/g, "+"); let error = new Error(diffObject); throw error; — Reply to this email directly, view it on GitHub <#4084 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABLBBTAIT6KLFAOWIUCWNMLVDZ4LNANCNFSM4HJRN2TQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[hydrajump]

In the @franzvezuli solution, change

    let diffObject = "expect objects to match:\n";
    diffObject += JSON.stringify(diff(str, obj), null, 4)
        .replace(/__old/g, "-")
        .replace(/__new/g, "+");
    throw(diffObject);

to

    let diffObject = "expect objects to match:\n";
    diffObject += JSON.stringify(diff(str, obj), null, 4)
        .replace(/__old/g, "-")
        .replace(/__new/g, "+");
    let error = new Error(diffObject);
    throw error;

Unfortunately, that didn't work.

SyntaxError: /opt/build/repo/cypress/support/index.js: Identifier 'error' has already been declared. (31:12)

  29 |           .replace(/__old/g, "-")
  30 |           .replace(/__new/g, "+");
> 31 |         let error = new Error(diffObject);
     |             ^
  32 |         throw error;
  33 |       }
  34 |     }

I'll just stick with the Netlify logs that do the diff with magic.

[karlhorky]

@jennifer-shehane now that #3762 has been closed (via PR #3930 and #6724), what is left to show the diff of deeply-nested objects? Is this planned for one of the next Cypress versions?

[cesarvarela]

Seeing this issue open for more than 3 years makes me wonder what people do instead 🤔

[cesarvarela]

chai.config.truncateThreshold = 0;

Does help with this, but there is still so much busy work writing expectations property by property instead of just deep comparing two objects and getting a useful diff on error.

[jsodeman]

A note for those trying the code from @franzvezuli

json-diff should be installed in the dev dependencies npm i json-diff -D

and the code block from #4084 (comment) should be be pasted into /support/e2e.js or similar, see https://docs.cypress.io/guides/core-concepts/writing-and-organizing-tests#Support-file for the new file locations.

then the last few lines should be replaced with the code from @tsonnen-eventus but with a renamed variable

let diffObject = "expect objects to match:\n";
diffObject += JSON.stringify(diff(str, obj), null, 4)
	.replace(/__old/g, "-")
	.replace(/__new/g, "+");
let diffError = new Error(diffObject);
throw diffError;

Thank you to both of them for providing help.

[colcherica]

Hi guys, I have the same issue on my side, the assertion error is not fully displayed

[Brachacz]

4 years in, no luck yet to have it out of the box?

[Snailpower]

Ran into this issue today, as wel as no option out of the box to use deep.equalInAnyOrder.

Would love to see both in a new version! Helps a lot with (cucumber) reports when a screenshot is made

[johngallagher]

I also ran into this problem!

expected [ Array(3) ] to deeply equal [ 'agent-1', 'agent-2', 'agent' ]

This isn't a very helpful error message. Granted, I can switch to the console to get the error. Any plans to fix this?

[jpierson-at-riis]

I just had a look at this jsondiffpatch library and realized that the live demo is pretty impressive in terms of diff visuals. Perhaps this could be a possible direction for Cypress especially in the visual GUI runner.
https://github.com/benjamine/jsondiffpatch

[steelx]

still facing is there any function equalInAnyOrder for cypress ?

[elaichenkov]

I've just released a plugin that highlights the difference. Simple installation and usage:

https://github.com/elaichenkov/cypress-diff

npm i -D cypress-diff

// cypress/support/e2e.ts
import 'cypress-diff';

Result:

Feel free to open an issue with improvements. Also, don't forget to give it a star.