DOM and heap leak with Playwright

[kallebjork]

Discussed in https://github.com/microsoft/playwright/discussions/16718

^{Originally posted by kallebjork August 22, 2022, converted to an issue since I feel this is not just me doing something wrong?}

Hi Playwright team!

First of all I want to say that you are doing a great job - Playwright is really really nice to work with. 🥇
** End of praise **

The problem? We are currently working on moving from Protractor to Playwright at my company. We create many apps which must be able to run on low spec hardware. We have therefore implemented some tools for tracking memory leaks and performance. These tools work just fine with Protractor today but when we move to Playwright nothing can be measured because pretty much everything seems to be retained. So we need your help in understanding what is going on here.

Are we doing something wrong in our tests or is this caused by how Playwright works?

If this is an issue on your end, are you planning on making this work? It is a crucial part for us 😞

Below is a reproduction which simply visits your site playwright.dev and keeps track of DOM nodes in memory while going between GETTING STARTED and landing.

Steps to reproduce:

1. Go to Stackblitz repro project https://stackblitz.com/edit/js-mzjyem?file=tests/playwright.e2e.ts&view=editor
2. Download the project to <some folder>
3. Go into directory <some folder>
4. Run npm i
5. Run npx playwright install
6. Go into directory tests
7. Run npx playwright test
8. Wait for the test to finish (we do some sleeps inside the test for the garbage collector to get a chance to do its work so have patience 🙂)

Expected results:
Since we are only going back and forth between documentation and landing, DOM nodes in memory should stay roughly the same. Between iterations we run garbage collection and wait a bit.

Actual results:
Each iteration results in ~1000 new DOM nodes in memory. You can see this in the log. The active DOM count seem to stay at 431 nodes while the in memory DOM count increases every iteration, suggesting that we have lots and lots of detached nodes?

[yury-s]

With the clicks (inside walk) you do same page navigation back and forth, some of the detached nodes may still be in memory until the page navigates away or the browser decides to dispose them. You can actually see how those numbers grow if you perform the actions manually, it's true that with playwright the numbers are bigger but that's because we create some auxiliary structures in the page (which will also be disposed on navigation), so this is expected.

Do you have a simple page where you think nodes are not leaking without playwright and leak with it?

[kallebjork]

@yury-s
The thing I get with Playwright is that the DOM count increases with every iteration even though we trigger a garbage collect. If I do this in Protractor or manually I do not see that happening.

Below is a manual test where the DOM count returns to 1122 when you trigger garbage collect, it does not matter if I repeat the walk 10 or 20 times:

[Smrtnyk]

We also plan to do some memory testing in our company using playwright, and would like to upvote this issue if more people upvoting can add weight to the priority of this issue

[jleufven]

I can give a bit more insight into what we currently do with Protractor (mostly developed by @kallebjork). Maybe @Smrtnyk can find some inspiration and it might (hopefully) give some weight to this issue.

For Protractor we have developed a small utility (~100 lines) that loop part of a test 100 times. The utility currently records:

• DOM nodes (through: browser.driver.sendChromiumCommandAndGetResult('DOM.getFlattenedDocument'))
• Event listeners (through: browser.driver.sendChromiumCommandAndGetResult('Memory.getDOMCounters'))
• usedJsHeap & totalHeapSize (using: window.performance.memory)
• "tab memory" (using a 70 lines chrome extension that injects the result from chrome.processes.onUpdatedWithMemory)
• KPI measurements like largest contentful paint and others, Angular APP_INITIALIZER done, first onStable (Testability.whenStable) (app should be fully rendered by then) and some internal ones.

The values are taken before the first iteration to get a baseline and after each iteration. These tests are run each night and results are exported to Grafana to visualize trends and are shown on the teams build monitors. Since we added it a year ago our teams have found a lot of leaks, regressions and other bugs (like leaks in Angular).

On top of normal tests, teams normally have stress tests that clicks through most of their app in one sequence (repeating that sequence). Ex: search, look at details and add X items to an order, fill in the ordering details, abort and clear the order, repeat 100 times.

It should be noted that in Protractor we need to nuke the element cache that it keeps in the browser before each measurement. We also try our best to get a stable memory reading by calling both window.gc and browser.driver.sendChromiumCommand('HeapProfiler.collectGarbage') a few times 😄 and then waiting quite some time to let the browser release memory. Hence these tests are quite slow and we only run the stress test once daily.

[mastrzyz]

Same, we are seeing the Dom Cache leaking as well, for Selenium we had a "hack" to get past this limitation ->

    const clearScript = `
      return window.document.$cdc_asdjflasutopfhvcZLmcfl_.cache_ = Object.create(null);
    `;

[mastrzyz]

Anything we can do to help further this investigation @yury-s ?

[pavelfeldman]

I downloaded your project and repeated the scenario manually in my Chrome browser:

• navigated to playwright.dev
• opened 'Performance monitor' in the DevTools console drawer
• started clicking getting started and playwright logo to go back and forth.

What I'm seeing is that DOM counter grows, there probably is a leak somewhere in docusaurus react effects. Could you share a repro that would demonstrate the issue in a self-contained manner? Local server or a single page.setContent call?

[aslushnikov]

It looks like the discussion has stalled; closing this. Please feel free to file a new issue if this still bothers you and you have a repro that we can use to reproduce!

[kallebjork]

It took you 213 days to follow my step by step reproduction and then you give me max 5 days to respond with a new reproduction before you close the issue, saying it ”stalled”. Not fair 😂 Playwright still #1 though, keep up the good work. Maybe we meet again in a new thread.

[aslushnikov]

@kallebjork Yeah, sorry! We try to avoid stale issues. It's also hard to come back to long discussions that swing from one direction to another and update only once-in-a-while. A new issue will be right in our face, and we definitely won't miss it!