[BUG] [Playwright] Lambda /tmp fills up quickly #231
Comments
sg650
changed the title
|
@Sparticuz This seems to only happen when using Playwright. With puppeteer the /tmp space doesn't grow like this. Any ideas? |
|
Did you ever resolve this? Getting similiar results with playright |
|
@Sparticuz Any insight here? Enjoy your travels, and thanks for the great tool. |
|
I had the same problem with Playwright even running on a Python Docker Image on Lambda. In my post I said Lambda is running out of memory until it crashes, but it had the same problem with So I think this issue should probably be moved to the Playwright repo, even though I don't think you'll get any help tehre. |
|
Any update on this? I assume it's more of a Puppeteer issue though, but we have the same issue when trying to generate a large number of PDF's... We get a few thousand Invoices (as XML) from a customer and then we need to generate PDF's for these. We have set ephemeral storage to 10GB and we can generate some 600-700 PDF's (invokations) before it runs out of /tmp space, with either AWS is not using "isolated" /tmp storage per invokation, even if we set ephemeral storage size for a Lambda, so it runs out of /tmp space when we have concurrent invokations. We can't delete /tmp in one Lambda either, as each concurrent Lambda is using the same "disk". So, is it possible to know which files "belongs" to which invokation to only delete the ones belonging to that instance? I've raised a support ticket with AWS as IMO the ephemeral storage should be available as the set value for EVERY invokation. That's how I interpret this article as well: https://aws.amazon.com/blogs/compute/using-larger-ephemeral-storage-for-aws-lambda/ To add to this, I can see in the /**
* Defines the directory to be used by Puppeteer for creating temporary files.
*
* Can be overridden by `PUPPETEER_TMP_DIR`.
*
* @defaultValue `os.tmpdir()`
*/
temporaryDirectory?: string;However, I can't find any code where |
|
After some more investigation I came up with a slight "ugly-fix" to the problem... const puppeteer = require('puppeteer-core');
const chromium = require('@sparticuz/chromium');
const uuid = require('uuid');
const fs = require('fs');
// code here...
chromium.args = [
'--no-sandbox',
'--headless',
'--disable-gpu',
'--disable-dev-shm-usage'
];
const {args} = chromium;
const tmpDirKey = uuid.v4();
args.push(`--user-data-dir=/tmp/${tmpDirKey}`); // chromium neglects this arg, so we add it ourselves
const browser = await puppeteer.launch({
args,
defaultViewport: chromium.defaultViewport,
executablePath,
headless: true,
ignoreHTTPSErrors: true
});
// more code to do stuff
console.info(`Cleaning up /tmp/${tmpDirKey}...`);
await fs.rmSync(`/tmp/${tmpDirKey}`, { recursive: true, force: true });
|
Environment
@sparticuz/chromiumVersion: ^121.0.0playwright/playwright-coreVersion: 1.41.2Expected Behavior
Should be able to request a
await page.goto(c.req.query('url'));indefinitely.Current Behavior
With 1GB ephemeral storage allocated to the Lambda, the /tmp spaces fills up after about 30-40 invocations and every subsequent request fails with:
ERROR Unexpected Error: page.goto: net::ERR_INSUFFICIENT_RESOURCES at https://www.google.com/Steps to Reproduce
Create a basic lambda that runs the following code...
LOGS
Possible Solution
I wish i knew of one, I am stumped so far. I have tried increasing to 4GB and 8 GB and it still fills up all the space.
The text was updated successfully, but these errors were encountered: