Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Google Cloud Run deployment not working #33313

Closed
iKK001 opened this issue Oct 27, 2024 · 2 comments
Closed

[Bug]: Google Cloud Run deployment not working #33313

iKK001 opened this issue Oct 27, 2024 · 2 comments

Comments

@iKK001
Copy link

iKK001 commented Oct 27, 2024

Version

1.48.2

Steps to reproduce

I try to deploy my nodeJS project to a Google Cloud Run.

But unfortunately, the location where chromium gets installed is corrupt !!

Normally, I do not have problems to deploy to Google Cloud Run.

But using your library playwright makes it hard.

Inside my package.json file I add the following dependency:

"scripts": {
    "build": "nest build",
    "format": "prettier --write \"src/**/*.ts\" \"test/**/*.ts\"",
    "start": "nest start",
    "start:dev": "nest start --watch",
    "start:debug": "nest start --debug --watch",
    "start:prod": "node dist/main",
    "start:emulator": "export FIRESTORE_EMULATOR_HOST='localhost:8080' && firebase emulators:start --project counter-1227a --import emulatorBaseData",
    "lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
    "test": "export FIRESTORE_EMULATOR_HOST='localhost:8080' && jest",
    "test:watch": "jest --watch",
    "test:cov": "jest --coverage",
    "test:debug": "node --inspect-brk -r tsconfig-paths/register -r ts-node/register node_modules/.bin/jest --runInBand",
    "test:e2e": "jest --config ./test/jest-e2e.json"
  },
  "engines": {
    "node": "20.x"
  },
"dependencies": {
    "playwright": "^1.48.2"
},

And I am using yarn.

My method in my application looks as follows:

async createBuffer(): Promise<Buffer> {

    const url = 'https://www.google.com';

    // Launch a new browser instance
    const browser = await chromium.launch({
      executablePath: process.env.PLAYWRIGHT_BROWSERS_PATH,
      headless: true,
    });

    // Create a new incognito browser context.
    const context = await browser.newContext();

    // Create a new page in a this context.
    const page = await context.newPage();

    // Navigate to the specified URL
    await page.goto(url);

    // take a screenshot and return the buffer
    const buffer: Buffer = await page.screenshot();

    // Close the browser
    await context.close();
    await browser.close();

    return buffer;
  }

I am using the following ENV variable inside my Google Cloud Run:

PLAYWRIGHT_BROWSERS_PATH=0

(And I tried endless times to set this env var to something else so that it would work.

BUT I HAVE NO IDEA WHAT THIS PLAYWRIGHT_BROWSERS_PATH MUST BE ???? Any help here appreciated.

But let'ts continue with my explanation:

The deployment works !
The chromium gets installed. I see this in the log of the GCR deployment log.

2024-10-27 03:39:10.698 CET
Step #0 - "Build": Downloading Chromium 130.0.6723.31 (playwright build v1140) from https://playwright.azureedge.net/builds/chromium/1140/chromium-linux.zip
2024-10-27 03:39:10.975 CET
Step #0 - "Build": | | 0% of 164.5 MiB
2024-10-27 03:39:11.327 CET
Step #0 - "Build": |■■■■■■■■ | 10% of 164.5 MiB
2024-10-27 03:39:11.575 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■ | 20% of 164.5 MiB
2024-10-27 03:39:11.776 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■ | 30% of 164.5 MiB
2024-10-27 03:39:11.951 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ | 40% of 164.5 MiB
2024-10-27 03:39:12.124 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ | 50% of 164.5 MiB
2024-10-27 03:39:12.280 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ | 60% of 164.5 MiB
2024-10-27 03:39:12.445 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ | 70% of 164.5 MiB
2024-10-27 03:39:12.627 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ | 80% of 164.5 MiB
2024-10-27 03:39:12.791 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ | 90% of 164.5 MiB
2024-10-27 03:39:12.948 CET
Step #0 - "Build": |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■| 100% of 164.5 MiB
2024-10-27 03:39:18.877 CET
Step #0 - "Build": Chromium 130.0.6723.31 (playwright build v1140) downloaded to /root/chromium-browser/chromium-1140

As you can see, it downloaded it to /root/chromium-browser/chromium-1140
(or any other folder that I define in the Dockerfile.

--> Now the problem:

Whenever I call my method createBuffer(), then I get the following error-message inside the application log:

CASE 1

--> if I use the followng cmd inside Dockerfile: RUN npx -y [email protected] install --with-deps chromium

2024-10-27 04:01:09.622 CET
�[31m[Nest] 1 - �[39m10/27/2024, 3:01:09 AM �[31m ERROR�[39m �[38;5;3m[ExceptionsHandler] �[39m�[31mbrowserType.launch: Executable doesn't exist at /usr/src/app/chromium-1140/chrome-linux/chrome
2024-10-27 04:01:09.622 CET
╔═════════════════════════════════════════════════════════════════════════╗
2024-10-27 04:01:09.622 CET
║ Looks like Playwright Test or Playwright was just installed or updated. ║
2024-10-27 04:01:09.622 CET
║ Please run the following command to download new browsers: ║
2024-10-27 04:01:09.622 CET
║ ║
2024-10-27 04:01:09.622 CET
║ npx playwright install ║
2024-10-27 04:01:09.622 CET
║ ║
2024-10-27 04:01:09.622 CET
║ <3 Playwright Team ║
2024-10-27 04:01:09.622 CET
╚═════════════════════════════════════════════════════════════════════════╝�[39m

CASE 2

--> if I use the followng cmd inside Dockerfile: RUN PLAYWRIGHT_BROWSERS_PATH=/root/chromium-browser npx -y [email protected] install --with-deps chromium

Error message :

Failed to launch chromium because executable doesn't exist at

My Dockerfile:

######################
# BUILD FOR PRODUCTION
######################

FROM node:20-bookworm AS build

WORKDIR /usr/src/app

RUN npx -y [email protected] install --with-deps chromium

# I also tried:
# RUN PLAYWRIGHT_BROWSERS_PATH=/root/chromium-browser npx -y [email protected] install --with-deps chromium    

COPY --chown=node:node package*.json ./

COPY --chown=node:node --from=development /usr/src/app/node_modules ./node_modules

COPY --chown=node:node . .

RUN yarn build

ENV NODE_ENV=production

RUN yarn install --production && yarn cache clean

USER node


############
# PRODUCTION
############

FROM node:20-bookworm AS production

COPY --chown=node:node --from=build /usr/src/app/node_modules ./node_modules
COPY --chown=node:node --from=build /usr/src/app/dist ./dist

CMD [ "node", "dist/main.js" ]

Expected behavior

I expect to be able to deploy into a Google Cloud Run, installing chromium and using the browser from playwright in my calling method in nodeJS.

Actual behavior

Two bugs:

Case I: If I don't use PLAYWRIGHT_BROWSERS_PATH inside the Dockerfile as ENV var :
It keeps telling me

Please run the following command to download new browsers:  
npx playwright install

Case II: I I do use PLAYWRIGHT_BROWSERS_PATH in the Dockerfile:
It keeps telling me
Failed to launch chromium because executable doesn't exist at <whatever I define as the ENV var>

Additional context

No response

Environment

System:
 OS: AMD64 (Google Cloud Run environment)
Binaries:
    Node: 20.18.0 - ~/.nvm/versions/node/v20.18.0/bin/node
    Yarn: 1.22.19 - ~/.yarn/bin/yarn
    npm: 10.8.2 - ~/.nvm/versions/node/v20.18.0/bin/npm
npmPackages:
    playwright: ^1.48.2 => 1.48.2
Dockerfile image:
    FROM node:20-bookworm AS build

Questions:

1. How can I make this deployment in Google Cloud Run work ?

2. How can I better determine where the Chromium is really installed ? And how do I refer to it in executablePath of the calling method ?

3. How do I use PLAYWRIGHT_BROWSERS_PATH exactly ?? Is it needed in the Dockerfile ? If yes, how ?

4. What else is missing ? Is there a working example of a Google Cloud Run deployment using nodeJS and Dockerfile ?

@iKK001 iKK001 changed the title [Bug]: [Bug]: Google Cloud Run deployment not working Oct 27, 2024
@iKK001
Copy link
Author

iKK001 commented Oct 27, 2024

After 48 hours of non-stop debugging, I found a solution:

Turns out that many linux versions (such as debian, alpine, bookworm, ubuntu etc.) don't really work for playwright in a Google Cloud Run deployment.

However, there is a linux image that you can take :

FROM mcr.microsoft.com/playwright:v1.48.2-noble

Here is the final Dockerfile that works now :

#############################
# BUILD FOR LOCAL DEVELOPMENT
#############################

# Use the Playwright image as the base (its based on Ubuntu 24.04 LTS)
FROM mcr.microsoft.com/playwright:v1.48.2-noble AS development

# Create app directory
WORKDIR /usr/src/app

# Copy application dependency manifests to the container image.
# Remove the chown option since the user may not exist in the Playwright image
COPY package*.json ./

# Install app dependencies
RUN yarn install

# Install required dependencies for Playwright and Chromium
RUN apt-get update && \
    apt-get install -y libnss3 libatk-bridge2.0-0 libgtk-3-0 libgbm-dev \
    libx11-xcb1 libxcomposite1 libxcursor1 libxi6 libxtst6 \
    libxrandr2 libxss1 fonts-liberation libxshmfence1 \
    --no-install-recommends && \
    rm -rf /var/lib/apt/lists/*

# Set environment variable for Playwright browser path
ENV PLAYWRIGHT_BROWSERS_PATH=/root/.cache/ms-playwright

# Ensure the Playwright cache directory exists and has proper permissions
RUN mkdir -p $PLAYWRIGHT_BROWSERS_PATH && chmod -R 777 $PLAYWRIGHT_BROWSERS_PATH

# Install Playwright browsers with all dependencies
RUN npx playwright install --with-deps chromium

# Debugging: Print out the installation location
RUN echo "Chromium installed at (development):" && ls -l $PLAYWRIGHT_BROWSERS_PATH

######################
# BUILD FOR PRODUCTION
######################

# Use the Playwright image as the base (its based on Ubuntu 24.04 LTS)
FROM mcr.microsoft.com/playwright:v1.48.2-noble AS build

WORKDIR /usr/src/app

# Copy all application files, including tsconfig.json and other configs
COPY . .

# Copy node_modules from development stage to save time on installing
COPY --from=development /usr/src/app/node_modules ./node_modules

# Install required dependencies for Playwright and Chromium
RUN apt-get update && \
    apt-get install -y libnss3 libatk-bridge2.0-0 libgtk-3-0 libgbm-dev \
    libx11-xcb1 libxcomposite1 libxcursor1 libxi6 libxtst6 \
    libxrandr2 libxss1 fonts-liberation libxshmfence1 \
    --no-install-recommends && \
    rm -rf /var/lib/apt/lists/*

# Set environment variable for Playwright browser path
ENV PLAYWRIGHT_BROWSERS_PATH=/root/.cache/ms-playwright

# Ensure the Playwright cache directory exists
RUN mkdir -p $PLAYWRIGHT_BROWSERS_PATH && chmod -R 777 $PLAYWRIGHT_BROWSERS_PATH

# Install Playwright browsers with all dependencies
RUN npx playwright install --with-deps chromium

# Build production bundle
RUN yarn build

# Set NODE_ENV environment variable
ENV NODE_ENV=production

# Install production dependencies only and clean yarn cache
RUN yarn install --production && yarn cache clean

############
# PRODUCTION
############

# Use the Playwright image as the base (its based on Ubuntu 24.04 LTS)
FROM mcr.microsoft.com/playwright:v1.48.2-noble AS production

# Create app directory
WORKDIR /usr/src/app

# Copy the bundled code and dependencies from the build stage
COPY --from=build /usr/src/app/node_modules ./node_modules
COPY --from=build /usr/src/app/dist ./dist

# Copy Playwright browsers from build stage to the production image (otherwise Chromium cannot be found)
COPY --from=build /root/.cache/ms-playwright /root/.cache/ms-playwright

# Ensure the Playwright cache directory exists and has proper permissions
RUN mkdir -p /root/.cache/ms-playwright && chmod -R 777 /root/.cache/ms-playwright

# Start the server using the production build
CMD [ "node", "dist/main.js" ]

@mxschmitt
Copy link
Member

Awesome! Closing then as per above. We don't support Alpine as per here and recommend our Docker image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants