Development: Run e2e tests on bamboo

docs/dev/playwright.rst

@@ -7,6 +7,18 @@ Set up Playwright locally
 To run the tests locally, developers need to set up Playwright on their machines.
 End-to-end tests test entire workflows; therefore, they require the whole Artemis setup - database, client, and server to be running.
 Playwright tests rely on the Playwright Node.js library, browser binaries, and some helper packages.
+To run playwright tests locally, you need to start the Artemis server and client, have the correct users set up and install and run playwright.
+This setup should be used for debugging, and creating new tests for your code, but needs intellij to work, and relies on fully setting up your local Artemis instance
+following :ref:`the server setup guide<dev_setup>`.
+
+
+For a quick test setup with only three steps, you can use the scripts provided in `supportingScripts/playwright`.
+The README explains what you need to do.
+It sets up Artemis inside a dockerized environment, creates users and directly starts playwright. The main drawback with this setup is, that you cannot
+easily change the version of Artemis itself.
+
+
+If you want to manually install playwright, you can follow these steps:
 
 1. Install dependencies:
 
@@ -29,40 +41,49 @@ Playwright tests rely on the Playwright Node.js library, browser binaries, and s
 
     .. code-block:: text
 
-        PLAYWRIGHT_USERNAME_TEMPLATE=artemis_test_user_USERID
-        PLAYWRIGHT_PASSWORD_TEMPLATE=artemis_test_user_USERID
+        PLAYWRIGHT_USERNAME_TEMPLATE=artemis_test_user_
+        PLAYWRIGHT_PASSWORD_TEMPLATE=artemis_test_user_
         ADMIN_USERNAME=artemis_admin
         ADMIN_PASSWORD=artemis_admin
         ALLOW_GROUP_CUSTOMIZATION=true
         STUDENT_GROUP_NAME=students
         TUTOR_GROUP_NAME=tutors
         EDITOR_GROUP_NAME=editors
         INSTRUCTOR_GROUP_NAME=instructors
-        CREATE_USERS=true
         BASE_URL=http://localhost:9000
         EXERCISE_REPO_DIRECTORY=test-exercise-repos
+        FAST_TEST_TIMEOUT_SECONDS=45
+        SLOW_TEST_TIMEOUT_SECONDS=180
+
 
     Make sure ``BASE_URL`` matches your Artemis client URL and ``ADMIN_USERNAME`` and
     ``ADMIN_PASSWORD`` match your Artemis admin user credentials.
 
 3. Configure test users
 
-     Playwright tests require users with different roles to simulate concurrent user interactions. You can configure
-     user IDs and check their corresponding user roles in the ``src/test/playwright/support/users.ts`` file. Usernames
-     are defined automatically by replacing the ``USERID`` part in ``PLAYWRIGHT_USERNAME_TEMPLATE`` with the
-     corresponding user ID. If users with such usernames do not exist, set ``CREATE_USERS`` to ``true`` on the
-     ``playwright.env`` file for users to be created during the setup stage. If users with the same usernames but
-     different user roles already exist, change the user IDs to different values to ensure that new users are created
-     with roles defined in the configuration.
+     Playwright tests require users with different roles to simulate concurrent user interactions. If you already
+     have generated test users, you can skip this step. Generate users with the help of the user creation scripts under the
+     `supportingScripts/playwright` folder:
+
+    .. code-block:: bash
+
+        setupUsers.sh
+
+    You can configure user IDs and check their corresponding user roles in the ``src/test/playwright/support/users.ts`` file.
+    Usernames are defined automatically by appending the userId to the ``PLAYWRIGHT_USERNAME_TEMPLATE``.
+    At the moment it is discouraged to change the template string, as the user creation script does not support other names yet.
 
 4. Setup Playwright package and its browser binaries:
 
-    Install Playwright browser binaries, set up the environment to ensure Playwright can locate these binaries, and
-    create test users (if creating users is enabled in the configuration) with the following command:
+    Install Playwright browser binaries, set up the environment to ensure Playwright can locate these binaries.
+    On some operating systems this might not work, and playwright needs to be manually installed via a package manager.
 
     .. code-block:: bash
 
-        npm run playwright:setup
+        npm run playwright:setup-local
+        npm run playwright:init
+
+
 
 5. Open Playwright UI
 

src/test/playwright/package.json

@@ -20,6 +20,8 @@
         "playwright:test:sequential": "cross-env PLAYWRIGHT_JUNIT_OUTPUT_NAME=./test-reports/results-sequential.xml playwright test e2e --project=sequential-tests --workers 1",
         "playwright:open": "playwright test e2e --ui",
         "playwright:setup": "npx playwright install --with-deps chromium && playwright test init",
+        "playwright:setup-local": "npx playwright install --with-deps chromium",
+        "playwright:init": "playwright test init",
         "merge-reports": "junit-merge ./test-reports/results-parallel.xml ./test-reports/results-sequential.xml -o ./test-reports/results.xml",
         "update": "ncu -i --format group"
     }

src/test/playwright/playwright.env

@@ -1,13 +1,12 @@
-PLAYWRIGHT_USERNAME_TEMPLATE=artemis_test_user_USERID
-PLAYWRIGHT_PASSWORD_TEMPLATE=artemis_test_user_USERID
+PLAYWRIGHT_USERNAME_TEMPLATE=artemis_test_user_
+PLAYWRIGHT_PASSWORD_TEMPLATE=artemis_test_user_
 ADMIN_USERNAME=artemis_admin
 ADMIN_PASSWORD=artemis_admin
 ALLOW_GROUP_CUSTOMIZATION=true
 STUDENT_GROUP_NAME=students
 TUTOR_GROUP_NAME=tutors
 EDITOR_GROUP_NAME=editors
 INSTRUCTOR_GROUP_NAME=instructors
-CREATE_USERS=true
 BASE_URL=http://localhost:9000
 EXERCISE_REPO_DIRECTORY=test-exercise-repos
 FAST_TEST_TIMEOUT_SECONDS=45

supporting_scripts/playwright/README.md

@@ -0,0 +1,27 @@
+# Easy Artemis set up and running playwright locally
+
+Running playwright locally involves three steps:
+1. Run an Artemis application instance, with client and server.
+2. If no users have been set up, set up users.
+3. Install and run playwright.
+
+Note: run all setup scripts from within this directory. (cd supporting_scripts/playwright)
+
+## 1. Start Artemis
+
+To start Artemis, depending on your OS, either run `runArtemisInDocker_macOS.sh` or `runArtemisInDocker_linux.sh`.
+This will set up the database, start Artemis inside a docker container, and start the client via npm.
+After this step, you are be able to access Artemis locally as you usually would be.
+Note that you need to run the scripts in step 2 and 3 in another shell, as the client needs to keep running.
+In case you stop the client, you can simply re-run it at the root of the Artemis project with `npm run start`.
+
+## 2. Setup users
+
+Playwright needs users for it's tests. If you do not have users set up, you can simply do so by running:
+`setupUsers.sh`
+This will create 20 test users.
+
+## 3. Setup Playwright and run Playwright in UI-mode
+
+Simply run: `startPlaywright.sh`. This will install the necessary dependencies for playwright and start it in UI mode.
+If you already have playwright installed, you can also start playwright directly from the `src/test/playwright` directory with `npm run playwright:open`.

supporting_scripts/playwright/runArtemisInDocker_linux.sh

@@ -0,0 +1,22 @@
+#!/bin/sh
+
+set -e
+
+cd ../..
+
+echo "Copied configuration"
+
+cd docker
+
+echo "Updating docker group ID in the docker compose file"
+sed -i "s/999/$(getent group docker | cut -d: -f3)/g" artemis-dev-local-vc-local-ci-mysql.yml
+
+docker compose -f artemis-dev-local-vc-local-ci-mysql.yml up -d
+echo "Finished docker compose"
+
+cd ..
+
+echo "Installing Artemis npm dependencies and start Artemis client"
+
+npm install
+npm run start

supporting_scripts/playwright/runArtemisInDocker_macOS.sh

@@ -0,0 +1,26 @@
+#!/bin/sh
+
+set -e
+
+cd ../..
+
+cd docker
+open -a Docker
+
+echo "Updating docker group ID in the docker compose file"
+PRIMARY_GROUP_ID=$(dscl . -read /Groups/docker PrimaryGroupID | awk '{print $2}')
+if [ -n "$PRIMARY_GROUP_ID" ]; then
+    sed -i '' "s/999/$PRIMARY_GROUP_ID/g" artemis-dev-local-vc-local-ci-mysql.yml
+else
+    echo "PrimaryGroupID not found, skipping replacement"
+fi
+
+docker compose -f artemis-dev-local-vc-local-ci-mysql.yml up -d
+echo "Finished docker compose"
+
+cd ..
+
+echo "Installing Artemis npm dependencies and start Artemis client"
+
+npm install
+npm run start

supporting_scripts/playwright/setupUsers.sh

@@ -0,0 +1,18 @@
+#!/bin/sh
+
+# We use the supporting scripts to create users
+
+set -e
+
+cd ..
+
+if [ ! -d "venv" ]; then
+    python -m venv venv
+fi
+
+source venv/bin/activate
+
+cd  course-scripts/quick-course-setup
+
+python3 -m pip install -r requirements.txt
+python3 create_users.py

supporting_scripts/playwright/startPlaywright.sh

@@ -0,0 +1,16 @@
+#!/bin/sh
+
+set -e
+
+cd ../..
+
+echo "Installing Playwright and dependencies"
+
+cd src/test/playwright
+
+npm install
+
+npm run playwright:setup-local || true
+
+echo "Starting Playwright in UI mode"
+npm run playwright:open