From 9ce55edd36ee7f8f3f8c9a0adcac7b74d62b6bc7 Mon Sep 17 00:00:00 2001 From: Augusto Date: Wed, 4 Dec 2024 13:18:01 +0100 Subject: [PATCH] First version of the README with inputs-ouputs detailed --- README.md | 187 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 187 insertions(+) diff --git a/README.md b/README.md index 5d5a6e4..3ab813e 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,193 @@ The RETORCH framework provides a tool that generates an Execution Plan, along wi files for execution in a CI environment. The generation of scripts and pipelining code is based on the Access Modes annotated within the test cases and the Resource information specified in `/retorchfiles/[SUT_NAME]SystemResources.json`. +The RETORCH orchestration tool requires 4 inputs: +- The annotated E2E test cases with the [RETORCH access modes](#retorch-annotations). +- A file with the Resources in JSON format. +- A properties file with the Environment configuration. +- A custom `docker-compose.yml` file. + +Give this inputs, the tool generates as output the necessary scripting code and the `Jenkinsfile` to execute the E2E test +suite into a Continuous Integration system. + +### Prepare the E2E Test suite +To execute the RETORCH Orchestration tool and generate the script and pipelining code, requires to perform series of configurations +into the test suite. The first step is create several folders to store the configurations and place the docker-compose.yml +in the root of the repository. +The resulting directory tree should look like as: +``` +. +├── src +├── docker-compose.yml +├── retorchfiles/ +│ ├── configurations/ +│ └── customscriptscode/ + +``` +- The `retorchfiles/` directory would contain all the configuration files and scripting snippets that would be used to generate the +pipelining code as well as the scripts to set-up, deploy and tear-down the different Resources and TJob. Contains two subdirectories: + - `configurations/`: contains the Resource and CI configuration files + - `customscriptscode/`: stores the different script snippets for the tear-down, set-up and environment. +- The `docker-compose.yml` in the root of the directory. +- The different project directories-files. + +The following subsections explain how to create each configuration file and how to prepare the docker-compose.yml file. + +#### Create the Resource.json file +The Resource file must be placed in the `retorchfiles/configurations/` and named with the system or test suite name, followed +by `SystemResources.json`. This file contains a Map with a series of Resources, using as key their unique ResourceID. For each +Resource the tester needs specify the following attributes: +- `resourceID`: A unique identifier for the Resource. +- `replaceable`: A list of Resources that can replace the current one. +- `hierarchyParent`: A resourceID of the hierarchical parent of the Resource. +- `elasticityModel`: The elasticity model of the Resource, is composed by the following attributes: + - `elasticityID`: A unique identifier for the elasticity model. + - `elasticity`: Integer with the available Resources + - `elasticityCost`: Instantiation cost of each Resource +- `resourceType`: String with the type of the Resource(e.g. LOGICAL, PHYSICAL or COMPUTATIONAL) +- `minimalCapacities`: List with the Minimal Capacities required by the Resource, each Capacity is composed by: + - `name`: String between "memory","processor" and "storage" + - `quantity`: float with the amount fo Capacity Required: +- `dockerImage`: String with the concatenation of the placeholder name in the docker-compose, "[IMG:]" and the image name + +The following snippet shows an example of two Resources declared in the JSON file: + +```json +{ + "userservice": { + "hierarchyParent": ["mysql"], + "replaceable": [], + "elasticityModel": {"elasticityID": "elasmodeluserservice", "elasticity": 5, "elasticityCost": 30.0}, + "resourceType": "LOGICAL", "resourceID": "userservice", + "minimalCapacities": [ + {"name": "memory", "quantity": 0.2929}, + {"name": "processor", "quantity": 0.2}, + {"name": "storage", "quantity": 0.5}], + "dockerImage": "userservice[IMG:]wigo4it/identityserver4:latest" +}, + "frontend": { + "hierarchyParent": [], "replaceable": [], + "elasticityModel": {"elasticityID": "elasmodelfrontend", "elasticity": 1, "elasticityCost": 300.0}, + "resourceType": "LOGICAL", "resourceID": "frontend", + "minimalCapacities": [ + {"name": "memory", "quantity": 2}, + {"name": "processor", "quantity": 1}, + {"name": "storage", "quantity": 0.88}], + "dockerImage": "frontend[IMG:]nginx:latest" + } + +} +``` + +#### Create the retorchCI.properties file +The CI file must be placed in `retorchfiles/configurations/`, namely `retorchCI.properties` contain several parameters +related to the SUT and the Continuous Integration Infrastructure, has the following parameters: +- `agentCIName`: Specific Jenkins agent used to execute the test suite. +- `sut-wait-html`: State in the frontend (html displayed) when the SUT is ready for execute the test SUITE. +- `sut-location`: Location of the docker-compose file used to deploy the SUT. +- `docker-frontend-name`: ID of the container used as frontend . +- `docker-frontend-port`: PORT on which the frontend is available. +- `external-binded-port`: EXTERNAL PORT where the frontend is made available (if its available). +- `external-frontend-url`: EXTERNAL URI where the frontend is made available. +- `testsBasePath`: Path to the java project. + +The following snippet provides an example of how this file looks like: + +```properties + agentCIName=any + sut-wait-html=Hello World + sut-location=$WORKSPACE + docker-frontend-name=https://sutexample- + docker-frontend-port=5000 + external-binded-port= + external-frontend-url= + testsBasePath=./ +``` + +#### Preparing the docker-compose.yml file +The RETORCH tool also requires to parametrize the docker-compose used to deploy the application, by means include the +necessary environment variables in the containers names and URIs, as well as the placeholders of the images above specified. +The next code present how was done in one of the services of the [FullTeaching Test Suite](https://github.com/giis-uniovi/retorch-st-fullteaching): + +```diff +services: + full-teaching-mysql: +- container_name: full-teaching-mysql ++ container_name: full-teaching-mysql-${tjobname} + ... ++ networks: ++ - jenkins_network + + full-teaching-openvidu-server-kms: +- container_name: full-teaching-openvidu-server-kms +- image: openvidu/openvidu-server-kms:1.7.0 ++ container_name: full-teaching-${tjobname}-openvidu-server-kms ++ image: ${mediaserver} + ... + environment: +- - openvidu.publicurl=https://full-teaching-openvidu-server-kms:8443 ++ - openvidu.publicurl=https://full-teaching-${tjobname}-openvidu-server-kms:8443 ++ networks: ++ - jenkins_network + + full-teaching: +- container_name: full-teaching ++ container_name: full-teaching-${tjobname} + ... + environment: +- - WAIT_HOSTS=full-teaching-mysql:3306 ++ - WAIT_HOSTS=full-teaching-mysql-${tjobname}:3306 +- - MYSQL_PORT_3306_TCP_ADDR=full-teaching-mysql ++ - MYSQL_PORT_3306_TCP_ADDR=full-teaching-mysql-${tjobname} +- - openvidu.url=https://full-teaching-openvidu-server-kms:8443 ++ - openvidu.url=https://full-teaching-${tjobname}-openvidu-server-kms:8443 + - ... ++ networks: ++ - jenkins_network + ++ networks: ++ jenkins_network: ++ external: true +``` + +#### (Optional) Specify script snippets to include in the set-up tear-down and environment +The RETORCH orchestration tool allows to specify scripting code/commands to be included in the generated set-up, tear-down and +the environment declaration of each TJob. To include it, the tester must create the following files in `retorch\customscriptscode` +- `custom-tjob-setup`: Contains the custom set-up code (e.g. deploy the SUT) or custom logging systems. +- `custom-tjob-teardown`: Contains the custom tear-down code (e.g. tear-down the SUT) +- `custom.env`: Contains environment variables common to all TJobs + +Examples of the three snippets files can be consulted in [FullTeaching Test Suite](https://github.com/giis-uniovi/retorch-st-fullteaching) +and [eShopOnContainers](https://github.com/giis-uniovi/retorch-st-eShopContainers) + +Once created the different properties and configuration files, the tree directory might look like: + +``` +. +├── src +├── docker-compose.yml +├── retorchfiles/ +│ ├── configurations/ +│ │ ├── [SUTNAME]SystemResource.json +│ │ └── retorchCI.properties +│ └── customscriptscode/ + ├── custom-tjob-setup + ├── custom-tjob-teardown + └── custom.env +``` + +### Executing the Orchestration tool +[TO-DO] Pending to decide-implement the final version + +### RETORCH Orchestration Tool outputs +The tool provides four different outputs, the pipelining code, the necessary scripts to set-up, tear-down and execute the TJobs(`/retorchfiles/tjoblifecycles`), +the infrastructure(`/retorchfiles/tjoblifecycles`) and the different environment files of each TJob (`/retorchfiles/envfiles`) : +- `Jenkinsfile`: located in the root of the project, contains the pipelining code with the different stages in sequential-parallel +that perform the set-up, execute and tear-down the TJobs. +- `/retorchfiles/tjoblifecycles` and `/retorchfiles/coilifecycles` contains the set-up, execution and tear-down scripts for the TJobs and infrastructure +- `/retorchfiles/envfiles`: contains the generated custom environment of each TJob. + + ## Contributing See the general contribution policies and guidelines for *giis-uniovi* at