Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[minor] Review fvt-launcher design for manage and its dependencies #1025

Merged
merged 31 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
bc1cda6
[minor] Replace wait-for-install task
Jun 6, 2024
5891234
[patch] Remove fvt-wait-for-install
Jun 6, 2024
80001a4
[patch] Missing parameter
Jun 6, 2024
9aed9c9
Merge branch 'master' into synclayer4
alequint Jun 7, 2024
068d552
Merge branch 'master' into synclayer4
alequint Jun 7, 2024
6ba79c7
[patch] Fix
Jun 7, 2024
33a9da8
Merge branch 'synclayer4' of github.com:ibm-mas/cli into synclayer4
Jun 7, 2024
685eccb
Merge branch 'master' into synclayer4
alequint Jun 12, 2024
b6d2ab4
Merge branch 'master' of github.com:ibm-mas/cli into syncpipe
Jun 13, 2024
517a838
[minor] IVT launcher changes (#1014)
unnati-solanki-git Jun 13, 2024
6af6fd8
[patch] Typos
Jun 13, 2024
5aecee8
[patch] Resolving conflicts with launchivt PR
Jun 13, 2024
6dbad46
[patch] Structure maange-is pipeline in phases
Jun 13, 2024
6d23b86
[minor] Added launchfvt-is in fvt-launcher
Jun 13, 2024
5f2957a
[minor] Move manage-setup to fvt-launcher
Jun 14, 2024
ad56876
[patch] Add escape route to waitfor-installation task
Jun 14, 2024
a3e3243
[patch] Remove addressed TODO comments
Jun 14, 2024
1ea05e0
[minor] Modified wait-for-config to receive more than one cm target
Jun 14, 2024
62a6cf3
[patch] Add Succeeded as possible target in post-install
Jun 14, 2024
0831827
[patch] Update parameter description
Jun 14, 2024
1976e2d
[patch] Added leo-miran
Jun 14, 2024
f8151f3
[patch] Fix
Jun 14, 2024
fd2ed86
[patch] Fix criteria to run waitfor-manage
Jun 14, 2024
714afea
[patch] Fix tekton build filters
Jun 14, 2024
6cc22cc
[patch] Debug
Jun 14, 2024
3948a6e
[patch] Text
Jun 14, 2024
4cc6b1c
[patch] Fix secret name in manage-is task
Jun 14, 2024
0909d5d
Merge branch 'master' into syncpipe
alequint Jun 14, 2024
f04377b
[patch] Fix criteria of approve-manage task
Jun 14, 2024
cd7c31d
Merge branch 'syncpipe' of github.com:ibm-mas/cli into syncpipe
Jun 14, 2024
1af5d8c
Merge branch 'master' into syncpipe
unnati-solanki-git Jun 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* @durera @andrercm @sanjayprab @terenceq @whitfiea @alequint
* @durera @andrercm @sanjayprab @terenceq @whitfiea @alequint @leo-miran
4 changes: 2 additions & 2 deletions build/bin/build-tekton.sh
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ for FILE in $TASK_FILES; do
FILE_NAME=$(basename $FILE)
addToFile $FILE $TARGET_FILE_FVT

if [[ ! "$FILE_NAME" == fvt-* ]]
if [[ ! "$FILE_NAME" == fvt-* ]] && [[ ! "$FILE_NAME" == ivt-* ]] && [[ ! "$FILE_NAME" == launchfvt-* ]]
then
addToFile $FILE $TARGET_FILE
fi
Expand All @@ -69,7 +69,7 @@ for FILE in $PIPELINE_FILES; do
FILE_NAME=$(basename $FILE)
addToFile $FILE $TARGET_FILE_FVT

if [[ ! "$FILE_NAME" == fvt-* ]] && [[ ! "$FILE_NAME" == *-after-install.yaml ]] && [[ ! "$FILE_NAME" == *-with-fvt.yaml ]]
if [[ ! "$FILE_NAME" == fvt-* ]] && [[ ! "$FILE_NAME" == ivt-* ]] && [[ ! "$FILE_NAME" == *-after-install.yaml ]] && [[ ! "$FILE_NAME" == *-with-fvt.yaml ]]
then
addToFile $FILE $TARGET_FILE
fi
Expand Down
21 changes: 18 additions & 3 deletions image/cli/app-root/src/wait-for-configmap.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,21 @@ do
esac
done

echo ""
echo "Inputs"
echo "------------------------------------------------------------------"
echo "Namespace .................. $NAMESPACE"
echo "Config Map ................. $CONFIGMAP_NAME"
echo "Config Map Key ............. $CONFIGMAP_KEY"
echo "Config Map Initial Value.... $CONFIGMAP_INITIAL_VALUE"
echo "Config Map Target Value..... $CONFIGMAP_TARGET_VALUE"
echo "Escape Config Map .......... $ESCAPE_CONFIGMAP_NAME"
echo "Escape Config Map Key ...... $ESCAPE_CONFIGMAP_KEY"
echo "Max Retries ................ $MAX_RETRIES"
echo "Delay ...................... $DELAY"
echo "Ignore Failure ............. $IGNORE_FAILURE"
echo ""

if [[ -z "$CONFIGMAP_NAME" || -z "$CONFIGMAP_KEY" || -z "$NAMESPACE" ]]; then
echo "NAMESPACE, CONFIGMAP_NAME, and CONFIGMAP_KEY must all be defined, there is nothing to wait for."
exit 0
Expand Down Expand Up @@ -128,12 +143,12 @@ elif [[ "$KEY_VALUE" == "" && -z "${CONFIGMAP_TARGET_VALUE}" ]]; then
else
echo "Waiting for configmap/${CONFIGMAP_NAME} in ${NAMESPACE} to contain key '${CONFIGMAP_KEY}' with value '${CONFIGMAP_TARGET_VALUE}' ..."
KEY_VALUE=$(oc -n ${NAMESPACE} get configmap/${CONFIGMAP_NAME} -o jsonpath="{.data.${CONFIGMAP_KEY}}" 2> /dev/null)
while [[ "$KEY_VALUE" != "${CONFIGMAP_TARGET_VALUE}" && "$RETRIES_USED" -le "$MAX_RETRIES" ]]; do
while [[ "${CONFIGMAP_TARGET_VALUE}" != *"$KEY_VALUE"* && "$RETRIES_USED" -le "$MAX_RETRIES" ]]; do
echo "[$RETRIES_USED/$MAX_RETRIES] ${CONFIGMAP_KEY}=${KEY_VALUE} does not equal '${CONFIGMAP_TARGET_VALUE}' yet in configmap/${CONFIGMAP_NAME}. Waiting ${DELAY} seconds before checking again"
sleep $DELAY
KEY_VALUE=$(oc -n ${NAMESPACE} get configmap/${CONFIGMAP_NAME} -o jsonpath="{.data.${CONFIGMAP_KEY}}" 2> /dev/null)

if [[ "${KEY_VALUE}" != "${CONFIGMAP_TARGET_VALUE}" && -n "${ESCAPE_CONFIGMAP_NAME}" ]]; then
if [[ "${CONFIGMAP_TARGET_VALUE}" != *"${KEY_VALUE}"* && -n "${ESCAPE_CONFIGMAP_NAME}" ]]; then
# Check if the entire install pipeline has stopped so we can exit early
ESCAPE_VALUE=$(oc -n ${NAMESPACE} get configmap/${ESCAPE_CONFIGMAP_NAME} -o jsonpath="{.data.${ESCAPE_CONFIGMAP_KEY}}" 2> /dev/null)
if [[ "$ESCAPE_VALUE" != "" ]]; then
Expand All @@ -148,7 +163,7 @@ else
done

echo
if [[ "$KEY_VALUE" == "${CONFIGMAP_TARGET_VALUE}" ]]; then
if [[ "${CONFIGMAP_TARGET_VALUE}" == *"$KEY_VALUE"* ]]; then
echo "Located key ${CONFIGMAP_KEY} in configmap/${CONFIGMAP_NAME} with value '${CONFIGMAP_TARGET_VALUE}'"
exit 0
else
Expand Down
59 changes: 59 additions & 0 deletions image/cli/masfvt/ivt.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
- hosts: localhost
any_errors_fatal: true
vars:
# Image Pull Policy
image_pull_policy: "{{ lookup('env', 'IMAGE_PULL_POLICY') }}"
# MAS Details
mas_app_channel_manage: "{{ lookup('env', 'MAS_APP_CHANNEL_MANAGE') }}"
mas_app_channel_monitor: "{{ lookup('env', 'MAS_APP_CHANNEL_MONITOR') }}"
mas_instance_id: "{{ lookup('env', 'MAS_INSTANCE_ID') }}"
mas_workspace_id: "{{ lookup('env', 'MAS_WORKSPACE_ID') }}"
# FVT Configuration
fvt_image_registry: "{{ lookup('env', 'FVT_IMAGE_REGISTRY') }}"
fvt_artifactory_username: "{{ lookup('env', 'FVT_ARTIFACTORY_USERNAME') }}"
fvt_artifactory_token: "{{ lookup('env', 'FVT_ARTIFACTORY_TOKEN') }}"
fvt_digest_manage: "{{ lookup('env', 'FVT_DIGEST_MANAGE') }}"
fvt_digest_manage_pytest: "{{ lookup('env', 'FVT_DIGEST_MANAGE_PYTEST') }}"
fvt_digest_ctf: "{{ lookup('env', 'FVT_DIGEST_CTF') }}"
# Black and white listing
fvt_blacklist: "{{ lookup('env', 'FVT_BLACKLIST') }}"
fvt_whitelist: "{{ lookup('env', 'FVT_WHITELIST') }}"
# Pipeline Run Info
devops_build_number: "{{ lookup('env', 'DEVOPS_BUILD_NUMBER') | default('0', True) }}"
pipelinerun_name: "{{ lookup('env', 'PIPELINERUN_NAME') | default('mas-fvt-manage', True) }}-{{ devops_build_number }}"
pipelinerun_namespace: "{{ lookup('env', 'PIPELINERUN_NAMESPACE') | default('mas-' ~ mas_instance_id ~ '-pipelines', True) }}"
# Test Data
ldap_url: "{{ lookup('env', 'LDAP_URL') }}"
ldap_base_dn: "{{ lookup('env', 'LDAP_BASE_DN') }}"
ldap_bind_dn: "{{ lookup('env', 'LDAP_BIND_DN') }}"
ldap_bind_password: "{{ lookup('env', 'LDAP_BIND_PASSWORD') }}"
ldap_user_map: "{{ lookup('env', 'LDAP_USER_MAP') }}"
ldap_cert_alias: "{{ lookup('env', 'LDAP_CERT_ALIAS') }}"
ldap_crt: "{{ lookup('env', 'LDAP_CRT') }}"
tasks:
- name: "Debug"
debug:
msg:
- "pipelinerun_name ................ {{ pipelinerun_name }}"
- "pipelinerun_namespace ........... {{ pipelinerun_namespace }}"
- ""
- "mas_app_channel_monitor ......... {{ mas_app_channel_monitor }}"
- "mas_app_channel_manage .......... {{ mas_app_channel_manage }}"
- "mas_instance_id ................. {{ mas_instance_id }}"
- "mas_workspace_id ................ {{ mas_workspace_id }}"
- ""
- "fvt_image_registry .............. {{ fvt_image_registry }}"
- "fvt_artifactory_username ........ {{ fvt_artifactory_username }}"
- "fvt_artifactory_token ........... {{ fvt_artifactory_token }}"
- "fvt_digest_manage ............... {{ fvt_digest_manage }}"
- "fvt_digest_manage_pytest ........ {{ fvt_digest_manage_pytest }}"
- "fvt_digest_ctf .................. {{ fvt_digest_ctf }}"
- ""
- "fvt_blacklist ................... {{ fvt_blacklist }}"
- "fvt_whitelist ................... {{ fvt_whitelist }}"

- name: "Start fvt-manage pipeline"
kubernetes.core.k8s:
apply: true
template: templates/mas-ivt.yml.j2
58 changes: 58 additions & 0 deletions image/cli/masfvt/templates/mas-ivt.yml.j2
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: "{{ pipelinerun_name }}"
namespace: "{{ pipelinerun_namespace }}"
labels:
tekton.dev/pipeline: mas-ivt
spec:
pipelineRef:
name: mas-ivt

serviceAccountName: pipeline
timeouts:
pipeline: "8h"

params:
# Pull Policy
- name: image_pull_policy
value: "{{ image_pull_policy }}"

# MAS Info
- name: mas_app_channel_manage
value: "{{ mas_app_channel_manage }}"
- name: mas_instance_id
value: "{{ mas_instance_id }}"
- name: mas_workspace_id
value: "{{ mas_workspace_id }}"
- name: mas_app_channel_monitor
value: "{{ mas_app_channel_monitor }}"

# Registry
- name: fvt_image_registry
value: "{{ fvt_image_registry }}"
- name: fvt_artifactory_username
value: "{{ fvt_artifactory_username }}"
- name: fvt_artifactory_token
value: "{{ fvt_artifactory_token }}"

# Digests of manage
- name: fvt_digest_manage
value: "{{ fvt_digest_manage }}"
- name: fvt_digest_manage_pytest
value: "{{ fvt_digest_manage_pytest }}"

# Digests of monitor
- name: fvt_digest_ctf
value: "{{ fvt_digest_ctf }}"

workspaces:
# The generated configuration files
- name: shared-configs
persistentVolumeClaim:
claimName: config-pvc
# PodTemplates configurations
- name: shared-pod-templates
secret:
secretName: pipeline-pod-templates
1 change: 1 addition & 0 deletions tekton/generate-tekton-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
- rollback
- uninstall
- fvt-launcher
- ivt

# 2. Generate Pipelines
# -------------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion tekton/generate-tekton-tasks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@
- fvt-sls
- fvt-data-dictionary
- fvt-run-suite
- fvt-wait-for-install

- name: Generate Tasks (FVT Launchers)
ansible.builtin.template:
Expand All @@ -86,6 +85,7 @@
- launchfvt-visualinspection
- launchfvt-upgrade-post
- launchfvt-upgrade-pre
- launchivt

# 4. Generate Tasks (IVT)
# -------------------------------------------------------------------------
Expand Down
64 changes: 0 additions & 64 deletions tekton/src/params/fvt.yml.j2

This file was deleted.

Loading
Loading