Configuration for new environment (#55)

* Fix loss (#1668) * Fix loss * Add test * Add annotator stats panel (#1662) * Add delete button & modify locked/rejected status (#1673) * Fix #1671 (#1675) * first commit for the cicd * add deploy and dockerfile * maj ci * feat: ci try in re7 for label * fix: remove comment to comply with gitlab-ci.yml syntax * test: use echos for https proxy * fix: remove backend and client from project name on push * Ajout de l'étape de compilation du front * Ajout proxy pour yarn * Correction gitlab-ci.yml * Correction gitlab-ci.yml * Tentative build * utilisation de node 14 * feat: try to emulate judilibre-ops process * fix: add update in dind build * fix: typo in node name * fix: add yarn in addition of nodejs * fix: add git to packages * debug: ls content after compile * fix: try changing build path * fix: add login to registry + docker tag for runner * feat: add services for label client and backend * feat: add deploy namespace label * fix: indent * fix: change port for label client * fix: service name errror * fix: use "ugly workaround" to make build work * feat: partially add env variables for label deploy * feat: make backend work in local. TODO : rework prodEnvironment ugly mechanism * feat: initiate mongo from env var instead of json * feat: add jobs deployment * fix: typo in gitlab-ci.yml * fix: move comments for yaml syntax * fix: yaml syntax * fix: temporarily disable backend-client build and deploy * fix: add env var and rules for jobs deploys * test: force redeploy * fix: try escaping job schedules * fix: add quotes to avoid yaml treatment for sched * fix: temporarily disable volume mount config on label * fix: disable volume mounts pt2 * fix: add target for build * fix: add shell tag for label backend * feat: add yarn compile for backend * fix: use docker for build label backend * fix: try node options openssl * fix: apk add npm * fix: add CI false to test yarn build * fix: put back fixed frontend build * feat: add mongodb url in deploy * fix: add quotes for deployment variables * fix: try without curly brackets for mongodb var * fix: try mongodb url withour quotees * test: env var * fix: put back mongodb url * test: test variable on gitlab ci * fix: put back mognodburl * clean: clean code * feat: add connection to nlp pseudonymisation service * fix: use preprod url for label client * fix: add root path in addition of public url * feat: add root path and public url on deploy too * Add summary (#6) * trigger GitHub actions * Add summary * Fix rejected documents (#7) * Remove monitoringEntries (#10) * Fix admin view (#11) * Improve docs (#15) * Add assignation date (#16) * Fix statistics box (#17) * Add ellipsis lines (#19) * Fix job export publishable (#21) * Change route after annotation (#22) * Add importer field (#20) * Add importer field * Add importer field * Fix * Improve personal stats (#23) * Change manual import script (#24) * Clean unused features, remove deprecated & improve docs (#25) * Clean unused script * Remove deprecated * Improve docs & minor fixes * Fix local script * Remove couldBe feature * Remove csvExtractor * Fix INSTALL docs * Fix additionnal annotations (#26) * Fix loss (#28) * first commit for the cicd * add deploy and dockerfile * maj ci * feat: ci try in re7 for label * fix: remove comment to comply with gitlab-ci.yml syntax * test: use echos for https proxy * fix: remove backend and client from project name on push * Ajout de l'étape de compilation du front * Ajout proxy pour yarn * Correction gitlab-ci.yml * Correction gitlab-ci.yml * Tentative build * utilisation de node 14 * feat: try to emulate judilibre-ops process * fix: add update in dind build * fix: typo in node name * fix: add yarn in addition of nodejs * fix: add git to packages * debug: ls content after compile * Add juritj (#29) * Fix wording & doc (#18) * Fix wording * Fix doc * Fix anonymizer (#27) * Fix anonymizer * Add replacementTerm module * Fix anonymizer * Add replacementTerm module * Fix cron jobs (#30) * Display checklist (#32) * Display checklist * Fix * Fixes * Fixes * Lint * Improve error page (#33) * Adding priorities to statuses while deleting duplicated documents (#34) * Adding status priority to cleanDuplicated (ex cleanDoubled) * Updating test to assess priority in duplicates deletion * Linting * Fix priority by importer (#35) * Fix priority by importer * Add first mapCourtDecisionToDocument test * Fixes & improve docs (#37) * Fixes & improve docs * Fix docs * Send ids to NLP API (#38) * Send ids to NLP API * Key changes * Remove non public or empty NACCode check (#39) * fix: try changing build path * fix: add login to registry + docker tag for runner * feat: add services for label client and backend * feat: add deploy namespace label * fix: indent * fix: change port for label client * fix: service name errror * fix: use "ugly workaround" to make build work * feat: partially add env variables for label deploy * feat: make backend work in local. TODO : rework prodEnvironment ugly mechanism * feat: initiate mongo from env var instead of json * feat: add jobs deployment * fix: typo in gitlab-ci.yml * fix: move comments for yaml syntax * fix: yaml syntax * fix: temporarily disable backend-client build and deploy * fix: add env var and rules for jobs deploys * test: force redeploy * fix: try escaping job schedules * fix: add quotes to avoid yaml treatment for sched * fix: temporarily disable volume mount config on label * fix: disable volume mounts pt2 * fix: add target for build * fix: add shell tag for label backend * feat: add yarn compile for backend * fix: use docker for build label backend * fix: try node options openssl * fix: apk add npm * fix: add CI false to test yarn build * fix: put back fixed frontend build * feat: add mongodb url in deploy * fix: add quotes for deployment variables * fix: try without curly brackets for mongodb var * fix: try mongodb url withour quotees * test: env var * fix: put back mongodb url * test: test variable on gitlab ci * fix: put back mognodburl * clean: clean code * feat: add connection to nlp pseudonymisation service * fix: use preprod url for label client * fix: add root path in addition of public url * feat: add root path and public url on deploy too * feat: adapt ingresses * feat: deploy via ansible part 1 * feat: use dictionnary for branches env link * feat: add branch.env array in deploy * fix: not use array for link definition * fix: use full path for inventories * fix: adapt rules for label client * fix: typo in label playbook name * init ansible vault for dev and preprod * fix: deploy using inventories selected in case * feat: add deploy files in ansible * fix: yaml typo for disabling build * add public vriables * kube namespace variable to all * add secret * fix: put back build - hash commit image needed * feat: add configmap for label backend * feat: label client configmap init * update secrets * feat: add secrets for label back and client * fix: escape yaml variable * fix: nlp variable * fix: ansible vault file + env branches mapping * fix: typo * add db secret * complete sder database string * fix: add pre hooks for managing rsa keys * fix: update label kube namespace variable name * fix: inject good image variables * fix: nlp api url * fix: nlp api url * fix: string as number definition * fix: solve sder db url syntax * fix: try to suppress spaces in vault * test: test with fake mongodb name * fix: uniformize dev vault * test: echo inventaire * fix: use proper name for inventories * fix: encode with b64 all variable * fix: add ansible definition for deployments * fix: typo triple dash * fix: handle definition of ingresses and services * fix typo triple dash in ingress.yml * fix: triple dash in client ingress * feat: put back builds * feat: use jobs template + map secrets on containers * feat: reconfigure jobs deployment * fix: path for deployment jobs template * fix: error in filename for deployment jobs template * fix: use integers instead of strings for some values in job template * debug: disable build and test with only one job * fix: unquote integer values in jobs template * fix: try to use int as value in defaults.yml * fix: try with int * fix: try another way of casting int * fix: try without dictionary * fix: try to use values set instead of dict * fix: try to cast into int with the new way * test templating * mise à jour gitlab-ci * Correction syntaxe ansible * correction nom fichier * ajout autres jobs * correction gitlab-ci * feat: activate full build and deploy * fix: put good root pathes for label client * fix: change ingressroute override cause name * fix: update sder connection string * fix: fix NLP api URL * try to use correct URL for label in preprod (#52) * use correct host * change label to lbl --------- Co-authored-by: Antoine Jeanneney <[email protected]> --------- Co-authored-by: David Libeau <[email protected]> Co-authored-by: Ilias Aaguida <[email protected]> Co-authored-by: Axel Marbois <[email protected]> Co-authored-by: Benjamin PILIA <[email protected]> Co-authored-by: Paul Déchorgnat <[email protected]> Co-authored-by: Antoine Jeanneney <[email protected]> Co-authored-by: bepi-octo <[email protected]>
Cour-de-cassation · Nov 29, 2023 · 43798b9 · 43798b9
1 parent 000cccd
commit 43798b9
Show file tree

Hide file tree

Showing 42 changed files with 1,324 additions and 5 deletions.
diff --git a/.deploys/deploy_backend.yaml b/.deploys/deploy_backend.yaml
@@ -0,0 +1,116 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: label-backend-deployment
+  namespace: label
+  labels:
+    app: 
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: label-backend
+  template:
+    metadata:
+      labels:
+        app: label-backend
+    spec:
+      containers:
+        - name: label-backend
+          image: $CI_REGISTRY/cour-de-cassation/label:$CI_COMMIT_BRANCH-$CI_COMMIT_SHORT_SHA-label-backend
+          ports:
+            - containerPort: 55430
+          env:
+            - name: SDER_DB_URL
+              value: ${MONGODB_URI}:${MONGODB_PORT}
+            - name: MONGODB_URL
+              value: ${MONGODB_URL} # ici quand je mets "testaxel" ça s'affiche bien
+            - name: GIT_BRANCH
+              value: "${LABEL_GIT_BRANCH}"
+            - name: API_PORT
+              value: "${LABEL_API_PORT}"
+            - name: APP_HOST
+              value: "${LABEL_APP_HOST}"
+            - name: APP_GROUP
+              value: "${LABEL_APP_GROUP}"
+            - name: APP_ID
+              value: "${LABEL_APP_ID}"
+            - name: APP_KEYWORD
+              value: "${LABEL_APP_KEYWORD}"
+            - name: APP_SCHEME
+              value: "${LABEL_APP_SCHEME}"
+            - name: APP_SELF_SIGNED
+              value: "${LABEL_APP_SELF_SIGNED}"
+            - name: APP_PORT
+              value: "${LABEL_APP_PORT}"
+            - name: DOCKER_USERNAME
+              value: "${LABEL_DOCKER_USERNAME}"
+            - name: DOCKER_TARGET
+              value: "${LABEL_DOCKER_TARGET}"
+            - name: KUBE_ZONE
+              value: "${LABEL_KUBE_ZONE}"
+            - name: KUBE_TYPE
+              value: "${LABEL_KUBE_TYPE}"
+            - name: KUBE_NAMESPACE
+              value: "${LABEL_KUBE_NAMESPACE}"
+            - name: ROOT_PATH
+              value: "${LABEL_ROOT_PATH}"
+            - name: NLP_API_HOST
+              value: "http://${DOLLAR}(NLP_PSEUDONYMISATION_API_SERVICE_SERVICE_HOST)"
+            - name: NLP_API_PORT
+              value: "${DOLLAR}(NLP_PSEUDONYMISATION_API_SERVICE_PORT)"
+            - name: START_TIMEOUT
+              value: "${LABEL_START_TIMEOUT}"
+            - name: KUBECONFIG
+              value: "${LABEL_KUBECONFIG}"
+            - name: NLP_API_URL # penser à injecter nlp api url et dollar dans gitlab
+              value: "http://${DOLLAR}(NLP_PSEUDONYMISATION_API_SERVICE_SERVICE_HOST):${DOLLAR}(NLP_PSEUDONYMISATION_API_SERVICE_SERVICE_PORT)"
+          # startupProbe:
+          #   exec:
+          #     command:
+          #     - /bin/sh
+          #     - -c
+          #     - curl -s --fail --max-time 2 "http://localhost:55430/${ROOT_PATH}/health" | egrep -q 'true'
+          #   initialDelaySeconds: 15
+          #   periodSeconds: 5
+          #   failureThreshold: 20
+          # livenessProbe:
+          #   exec:
+          #     command:
+          #     - /bin/sh
+          #     - -c
+          #     - curl -s --fail --retry-delay 0 --max-time 2 --retry 30 --retry-max-time 60 "http://localhost:55430/${ROOT_PATH}/health" | egrep -q 'true'
+          #   initialDelaySeconds: 15
+          #   periodSeconds: 15
+          #   failureThreshold: 10
+          #   timeoutSeconds: 60
+          # securityContext:
+          #   runAsUser: 1000
+          #   capabilities:
+          #     drop:
+          #       - all
+          #   allowPrivilegeEscalation: false
+          #   readOnlyRootFilesystem: true
+      #     volumeMounts:
+      #       - name: label-backend-config
+      #         mountPath: /home/node/packages/courDeCassation/environments
+      #         readOnly: true
+      # volumes:
+      # - name: label-backend-config
+      #   secret:
+      #     secretName: label-backend-config
+
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: label-backend-service
+  namespace: label
+spec:
+  selector:
+    app: label-backend
+  ports:
+    - name: label-backend-port
+      port: 55430
+      targetPort: 55430
+  type: ClusterIP
diff --git a/.deploys/deploy_client.yaml b/.deploys/deploy_client.yaml
@@ -0,0 +1,90 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: label-client-deployment
+  namespace: label
+  labels:
+    app: label-client
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: label-client
+  template:
+    metadata:
+      labels:
+        app: label-client
+    spec:
+      containers:
+        - name: label-client
+          image: $CI_REGISTRY/cour-de-cassation/label:$CI_COMMIT_BRANCH-$CI_COMMIT_SHORT_SHA-label-client
+          ports:
+            - containerPort: 55432
+          env:
+            - name: GIT_BRANCH
+              value: "${LABEL_GIT_BRANCH}"
+            - name: API_PORT
+              value: "${LABEL_API_PORT}"
+            - name: APP_HOST
+              value: "${LABEL_APP_HOST}"
+            - name: APP_GROUP
+              value: "${LABEL_APP_GROUP}"
+            - name: APP_ID
+              value: "${LABEL_APP_ID}"
+            - name: APP_KEYWORD
+              value: "${LABEL_APP_KEYWORD}"
+            - name: APP_SCHEME
+              value: "${LABEL_APP_SCHEME}"
+            - name: APP_SELF_SIGNED
+              value: "${LABEL_APP_SELF_SIGNED}"
+            - name: APP_PORT
+              value: "${LABEL_APP_PORT}"
+            - name: DOCKER_USERNAME
+              value: "${LABEL_DOCKER_USERNAME}"
+            - name: DOCKER_TARGET
+              value: "${LABEL_DOCKER_TARGET}"
+            - name: KUBE_ZONE
+              value: "${LABEL_KUBE_ZONE}"
+            - name: KUBE_TYPE
+              value: "${LABEL_KUBE_TYPE}"
+            - name: KUBE_NAMESPACE
+              value: "${LABEL_KUBE_NAMESPACE}"
+            - name: ROOT_PATH
+              value: "${LABEL_ROOT_PATH}"
+            - name: PUBLIC_URL
+              value: "${LABEL_PUBLIC_URL}"
+            - name: START_TIMEOUT
+              value: "${LABEL_START_TIMEOUT}"
+            - name: KUBECONFIG
+              value: "${LABEL_KUBECONFIG}"
+          # livenessProbe:
+          #   exec:
+          #     command:
+          #     - /bin/sh
+          #     - -c
+          #     - curl -s -k --fail --max-time 2 "http://localhost:55432/" | egrep -q 'true'
+          #   initialDelaySeconds: 60
+          #   periodSeconds: 15
+          #   failureThreshold: 3
+          #   timeoutSeconds: 5
+          securityContext:
+            runAsUser: 1000
+            capabilities:
+              drop:
+                - all
+            allowPrivilegeEscalation: false
+            readOnlyRootFilesystem: true
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: label-client-service
+  namespace: label
+spec:
+  selector:
+    app: label-client
+  ports:
+    - name: label-client-port
+      port: 55432
+      targetPort: 55432
+  type: ClusterIP
diff --git a/.deploys/deploy_ingress.yaml b/.deploys/deploy_ingress.yaml
@@ -0,0 +1,61 @@
+---
+apiVersion: traefik.containo.us/v1alpha1
+kind: IngressRoute
+metadata:
+  name: ingress-route
+  namespace: label
+spec:
+  entryPoints:
+    - web
+
+    # regarder la gueule des ingress de prod
+
+  routes:
+    # - match: Host(`juritj.judilibre-prive.local`) && PathPrefix(`/label-client`)
+    #   kind: Rule
+    #   services:
+    #     - name: label-client-service
+    #       port: label-client-port
+    #   middlewares:
+    #     - name: label-client-stripprefix
+    # - match: Host(`juritj.judilibre-prive.local`) && PathPrefix(`/label-backend`)
+    #   kind: Rule
+    #   services:
+    #     - name: label-backend-service
+    #       port: label-backend-port
+    #   middlewares:
+    #     - name: label-backend-stripprefix
+    - match: Host(`lbl.judilibre-prive.local`) && PathPrefix(`/label`)
+      kind: Rule
+      services:
+        - name: label-client-service
+          port: label-client-port
+    - match: Host(`lbl.judilibre-prive.local`) && PathPrefix(`/label/api`)
+      kind: Rule
+      services:
+        - name: label-backend-service
+          port: label-backend-port
+
+---
+apiVersion: traefik.containo.us/v1alpha1
+kind: Middleware
+metadata:
+  name: label-client-stripprefix
+  namespace: label
+spec:
+  replacePathRegex:
+    regex: /label-client(/|$)(.*)
+    replacement: /$2
+
+---
+apiVersion: traefik.containo.us/v1alpha1
+kind: Middleware
+metadata:
+  name: label-backend-stripprefix
+  namespace: label
+spec:
+  replacePathRegex:
+    regex: /label-backend(/|$)(.*)
+    replacement: /$2
+
+
diff --git a/.deploys/deploy_jobs_template.yaml b/.deploys/deploy_jobs_template.yaml
@@ -0,0 +1,44 @@
+---
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: label-backend-job-${JOB_NAME}
+  namespace: label
+spec:
+  schedule: "${JOB_SCHEDULE}"
+  successfulJobsHistoryLimit: ${SUCCESSFUL_JOBS_HISTORY_LIMIT}
+  failedJobsHistoryLimit: ${FAILED_JOBS_HISTORY_LIMIT}
+  concurrencyPolicy: Forbid
+  jobTemplate:
+    spec:
+      backoffLimit: ${BACKOFF_LIMIT}
+      parallelism: ${PARALLELISM}
+      template:
+        spec:
+          activeDeadlineSeconds: ${ACTIVE_DEADLINE_SECONDS}
+          containers:
+            - name: label-backend-${JOB_NAME}
+              image: $CI_REGISTRY/cour-de-cassation/label:$CI_COMMIT_BRANCH-$CI_COMMIT_SHORT_SHA-label-backend
+              # volumeMounts:
+              #   - name: label-backend-config
+              #     mountPath: /home/node/packages/courDeCassation/environments
+              #     readOnly: true
+              command:
+                - /bin/sh
+              args:
+                - -c
+                - node ${COMMAND} -e environments/prodEnvironment.json -s settings/settings.json
+              env:
+                - name: RUN_MODE
+                  value: PROD
+                - name: SDER_DB_URL
+                  value: ${MONGODB_URI}:${MONGODB_PORT}
+                - name: MONGODB_URL
+                  value: "${MONGODB_URL}"
+                # - name: NODE_TLS_REJECT_UNAUTHORIZED
+                #   value: "0"
+          restartPolicy: Never
+          # volumes:
+          #   - name: label-backend-config
+          #     secret:
+          #       secretName: label-backend-config
diff --git a/.deploys/deploy_namespace.yaml b/.deploys/deploy_namespace.yaml
@@ -0,0 +1,5 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: label
+  namespace: label
diff --git a/.github/workflows/build-publish-deploy.yml b/.github/workflows/build-publish-deploy.yml
@@ -10,7 +10,7 @@ jobs:
     env:
       APP_ID: label-* # specific to backend or client
       APP_GROUP: judilibre-prive
-      APP_HOST: label.judilibre-prive.local
+      APP_HOST: lbl.judilibre-prive.local
       APP_PORT: 443
       APP_SCHEME: https
       APP_SELF_SIGNED: "true"
@@ -200,7 +200,7 @@ jobs:
     env:
       APP_ID: label-* # specific to backend or client
       APP_GROUP: judilibre-prive
-      APP_HOST: label.judilibre-prive.local
+      APP_HOST: lbl.judilibre-prive.local
       APP_PORT: 443
       APP_SCHEME: https
       APP_SELF_SIGNED: "true"

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -10,7 +10,7 @@ jobs:
     env:
       APP_ID: label-* # specific to backend or client
       APP_GROUP: judilibre-prive
-      APP_HOST: label.judilibre-prive.local
+      APP_HOST: lbl.judilibre-prive.local
       APP_PORT: 443
       APP_SCHEME: https
       APP_SELF_SIGNED: "true"