Skip to content

Commit 4a331b5

Browse files
laurentC35nsenavedependabot[bot]renovate[bot]
authored
Feat: use rmes for the metadata repository (#296)
* build: update dependencies (#232) * build(deps): bump org.springdoc:springdoc-openapi-ui from 1.7.0 to 1.8.0 (#231) Bumps org.springdoc:springdoc-openapi-ui from 1.7.0 to 1.8.0. --- updated-dependencies: - dependency-name: org.springdoc:springdoc-openapi-ui dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump org.sonarsource.scanner.maven:sonar-maven-plugin (#230) Bumps [org.sonarsource.scanner.maven:sonar-maven-plugin](https://github.com/SonarSource/sonar-scanner-maven) from 3.10.0.2594 to 3.11.0.3922. - [Release notes](https://github.com/SonarSource/sonar-scanner-maven/releases) - [Commits](SonarSource/sonar-scanner-maven@3.10.0.2594...3.11.0.3922) --- updated-dependencies: - dependency-name: org.sonarsource.scanner.maven:sonar-maven-plugin dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump version * chore(deps): bumb to spring 3.2.4 * ref: move config package to configuration * chore: ref swagger for open-api * chore: ref security (spring 3) * chore: delete old classes * Delete ClientConfig.java * build: update dependencies (#251) * chore(deps): update dependency org.jacoco:jacoco-maven-plugin to v0.8.12 (#247) * build(deps): bump commons-io:commons-io from 2.15.1 to 2.16.0 (#246) * build(deps): bump softprops/action-gh-release from 1 to 2 (#245) Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 1 to 2. - [Release notes](https://github.com/softprops/action-gh-release/releases) - [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md) - [Commits](softprops/action-gh-release@v1...v2) --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * feat(pogues model): dynamic table line controls (#252) * build: bump pogues-model version Include a new property for dynamic table controls. * build: add maven snapshot repository in pom * chore: webclient for Eno * log: remove rolling file * chore: delete useless annotations * chore: fix import of classes * deps: restore deps * build: clean deps * chore: add stamp to oidcProperties * build: add lombok config * chore: manage execption handling * chore: change to JSON from jackson * chore: change to slf4j * chore: remove useless class * build: change properties convention name * build: remove RollingFile from log * chore: change to webClient for EnoClient * fix: pdf generation * chore: change impl, remove file from generation -> InputStream * fix: eno calling according to String input * chore: remove useless classes * test: adapt tests to new implementation * chire: organize import * deps: bump jacoco.version & remove commons-io * chore: improve variables service * chore: clean import * chore: use Pogues-Model methods to serialize and Deserialize Questionnaire * deps: clean useless deps and use latest version of pogues-model of snapshot * fix: healthcheck with auth * chore: properties, remove redondant props * bump: version to 4.5.0-SNAPSHOT * chore: remove org.json.simple dep, replace by jackson * ref: properties as yaml * chore: remove useless log * chore: organize end-points and depreacted some (ddias for ex.) * Update application.yaml * build: fix deps for snapshots * chore: add logging props * chore: remove log4j.xml conf * bump: version to 4.6.0-SNAPSHOT * chore: restore PR 265 (#277) See #265 * fix: composition in CATI mode (#276) * fix: #270 (#275) * fix: pdf generation with resources (img & fonts) inside jar (#278) * fix: pdf generation with resources (img & fonts) inside jar * chore: organize import * bump: version to 4.6.1-SNAPSHOT * deps: fix cve-2024-22262 * ref: change ModelMapper to IOStreamsUtils * fix: disable cache for font (#279) * bump: version to 4.6.2-SNAPSHOT * chore: create Java model * fix: uri call for existDB (#280) * ref: simplify error handling (#281) * bump: version to 4.6.3-SNAPSHOT * feat: rmes create utils classes * ref: improve handle of external request with token of user (#282) * chore: create empty Service & Client for metadata (magma & ddiAS) * chore: rename class & delete unecessary method * chore: update converter for rmes entity to pogues entity (dataCollection) * chore: change interface (returning dataCollection entity) * chore: add implementation of client with WebClient * chore: adjust frequenceType enum according to real * feat: add metadata end-point for Pogues-fo * tests: fix test * feat: cache reponse from Rmes to speed up * bump: version to 4.7.0-SNAPSHOT * fix: schema for swagger * chore: fix schema for swagger * Bumpt to 4.7.0 --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Nicolas Sénave <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Nicolas Senave <[email protected]> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
1 parent 20c3365 commit 4a331b5

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+912
-1251
lines changed

docs/fr/workflow/docker/docker-usage.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ L'API Colectica contourne un certain nombre de principes des architectures REST
9090

9191
#### Réécriture d'URL
9292

93-
Pour prendre l'exemple de la resources /items, cette resource est définie par json-server à partir de la clef items définie dans le fichier db.json et de la collection associée.
93+
Pour prendre l'exemple de la resources /items, cette resource est définie par json-server à partir de la clef items définie dans le fichier db.json et de la dataCollection associée.
9494

9595
Pour faire correspondre cette url de resource à l'URL appelée en production on éditera le fichier routes.json de la manière suivante:
9696

pom.xml

+14-1
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
<groupId>fr.insee</groupId>
1414
<artifactId>Pogues-BO</artifactId>
1515
<packaging>jar</packaging>
16-
<version>4.6.8</version>
16+
<version>4.7.0</version>
1717
<name>Pogues-Back-Office</name>
1818

1919
<properties>
@@ -26,6 +26,8 @@
2626
<jacoco.version>0.8.12</jacoco.version>
2727
<saxon.version>12.4</saxon.version>
2828

29+
<caffeine.version>3.1.8</caffeine.version>
30+
2931
<sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
3032
<sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
3133
<sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
@@ -67,6 +69,17 @@
6769
<artifactId>spring-boot-starter-webflux</artifactId>
6870
</dependency>
6971

72+
<!-- Cache -->
73+
<dependency>
74+
<groupId>org.springframework.boot</groupId>
75+
<artifactId>spring-boot-starter-cache</artifactId>
76+
</dependency>
77+
<dependency>
78+
<groupId>com.github.ben-manes.caffeine</groupId>
79+
<artifactId>caffeine</artifactId>
80+
<version>${caffeine.version}</version>
81+
</dependency>
82+
7083
<!-- Actuator Metrics -->
7184
<dependency>
7285
<groupId>org.springframework.boot</groupId>

src/main/java/fr/insee/pogues/configuration/AppConfiguration.java

-5
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,9 @@
11
package fr.insee.pogues.configuration;
22

3-
import fr.insee.pogues.configuration.properties.ApplicationProperties;
4-
import fr.insee.pogues.configuration.rest.AuthenticationHelper;
5-
import fr.insee.pogues.configuration.rest.WebClientTokenInterceptor;
63
import lombok.extern.slf4j.Slf4j;
74
import org.springframework.beans.factory.annotation.Autowired;
85
import org.springframework.beans.factory.annotation.Value;
96
import org.springframework.boot.context.properties.EnableConfigurationProperties;
10-
import org.springframework.cache.annotation.EnableCaching;
117
import org.springframework.context.annotation.Bean;
128
import org.springframework.context.annotation.ComponentScan;
139
import org.springframework.context.annotation.Configuration;
@@ -21,7 +17,6 @@
2117
@EnableConfigurationProperties
2218
@ComponentScan(basePackages = { "fr.insee.pogues" })
2319
@EnableTransactionManagement
24-
@EnableCaching
2520
@Slf4j
2621
public class AppConfiguration {
2722

Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
package fr.insee.pogues.configuration.cache;
2+
3+
import com.github.benmanes.caffeine.cache.Caffeine;
4+
import org.springframework.beans.factory.annotation.Value;
5+
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
6+
import org.springframework.cache.annotation.EnableCaching;
7+
import org.springframework.cache.caffeine.CaffeineCache;
8+
import org.springframework.context.annotation.Bean;
9+
import org.springframework.context.annotation.Configuration;
10+
11+
import java.util.concurrent.TimeUnit;
12+
13+
@ConditionalOnProperty(name = "feature.cache.enabled", havingValue = "true")
14+
@Configuration
15+
@EnableCaching
16+
public class CacheConfig {
17+
18+
@Value("${feature.cache.retention-time-in-days.ddi-as}")
19+
private long ddiAsCacheRetentionTime;
20+
21+
@Value("${feature.cache.retention-time-in-days.magma}")
22+
private long magmaCacheRetentionTime;
23+
24+
@Bean
25+
protected CaffeineCache unitsCacheCache() {
26+
return new CaffeineCache(CacheName.UNITS,
27+
Caffeine.newBuilder()
28+
.maximumSize(10)
29+
.expireAfterWrite(ddiAsCacheRetentionTime, TimeUnit.DAYS)
30+
.build());
31+
}
32+
33+
@Bean
34+
protected CaffeineCache seriesCache() {
35+
return new CaffeineCache(CacheName.SERIES,
36+
Caffeine.newBuilder()
37+
.maximumSize(100)
38+
.expireAfterWrite(magmaCacheRetentionTime, TimeUnit.DAYS)
39+
.build());
40+
}
41+
@Bean
42+
protected CaffeineCache serieCache() {
43+
return new CaffeineCache(CacheName.SERIE,
44+
Caffeine.newBuilder()
45+
.maximumSize(100)
46+
.expireAfterWrite(magmaCacheRetentionTime, TimeUnit.DAYS)
47+
.build());
48+
}
49+
50+
@Bean
51+
protected CaffeineCache operationsCache() {
52+
return new CaffeineCache(CacheName.OPERATIONS,
53+
Caffeine.newBuilder()
54+
.maximumSize(10000)
55+
.expireAfterWrite(magmaCacheRetentionTime, TimeUnit.DAYS)
56+
.build());
57+
}
58+
@Bean
59+
protected CaffeineCache operationCache() {
60+
return new CaffeineCache(CacheName.OPERATION,
61+
Caffeine.newBuilder()
62+
.maximumSize(10000)
63+
.expireAfterWrite(magmaCacheRetentionTime, TimeUnit.DAYS)
64+
.build());
65+
}
66+
67+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
package fr.insee.pogues.configuration.cache;
2+
3+
public class CacheName {
4+
5+
private CacheName() {
6+
throw new IllegalArgumentException("Utility class");
7+
}
8+
9+
public static final String UNITS = "units";
10+
public static final String SERIES = "series";
11+
public static final String SERIE = "serie";
12+
public static final String OPERATIONS = "operations";
13+
public static final String OPERATION = "operation";
14+
}

src/main/java/fr/insee/pogues/configuration/properties/ApplicationProperties.java

-2
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,7 @@
44
import org.springframework.boot.context.properties.ConfigurationProperties;
55
import org.springframework.validation.annotation.Validated;
66

7-
import java.util.Arrays;
87
import java.util.List;
9-
import java.util.Objects;
108

119
@Validated
1210
@ConfigurationProperties(prefix = "application")
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
package fr.insee.pogues.metadata.client;
2+
3+
import fr.insee.pogues.metadata.model.ddias.Unit;
4+
5+
import java.util.List;
6+
7+
public interface DDIASClient {
8+
List<Unit> getUnits() throws Exception;
9+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
package fr.insee.pogues.metadata.client;
2+
3+
import fr.insee.pogues.metadata.model.ddias.Unit;
4+
import fr.insee.pogues.webservice.rest.PoguesException;
5+
import lombok.extern.slf4j.Slf4j;
6+
import org.springframework.beans.factory.annotation.Autowired;
7+
import org.springframework.beans.factory.annotation.Value;
8+
import org.springframework.http.MediaType;
9+
import org.springframework.stereotype.Service;
10+
import org.springframework.web.reactive.function.client.WebClient;
11+
import org.springframework.web.reactive.function.client.WebClientResponseException;
12+
import org.springframework.web.util.UriComponentsBuilder;
13+
14+
import java.net.URI;
15+
import java.util.List;
16+
17+
@Service
18+
@Slf4j
19+
public class DDIASClientImpl implements DDIASClient {
20+
21+
private static final String UNITS_PATH = "/meta-data/units";
22+
23+
@Autowired
24+
private WebClient webClient;
25+
26+
@Value("${application.metadata.ddi-as}")
27+
String ddiAsHost;
28+
29+
@Override
30+
public List<Unit> getUnits() throws Exception {
31+
URI uri = UriComponentsBuilder
32+
.fromHttpUrl(ddiAsHost)
33+
.path(UNITS_PATH)
34+
.build().toUri();
35+
log.info("Call DDIAS with URI : {}", uri);
36+
try {
37+
Unit[] units = webClient.get()
38+
.uri(uri)
39+
.accept(MediaType.APPLICATION_JSON)
40+
.retrieve()
41+
.bodyToMono(Unit[].class)
42+
.block();
43+
return List.of(units);
44+
} catch (WebClientResponseException e) {
45+
log.error(e.getMessage());
46+
throw new PoguesException(e.getStatusCode().value(), e.getStatusText(), e.getResponseBodyAsString());
47+
} catch (Exception e){
48+
throw new PoguesException(500, "Unknow error during metadata call", "");
49+
}
50+
}
51+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
package fr.insee.pogues.metadata.client;
2+
3+
import fr.insee.pogues.metadata.model.magma.Operation;
4+
import fr.insee.pogues.metadata.model.magma.Serie;
5+
6+
import java.util.List;
7+
8+
public interface MagmaClient {
9+
10+
List<Serie> getSeries() throws Exception;
11+
Serie getSerieById(String id) throws Exception;
12+
List<Operation> getOperationsByIdSerie(String idSerie) throws Exception;
13+
Operation getOperationById(String idOperation) throws Exception;
14+
}
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
package fr.insee.pogues.metadata.client;
2+
3+
import fr.insee.pogues.metadata.model.magma.Operation;
4+
import fr.insee.pogues.metadata.model.magma.Serie;
5+
import fr.insee.pogues.webservice.rest.PoguesException;
6+
import lombok.extern.slf4j.Slf4j;
7+
import org.springframework.beans.factory.annotation.Autowired;
8+
import org.springframework.beans.factory.annotation.Value;
9+
import org.springframework.http.MediaType;
10+
import org.springframework.stereotype.Service;
11+
import org.springframework.web.reactive.function.client.WebClient;
12+
import org.springframework.web.reactive.function.client.WebClientResponseException;
13+
import org.springframework.web.util.UriComponentsBuilder;
14+
15+
import java.net.URI;
16+
import java.util.List;
17+
18+
@Service
19+
@Slf4j
20+
public class MagmaClientImpl implements MagmaClient {
21+
22+
private static final String SERIES_PATH = "/operations/series";
23+
private static final String OPERATIONS_SERIE_PATH = "/operations/serie";
24+
private static final String OPERATIONS_PATH = "operations";
25+
private static final String SINGLE_OPERATION_PATH = "/operations/operation";
26+
27+
@Autowired
28+
private WebClient webClient;
29+
30+
@Value("${application.metadata.magma}")
31+
String magmaHost;
32+
33+
@Override
34+
public List<Serie> getSeries() throws Exception {
35+
URI uri = UriComponentsBuilder
36+
.fromHttpUrl(magmaHost)
37+
.path(SERIES_PATH)
38+
.queryParam("survey",true)
39+
.build().toUri();
40+
log.info("Call Magma with URI : {}", uri);
41+
try {
42+
Serie[] series = webClient.get()
43+
.uri(uri)
44+
.accept(MediaType.APPLICATION_JSON)
45+
.retrieve()
46+
.bodyToMono(Serie[].class)
47+
.block();
48+
return List.of(series);
49+
} catch (WebClientResponseException e) {
50+
log.error(e.getMessage());
51+
throw new PoguesException(e.getStatusCode().value(), e.getStatusText(), e.getResponseBodyAsString());
52+
} catch (Exception e){
53+
log.error(e.getMessage());
54+
throw new PoguesException(500, "Unknow error during metadata call", "");
55+
}
56+
}
57+
58+
@Override
59+
public Serie getSerieById(String id) throws Exception {
60+
URI uri = UriComponentsBuilder
61+
.fromHttpUrl(magmaHost)
62+
.path(OPERATIONS_SERIE_PATH)
63+
.pathSegment(id)
64+
.build().toUri();
65+
log.info("Call Magma with URI : {}", uri);
66+
try {
67+
Serie serie = webClient.get()
68+
.uri(uri)
69+
.accept(MediaType.APPLICATION_JSON)
70+
.retrieve()
71+
.bodyToMono(Serie.class)
72+
.block();
73+
return serie;
74+
} catch (WebClientResponseException e) {
75+
log.error(e.getMessage());
76+
throw new PoguesException(e.getStatusCode().value(), e.getStatusText(), e.getResponseBodyAsString());
77+
} catch (Exception e){
78+
throw new PoguesException(500, "Unknow error during metadata call", "");
79+
}
80+
}
81+
82+
@Override
83+
public List<Operation> getOperationsByIdSerie(String id) throws Exception {
84+
URI uri = UriComponentsBuilder
85+
.fromHttpUrl(magmaHost)
86+
.path(OPERATIONS_SERIE_PATH)
87+
.pathSegment(id)
88+
.pathSegment(OPERATIONS_PATH)
89+
.build().toUri();
90+
log.info("Call Magma with URI : {}", uri);
91+
try {
92+
Operation[] operations = webClient.get()
93+
.uri(uri)
94+
.accept(MediaType.APPLICATION_JSON)
95+
.retrieve()
96+
.bodyToMono(Operation[].class)
97+
.block();
98+
return List.of(operations);
99+
} catch (WebClientResponseException e) {
100+
log.error(e.getMessage());
101+
throw new PoguesException(e.getStatusCode().value(), e.getStatusText(), e.getResponseBodyAsString());
102+
} catch (Exception e){
103+
throw new PoguesException(500, "Unknow error during metadata call", "");
104+
}
105+
}
106+
107+
@Override
108+
public Operation getOperationById(String idOperation) throws Exception {
109+
URI uri = UriComponentsBuilder
110+
.fromHttpUrl(magmaHost)
111+
.path(SINGLE_OPERATION_PATH)
112+
.pathSegment(idOperation)
113+
.build().toUri();
114+
log.info("Call Magma with URI : {}", uri);
115+
try {
116+
Operation operation = webClient.get()
117+
.uri(uri)
118+
.accept(MediaType.APPLICATION_JSON)
119+
.retrieve()
120+
.bodyToMono(Operation.class)
121+
.block();
122+
return operation;
123+
} catch (WebClientResponseException e) {
124+
log.error(e.getMessage());
125+
throw new PoguesException(e.getStatusCode().value(), e.getStatusText(), e.getResponseBodyAsString());
126+
} catch (Exception e){
127+
throw new PoguesException(500, "Unknow error during metadata call", "");
128+
}
129+
}
130+
131+
132+
}

src/main/java/fr/insee/pogues/metadata/client/MetadataClient.java

-18
This file was deleted.

0 commit comments

Comments
 (0)