Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs from master #291

Merged
merged 13 commits into from
Nov 22, 2023
11 changes: 5 additions & 6 deletions .github/workflows/publish-maven.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,12 @@ jobs:
java-version: "17"
distribution: "temurin"
server-id: ossrh
- name: Import GPG Key
uses: crazy-max/[email protected]
with:
gpg_private_key: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
passphrase: ${{ secrets.MAVEN_GPG_PASSPHRASE }}
server-username: MAVEN_USERNAME
server-password: MAVEN_PASSWORD
gpg-private-key: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
gpg-passphrase: MAVEN_GPG_PASSPHRASE
- name: Publish Trevas
run: mvn --batch-mode -Dgpg.passphrase=${{ secrets.MAVEN_GPG_PASSPHRASE }} deploy
run: mvn -B -Prelease deploy
env:
MAVEN_USERNAME: ${{ secrets.OSSRH_USERNAME }}
MAVEN_PASSWORD: ${{ secrets.OSSRH_TOKEN }}
Expand Down
70 changes: 70 additions & 0 deletions docs/blog/2023-09-01-trevas-1.1.0.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
slug: /trevas-1.1.0
title: Trevas 1.1.0
authors: [nicolas]
tags: [Trevas Jupyter]
---

### News

Trevas 1.1.0 includes hierarchical validation via operators `define hierarchical ruleset` and `check_hierarchy`.

### Example

#### Input

`ds1`:

| id | Me |
| :-: | :-: |
| ABC | 12 |
| A | 1 |
| B | 10 |
| C | 1 |
| DEF | 100 |
| E | 99 |
| F | 1 |
| HIJ | 100 |
| H | 99 |
| I | 0 |

#### VTL script

```
// Ensure ds1 metadata definition is good
ds1 := ds1[calc identifier id := id, Me := cast(Me, integer)];

// Define hierarchical ruleset
define hierarchical ruleset hr (variable rule Me) is
My_Rule : ABC = A + B + C errorcode "ABC is not sum of A,B,C" errorlevel 1;
DEF = D + E + F errorcode "DEF is not sum of D,E,F";
HIJ : HIJ = H + I - J errorcode "HIJ is not H + I - J" errorlevel 10
end hierarchical ruleset;

// Check hierarchy
ds_all := check_hierarchy(ds1, hr rule id all);
ds_all_measures := check_hierarchy(ds1, hr rule id always_null all_measures);
ds_invalid := check_hierarchy(ds1, hr rule id always_zero invalid);
```

#### Outputs

- `ds_all`

| id | ruleid | bool_var | errorcode | errorlevel | imbalance |
| :-: | :-----: | :------: | :-------: | :--------: | :-------: |
| ABC | My_Rule | true | null | null | 0 |

- `ds_always_null_all_measures`

| id | Me | ruleid | bool_var | errorcode | errorlevel | imbalance |
| :-: | :-: | :-----: | :------: | :-------: | :--------: | :-------: |
| ABC | 12 | My_Rule | true | null | null | 0 |
| DEF | 100 | hr_2 | null | null | null | null |
| HIJ | 100 | HIJ | null | null | null | null |

- `ds_invalid`

| id | Me | ruleid | errorcode | errorlevel | imbalance |
| :-: | :-: | :----: | :------------------: | :--------: | :-------: |
| HIJ | 100 | HIJ | HIJ is not H + I - J | 10 | 1 |
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import Card from '@theme/Card';

### Trevas good practices

The <a href={useBaseUrl('/developer-guide/basic-mode/data-sources/json')}>JSON</a> format is the only one allowing to store and manage VTL metadata when the Trevas engine is instanciated in basic mode.
The <a href={useBaseUrl('/developer-guide/basic-mode/data-sources/json')}>JSON</a> format is the only one allowing to store and manage VTL metadata when the Trevas engine is instantiated in basic mode.

It is thus strongly recommended to use this format.

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/developer-guide/basic-mode/data-sources/jdbc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-jdbc</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/developer-guide/basic-mode/data-sources/json.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-jackson</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
4 changes: 2 additions & 2 deletions docs/docs/developer-guide/index-developer-guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ import Card from '@theme/Card';
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-engine</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

### Instanciate the Trevas engine
### Instantiate the Trevas engine

```java
// Start engine
Expand Down
4 changes: 2 additions & 2 deletions docs/docs/developer-guide/spark-mode/index-spark-mode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ The `SparkDataset` data sets can represent statistical tables in a Java applicat
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-spark</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

### Spark session

In order to execute VTL via Trevas in Spark mode, a Spark session must be instanciated.
In order to execute VTL via Trevas in Spark mode, a Spark session must be instantiated.

The session can be:

Expand Down
3 changes: 3 additions & 0 deletions docs/docs/introduction/index-introduction.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,14 @@ pagination_prev: null

import useBaseUrl from '@docusaurus/useBaseUrl';
import ThemedImage from '@theme/ThemedImage';
import Video from '@theme/Video';

[![Maven - Trevas parent](https://maven-badges.herokuapp.com/maven-central/fr.insee.trevas/trevas-parent/badge.svg)](https://maven-badges.herokuapp.com/maven-central/fr.insee.trevas/trevas-parent)

[Trevas](https://github.com/InseeFr/Trevas) is a Java library for transforming and validating statistical data using the [VTL 2.0](https://sdmx.org/?page_id=5096) language.

<Video url={"https://www.youtube.com/watch?v=e8coNevJyXA"} />

## What is VTL?

- Business, logical-level and user-friendly language for statisticians
Expand Down
12 changes: 12 additions & 0 deletions docs/docs/introduction/releases/1.x.x.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ slug: /releases/1.x.x
custom_edit_url: null
---

## Version 1.1.1 - 09/09/23

(See technical release on [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.1.1))

- Fix `check_hierarchy` issues: serialization and possibility to return empty dataset

## Version 1.1.0 - 09/01/23

(See technical release on [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.1.0))

- Validation: `define hierarchical ruleset`, `check_hierarchy`

## Version 1.0.2 - 06/30/23

(See technical release on [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.0.2))
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
slug: /trevas-1.1.0
title: Trevas 1.1.0
authors: [nicolas]
tags: [Trevas Jupyter]
---

### Nouveautés

La validation hiérarchique est implémentée dans Trevas 1.1.0, via les opérateurs `define hierarchical ruleset` et `check_hierarchy`.

### Exemple

#### Données en entrée

`ds1`:

| id | Me |
| :-: | :-: |
| ABC | 12 |
| A | 1 |
| B | 10 |
| C | 1 |
| DEF | 100 |
| E | 99 |
| F | 1 |
| HIJ | 100 |
| H | 99 |
| I | 0 |

#### Script VTL

```
// Ensure ds1 metadata definition is good
ds1 := ds1[calc identifier id := id, Me := cast(Me, integer)];

// Define hierarchical ruleset
define hierarchical ruleset hr (variable rule Me) is
My_Rule : ABC = A + B + C errorcode "ABC is not sum of A,B,C" errorlevel 1;
DEF = D + E + F errorcode "DEF is not sum of D,E,F";
HIJ : HIJ = H + I - J errorcode "HIJ is not H + I - J" errorlevel 10
end hierarchical ruleset;

// Check hierarchy
ds_all := check_hierarchy(ds1, hr rule id all);
ds_all_measures := check_hierarchy(ds1, hr rule id always_null all_measures);
ds_invalid := check_hierarchy(ds1, hr rule id always_zero invalid);
```

#### Données en sortie

- `ds_all`

| id | ruleid | bool_var | errorcode | errorlevel | imbalance |
| :-: | :-----: | :------: | :-------: | :--------: | :-------: |
| ABC | My_Rule | true | null | null | 0 |

- `ds_always_null_all_measures`

| id | Me | ruleid | bool_var | errorcode | errorlevel | imbalance |
| :-: | :-: | :-----: | :------: | :-------: | :--------: | :-------: |
| ABC | 12 | My_Rule | true | null | null | 0 |
| DEF | 100 | hr_2 | null | null | null | null |
| HIJ | 100 | HIJ | null | null | null | null |

- `ds_invalid`

| id | Me | ruleid | errorcode | errorlevel | imbalance |
| :-: | :-: | :----: | :------------------: | :--------: | :-------: |
| HIJ | 100 | HIJ | HIJ is not H + I - J | 10 | 1 |
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-jdbc</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-jackson</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import Card from '@theme/Card';
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-engine</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Les datasets `SparkDataset` permettent de représenter les tables statistiques d
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-spark</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,14 @@ pagination_prev: null

import useBaseUrl from '@docusaurus/useBaseUrl';
import ThemedImage from '@theme/ThemedImage';
import Video from '@theme/Video';

[![Maven - Trevas parent](https://maven-badges.herokuapp.com/maven-central/fr.insee.trevas/trevas-parent/badge.svg)](https://maven-badges.herokuapp.com/maven-central/fr.insee.trevas/trevas-parent)

[Trevas](https://github.com/InseeFr/Trevas) est une librairie Java permettant de transformer et de valider des données statistiques grâce au langage [VTL 2.0](https://sdmx.org/?page_id=5096).

<Video url={"https://www.youtube.com/watch?v=e8coNevJyXA"} />

## Qu'est ce que VTL ?

- langage métier, au niveau logique et convivial pour les statisticiens
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ slug: /releases/1.x.x
custom_edit_url: null
---

## Version 1.1.1 - 09/09/23

(Voir la note de livraison technique sur [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.1.1))

- Corrections `check_hierarchy`: serialisation et possibilité de retourner un dataset vide

## Version 1.1.0 - 01/09/23

(Voir la note de livraison technique sur [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.1.0))

- Validation: `define hierarchical ruleset`, `check_hierarchy`

## Version 1.0.2 - 30/06/23

(Voir la note de livraison technique sur [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.0.2))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-jdbc</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-jackson</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import Card from '@theme/Card';
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-engine</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ custom_edit_url: null
<dependency>
<groupId>fr.insee.trevas</groupId>
<artifactId>vtl-spark</artifactId>
<version>1.0.2</version>
<version>1.1.1</version>
</dependency>
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,14 @@ pagination_prev: null

import useBaseUrl from '@docusaurus/useBaseUrl';
import ThemedImage from '@theme/ThemedImage';
import Video from '@theme/Video';

[![Maven - Trevas parent](https://maven-badges.herokuapp.com/maven-central/fr.insee.trevas/trevas-parent/badge.svg)](https://maven-badges.herokuapp.com/maven-central/fr.insee.trevas/trevas-parent)

[Trevas](https://github.com/InseeFr/Trevas) er et Java-bibliotek for å transformere og validere statistiske data ved å bruke [VTL 2.0](https://sdmx.org/?page_id=5096) språket.

<Video url={"https://www.youtube.com/watch?v=e8coNevJyXA"} />

## Hva er VTL?

- Forretningsmessig, logisk nivå og brukervennlig språk for statistikere
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,18 @@ slug: /releases/1.x.x
custom_edit_url: null
---

## Versjon 1.1.1 - 09/09/23

(Se teknisk utgivelse på [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.1.1))

- `check_hierarchy` rettelser: serialisering og mulighet for å returnere et tomt datasett

## Versjon 1.1.0 - 09/01/23

(Se teknisk utgivelse på [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.1.0))

- Validering: `define hierarchical ruleset`, `check_hierarchy`

## Versjon 1.0.2 - 06/30/23

(Se teknisk utgivelse på [Github](https://github.com/InseeFr/Trevas/releases/tag/v1.0.2))
Expand Down
Loading