IOXIO® Dataspace Sandbox Definitions

This repository contains Data Product definitions for the IOXIO® Dataspace Sandbox and contains plenty of example definitions. When setting up definitions for a new dataspace you should rather use the definitions-template than fork this repository.

Specification for describing data product definitions can be found in ./DataProducts/README.md.

Please note that this repository is under active development, and all the definitions are subjects to change at any time.

To view these definitions in a more human readable format instead of the raw technical format in this repository, you can check these resources:

• Definitions viewer
• API docs
• Swagger UI

Repo structure

• ./src - Definition sources in python format
• ./DataProducts - Final Definitions as OpenAPI 3.x specs
• .github/workflows - Pre-configured CI workflows for validating and converting definitions from sources

Getting started

Please check the Contribution guidelines to learn how to submit new data product definitions in this repo.

Test version of definitions

Everyone can submit to this repo whatever definitions they seem appropriate. It will allow to create data products using these definitions in IOXIO Dataspace Sandbox and experiment with the system. In order to do this:

1. Fork this repository
2. Create your definitions under src/test/<your_github_username>/
3. Submit a PR and wait for CI Workflow to run and validate the changes
4. Once PR is merged, it's possible to use the definitions in IOXIO Dataspace Sandbox

Python sources

Each python file located in the src folder is treated as a Data Product definition.

For example, src/AirQuality/Current_v1.0.py defines the AirQuality/Current_v1.0 data product.

These files are then converted to OpenAPI 3.x specs, which are final forms of definitions. To make the converter work correctly, each file must follow the same structure:

from pydantic import Field

from definition_tooling.converter import CamelCaseModel, DataProductDefinition, ErrorResponse


class Request(CamelCaseModel):
    ...


class Response(CamelCaseModel):
    ...


@ErrorResponse(description="...")
class Error418(CamelCaseModel):
    ...


DEFINITION = DataProductDefinition(
    version="1.0.0",
    title="...",
    description="...",
    request=Request,
    response=Response,
    requires_authorization=False,
    requires_consent=False,
    error_responses={
        418: Error418,
    },
    deprecated=False,
    tags=["Foo"],
)

Considering CamelCaseModel is a subclass of pydantic's BaseModel, it's better to understand how to use this library. Please read pydantic's documentation if you're not familiar with it yet.

Each data product definition represented as python file must define a DEFINITION variable which is an instance of DataProductDefinition class.

DataProductDefinition is a structure consisting of:

• version

  Version is used in the info block of the OpenAPI spec. The data product definitions use Semantic Versioning of the form MAJOR.MINOR.PATCH, for example 1.0.0. Definitions in the test folder must have versions of the form 0.0.z and the version number should not exist in the filename of the definition. In all other definitions the version number needs to be >= 0.1.0 and the corresponding short version number needs to be included in the filename. For example the version 0.1.0 of the Foo/Bar definition would correspond to the file src/Foo/Bar_v0.1.py. For more details about versions and filenames see the Versioning of definitions section in the contribution guidelines.

• title

  Title used in the info block of OpenAPI spec and the summary for the POST route

• description

  Data product description, used in the top of OpenAPI spec and in the POST route

• request

  pydantic model describing body of POST request

• response

  pydantic model describing expected response from data source

• requires_authorization

  Marks the Authorization header as required

• requires_consent

  Marks the X-Consent-Token header as required

• error_responses

  A mapping from HTTP error status codes to pydantic models, wrapped in the ErrorResponse decorator, describing expected error responses from the data source

• deprecated

  Marks the route as deprecated

• tags

  A list (or other iterable) of strings to use as tags for the POST route. The tags should be in Title Case.

Example

There's an example of Data Product Definition for current weather:

from pydantic import Field

from definition_tooling.converter import CamelCaseModel, DataProductDefinition


class CurrentWeatherMetricRequest(CamelCaseModel):
    lat: float = Field(
        ...,
        title="Latitude",
        description="The latitude coordinate of the desired location",
        ge=-90.0,
        le=90.0,
        examples=[60.192059],
    )
    lon: float = Field(
        ...,
        title="Longitude",
        description="The longitude coordinate of the desired location",
        ge=-180.0,
        le=180.0,
        examples=[24.945831],
    )


class CurrentWeatherMetricResponse(CamelCaseModel):
    humidity: float = Field(..., title="Current relative air humidity in %", examples=[72])
    pressure: float = Field(..., title="Current air pressure in hPa", examples=[1007])
    rain: bool = Field(
        ..., title="Rain status", description="If it's currently raining or not."
    )
    temp: float = Field(
        ..., title="Current temperature in Celsius", examples=[17.3], ge=-273.15
    )
    wind_speed: float = Field(..., title="Current wind speed in m/s", examples=[2.1], ge=0)
    wind_direction: float = Field(
        ...,
        title="Current wind direction in meteorological wind direction degrees",
        ge=0,
        le=360,
        examples=[220.0],
    )


DEFINITION = DataProductDefinition(
    version="1.0.0",
    title="Current weather in a given location",
    description="Current weather in a given location with metric units",
    request=CurrentWeatherMetricRequest,
    response=CurrentWeatherMetricResponse,
    tags=["Weather"],
)

Guides and help

Written guide for how to create data definitions

You can also check out our YouTube tutorial:

Also join our IOXIO Community Slack