Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to enforce validation based on referenced WoT model #1650

Closed
thjaeckle opened this issue Jun 10, 2023 · 2 comments · Fixed by #1936
Closed

Add option to enforce validation based on referenced WoT model #1650

thjaeckle opened this issue Jun 10, 2023 · 2 comments · Fixed by #1936
Assignees
Labels
WoT Web of Things related enhancements
Milestone

Comments

@thjaeckle
Copy link
Member

thjaeckle commented Jun 10, 2023

Currently, Ditto's WoT Integration does not ensure/enforce the referenced model.

As the reference implementation, Eclipse Thingweb did also not do that we thought that would be a common practice to move validation to eg producers of messages.

In practice I now see that an enforcement of the model by Ditto would have great benefits and is the best place to add validation.
This would eg ensure that different API consumers always modify a thing only in the intended and modeled way.

That would be especially important if the Ditto managed twin is used in an Event driven system, where changes of the twin state is reacted to by eg issuing a command downlink to an actual device (using "desired properties" approach).

This issue should track:

  • new config option to enable wot validation
  • implementation and making configurable all of the below:
    • validation of attributes on thing level
    • validation of properties of features
    • validation of desired properties of features (ignoring "required" constraints)
    • validation of thing messages
    • validation of feature messages
    • whether to allow modifying non modeled attributes/properties or to reject them
    • whether to allow sending of non modeled messages (wot actions/events) or to reject them

I expect this to be quite difficult to do, as we have to extract the JsonSchema parts from WoT TM models and apply them.
Both when eg a complete thing is modified and also partially, when eg only a single feature or even property or even part of a property is modified.

@thjaeckle thjaeckle changed the title Add option to enforce validatation based on referenced WoT model Add option to enforce validation based on referenced WoT model Jun 10, 2023
@thjaeckle
Copy link
Member Author

I think that we need to validate always what the target state of the complete thing (or feature if only a feature was modifier) would be against the model.

So after enforcement (different permissions should not yield different validation results), but before applying a modify command to the persistence.

We should also monitor how much time validation takes, by adding metrics (spans) to the existing signal processing trace.

It would ne nice to have the validation as separate Java Module so that it could also be used as a library.

@thjaeckle thjaeckle added this to the 3.4.0 milestone Jun 16, 2023
@thjaeckle thjaeckle moved this to Todo in Ditto Planning Jun 16, 2023
@thjaeckle
Copy link
Member Author

Another idea on the enforcement:
Configure a list of auth subjects (also supporting wildcards) for which validation is not done.

I am thinking especially about eg connections where we don't necessarily rely on a user input to validate.
Whereas for api users, eg authenticated via JWT and the http api, it would be important to validate all provided data.

@thjaeckle thjaeckle removed this from the 3.4.0 milestone Aug 25, 2023
@thjaeckle thjaeckle added the WoT Web of Things related enhancements label Jan 30, 2024
@thjaeckle thjaeckle added this to the 3.6.0 milestone Feb 24, 2024
@thjaeckle thjaeckle self-assigned this Feb 24, 2024
@thjaeckle thjaeckle moved this to In Progress in Ditto Planning Feb 24, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 6, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 6, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 6, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 6, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 6, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 7, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 7, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 7, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 7, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue May 13, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jun 3, 2024
…desired properties validation

* taking care of e.g. "ditto:category" defined
* first validation error will provide the error output (which is for now simpler and results in quicker obtained validation errors)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jun 5, 2024
* validating features and properties as part of "modify thing"
* handling "ditto:category" correctly
* ensuring completeness of defined features
* validation on creation/modification
  * of thing
  * of attributes
  * on attribute
  * on features
  * on feature

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jun 10, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jun 18, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jun 18, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jun 18, 2024
…desired properties validation

* taking care of e.g. "ditto:category" defined
* first validation error will provide the error output (which is for now simpler and results in quicker obtained validation errors)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
…desired properties validation

* taking care of e.g. "ditto:category" defined
* first validation error will provide the error output (which is for now simpler and results in quicker obtained validation errors)

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* validating features and properties as part of "modify thing"
* handling "ditto:category" correctly
* ensuring completeness of defined features
* validation on creation/modification
  * of thing
  * of attributes
  * on attribute
  * on features
  * on feature

Signed-off-by: Thomas Jäckle <[email protected]>
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* enforce modifying thing definition + feature definition
* enhance configuration options
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
…their responses

* also updated some dependencies
* simplified DefaultWotThingModelValidator a little
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
…iding "type" (valid for e.g. "oneOf") use

* update json-schema-validator library
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
…ation in smaller pieces

* strict split between config reading in DefaultWotThingModelValidation and static functionality in Internal* classes
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* based on either DittoHeaders or on ThingDefinition/FeatureDefinition URLs
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* fixed and harmonized other traces while doing that in order to get correct parent hierarchy of spans
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
…ased on the WoT model

* restructured some too long methods
* fixed remaining TODOs
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* also for dynamic part
* changed config structure a bit in order to better fit Helm YAML config
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
…e corner cases

* e.g. deeply nested updates, removing definition via PATCH
* enabled WoT based validation by default
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
* first choosing potentially new definition and only as fall-back the old one
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 29, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Jul 30, 2024
thjaeckle added a commit to beyonnex-io/ditto that referenced this issue Aug 1, 2024
thjaeckle added a commit that referenced this issue Aug 26, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Ditto Planning Aug 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
WoT Web of Things related enhancements
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants
@thjaeckle and others