opcua: nsu in nodeId (fixing #1334) #1335

erossignon · 2024-11-29T22:52:37Z

No description provided.

sebastiankb · 2024-12-02T13:30:29Z

beside of the "ns=..." nodeId, TD needs also to allow "nsu=..." as node ID. Still we need to clearify, if the nodeID can be part of the href or is dedicated ua:nodeId term is needed.

danielpeintner · 2024-12-03T09:54:17Z

The idea is that we use href instead of the own term opcua:nodeId (we use href in other bindings too).

For example, we now have

{
    "base": "opc.tcp://192.168.120.237:4840/",
    "properties": {
        "foo": {
            "forms": [
                {
                    "opcua:nodeId": "nsu=http://example.org/SpecialNamespace/;s=\"LED\".\"State\""
                }
            ]
        }
    }
}

which would become the following

{
    "base": "opc.tcp://192.168.120.237:4840/",
    "properties": {
        "foo": {
            "forms": [
                {
                    "href": "nsu=http://example.org/SpecialNamespace/;s=\"LED\".\"State\""
                }
            ]
        }
    }
}

The problem I see are href parsers since the absolute URL would become the following

opc.tcp://192.168.120.237:4840/nsu=http://example.org/SpecialNamespace/;s=\"LED\".\"State\"

A more Web style friendly format would be to use query parameters

opc.tcp://192.168.120.237:4840/?nsu=http://example.org/SpecialNamespace/&s=\"LED\".\"State\"

Note1: the difference is the ? after the host part and that arguments get concatenated via & and not ;

Note2: it has the downside that one needs to reconstruct the format string at OPC UA side.

Question: I wonder whether people share the concern that if we purely use the OPC UA format string href parsers will destroy/corrupt the information.

Note3: we also need to be careful with some specific characters like # that get interpreted as fragment identifier. That could be the case in the nsu part. Try to parse the following string
opc.tcp://192.168.120.237/nsu=http://example.org/#foo;s=SpecialVar
on https://www.freeformatter.com/url-parser-query-string-splitter.html

erossignon · 2024-12-03T13:17:03Z

May be one way is to encode the href portion using encodeUriComponent decodeUriComponent

encodeURIComponent("opc.tcp://192.168.120.237:4840/nsu=http://example.org/SpecialNamespace/;s=\"LED\".\"State\"")


'opc.tcp%3A%2F%2F192.168.120.237%3A4840%2Fnsu%3Dhttp%3A%2F%2Fexample.org%2FSpecialNamespace%2F%3Bs%3D%22LED%22.%22State%22'
)

to prevent any clashes

egekorkan · 2024-12-04T09:11:06Z

Going with URL encoding was also the preliminary decision taken for MQTT at w3c/wot-binding-templates#292 (comment)

danielpeintner · 2024-12-04T13:44:19Z

Going with URL encoding was also the preliminary decision taken for MQTT at w3c/wot-binding-templates#292 (comment)

The problem I see with doing so is that for humans, it is not easy to read the "encoded" string.
(Not sure if this is an actual problem)

Anyhow, It seems we need a general decision that can be taken across all/other bindings...

egekorkan · 2024-12-09T13:31:20Z

We had a chat about this together with @sebastiankb @danielpeintner @wiresio and @Kaz040. Here our points, that need to be discussed with the OPC UA community as well.

The TDs should be still human-readable: Encoding the entire URI breaks this. This can turn off the ones who are used to OPC UA.
It should be easy to copy-paste node ids from a tool like UA Expert to TDs
We have evaluated some aspects:
- Using single quotes inside the double quotes: It is valid JSON, but the UA server can still have double quotes that need to be converted when copy-pasted. Surrounding a string with single quotes is not valid JSON.
- Double quote seems to be the most annoying part since even when we do not use href it is a special character for JSON. We are not sure how common it is but in Siemens S7, it is quite common.
- Escaping the quotes would be tolerable for someone copy-pasting. It can be done by humans easily as opposed to URL-encoding.
- Best would be to use query parameters if people can get used to it.
- If we want to do encoding, we can simply encode the part after nsu

sebastiankb · 2025-01-08T12:36:44Z

In the OPC UA WoT Binding WG we discussed 2 relevant options to provide the nodeId information:

Option I:
The value of href can be used with the following patterns:

opc.tcp:// <address>:<port>/?nsu=<namespace>&s=urlEncoded(<stringId>) 
                                            &b=urlEncoded(<base64string>) 
                                            &i=<integer> 
                                            &g=<guid>

Note: The UA server endpoint information opc.tcp:// <address>:<port> can be specified in the base instead.

Pro:

compliant to RFC3986
compliant to WoT datapoint addressing assumption
only for sting-based IDs there is an URL encoding requested; increase readability if not everything is URL encoded

Con:

nodeID cannot be used as is (e.g., nsu=http://example.org/SpecialNamespace/;s=\"LED\".\"State\"); must be separated into the query parameters and URL encoding is needed for string-based IDs

Option II:

The value of href is empty or only used for UA server endpoint information (e.g., if base is not used).
A separate term is defined ua:nodeID that takes the UA nodeID as value, e.g.:

...
"href":"",
"ua:nodeID":"nsu=http://example.org/SpecialNamespace/;s=\"LED\".\"State\""
...

Pro:

compliant to RFC3986
nodeID cannot be used as is (e.g., no URL encoding is needed)

Con:

not compliant to WoT datapoint addressing assumption
href is most likely always empty

What do you think?

Ping @barnstee @randy-armstrong @erossignon @danielpeintner @egekorkan @relu91

egekorkan · 2025-01-08T15:32:15Z

W3C WoT TD Task Force call today:

Why is the " escaped and then encoded again
@relu91 , @danielpeintner , @lu-zero value consistency, so option 1.
namespace can need url encoding too since it can contain # too
@sebastiankb why are there multiple query parameters and why not everything is encoded?
We should verify that we can always decode, whether it was encoded first or not.

sebastiankb · 2025-01-08T17:17:46Z

Many thanks for your feedback!

Why is the " escaped and then encoded again

This example coming from a real sample implementation which uses nested " in the string-based nodeID.

namespace can need url encoding too since it can contain # too

OK, good point. It was hoped that the values of queries would be excluded...

@sebastiankb why are there multiple query parameters and why not everything is encoded?

Actually, the answer is already given above.
The idea was to keep the readability of nodeIDs in TDs as high as possible and to avoid the noise of URL encoding as much as possible. But if we also need to encode the namespace, then I see no advantage in defining separate query parameters. In that context, the proposal from @erossignon above makes most sense:

opc.tcp:// <address>:<port>/urlEncoded(<nodeID>)

It seems that a more readable approach is not possible...

sebastiankb · 2025-01-09T15:40:18Z

There also seems to be a nice alternative where the UA nodeID remains as is (=readability) and keeps the WoT consistency:

opc.tcp:// <address>:<port>/#<nodeID>

The fragment identifier # takes any character, an URL encoding is not needed.

(btw: this approach was also discussed for MQTT in the past)

barnstee · 2025-01-09T15:58:28Z

There also seems to be a nice alternative where the UA nodeID remains as is (=readability) and keeps the WoT consistency:

opc.tcp:// <address>:<port>/#<nodeID>

The fragment identifier # takes any character, an URL encoding is not needed.

(btw: this approach was also discussed for MQTT in the past)

This is perfect!

relu91 · 2025-01-09T18:22:05Z

There also seems to be a nice alternative where the UA nodeID remains as is (=readability) and keeps the WoT consistency:

opc.tcp:// <address>:<port>/#<nodeID>

The fragment identifier # takes any character, an URL encoding is not needed.

(btw: this approach was also discussed for MQTT in the past)

It might be a clean solution from the readability point of view, but it still breaks the URI semantics explained in RFC3986 (see my old comment in the thread cited by @sebastiankb).

We can still ofc bend a little bit the rules... but it doesn't convince me 100%. Encoding the string does not seem to be a big burden from my point of view, it has the benefit of working with JSON too... (even using the fragment solution you would still to \" all the quotes)

sebastiankb · 2025-01-10T17:54:18Z

I have just found an interesting piece of information from the OPC UA specification about NodeIds:

The URI portion of NodeIds are escaped with URI percent encoding as defined in RFC 3986. Semicolons are added to the list of reserved characters of all URI schemes.

I'm not sure I understand the examples presented, e.g.

nsu=tag:acme.com,2023:schemas:data#off%3B;b=M/RbKBsRVkePCePcx24oRA==

Why only # character is encoded here?

I will discuss this in the next UA WG meeting.

@relu91 do you have time to join us on Tuesday at 4 pm CET?

sebastiankb · 2025-01-15T14:34:56Z

At yesterday's working group meeting, we achieved a breakthrough with the following solution for OPC UA:

opc.tcp://<address>:<port>/?id=<nodeId>

Where is:

    {address} OPC UA server (IP) address
    {port} OPC UA server port number
    {nodeId} OPC UA NodeId with the following expectations:
            1) any hash character (#) must be URL encoded (%23)
            2) any ampersand character (&) must be URL encoded (%26)  
    
    Examples:
      - "href":"opc.tcp://192.168.120.237:4840/?id=ns=10;i=12345"   
      - "href":"opc.tcp://192.168.120.237:4840/?id=nsu=http://widgets.com/schemas/hello;s=水 World"  
      - "href":"/?id=nsu=http://example.com/hello%23;s=temperature" 
      - "href":"/?id=ns=10;i=12345"

egekorkan · 2025-01-15T15:39:33Z

W3C WoT TD TF call today:

We should add something saying that "Encoder SHOULD be special and only encode # and &" but this is not a MUST since nothing would break if someone encodes everything, other than humans trying to read it.
For the implementers, we should tell them that they can just decode it using any url decoder.
Following the both points above: Most compression algorithms specify the decoder and the encoder can be implemented as wanted (smarter or not).
We should probably escape % as well. We are not aware of any node id containing that but it is theoratically possible.
The spec text should reference both "URL Encoding/Decoding" and "Percent Encoding/Decoding". The latter is apparently the correct wording. See https://en.wikipedia.org/wiki/Percent-encoding

danielpeintner · 2025-02-03T10:51:16Z

Hi @erossignon
With #1343 we upgraded to TypeScript 5 and moved to "module": "Node16".
Can you merge in these updates into your PR? After thatwe can merge your contribution 👍. Thanks!

package.json

danielpeintner

I think we are almost there. I have some questions and propose some changes. See below.
Thank you @erossignon 👍

danielpeintner · 2025-02-05T08:21:51Z

packages/examples/tsconfig.json

@@ -2,7 +2,7 @@
    "extends": "../../tsconfig.json",
    "compilerOptions": {
        "outDir": "dist",
-        "module": "commonjs",
+        "module": "Node16",


@relu91 you kept "commonjs" because of scrambled output, right? Do you mind changing it?

Yes, this should be kept, any reason for changing it @erossignon ?

danielpeintner · 2025-02-05T08:23:33Z

packages/binding-opcua/tsconfig.json

+        "module": "Node16",
+        "moduleResolution": "Node16"


I don't think those 2 statements are necessary since the root tsconfig.json defines them already like this.

yes they could be moved, and modulResolution is also redundant when using Node16 as module. But we can keep both in the main tsconfig.json for documentation.

danielpeintner · 2025-02-05T08:24:07Z

packages/binding-opcua/tsconfig.json

@@ -2,7 +2,10 @@
    "extends": "../../tsconfig.json",
    "compilerOptions": {
        "outDir": "dist",
-        "rootDir": "src"
+        "rootDir": "src",
+        "skipLibCheck": true,


Do we run into issues if we do the checking?

I would use skipLibCheck only as a last resort.

danielpeintner · 2025-02-05T08:25:36Z

package.json

@@ -51,7 +52,7 @@
        "@types/chai-as-promised": "^7.1.4",
        "@types/chai-spies": "^1.0.4",
        "@types/mocha": "^9.0.0",
-        "@types/node": "^18.19.71",
+        "@types/node": "22.13.1",


Is this change necessary? @relu91 ?

we still support Node 18, so the maximum upgrade that we can do here is to move the minor. See https://github.com/eclipse-thingweb/node-wot/blob/master/.github/workflows/ci.yaml#L22

egekorkan · 2025-02-06T14:24:34Z

@sebastiankb @danielpeintner could you document the decision that was taken in this PR or point to the decision?

erossignon requested review from danielpeintner, relu91 and JKRhb as code owners November 29, 2024 22:52

erossignon force-pushed the opcua-nsu branch from 73543d3 to 71765c5 Compare November 29, 2024 22:56

erossignon mentioned this pull request Nov 29, 2024

Upgrading to Typescript 5.x #1313

Closed

erossignon force-pushed the opcua-nsu branch 2 times, most recently from 5982c62 to 32cbe50 Compare November 29, 2024 23:19

egekorkan mentioned this pull request Jan 8, 2025

URL Scheme Decision w3c/wot-thing-description#2069

Open

sebastiankb mentioned this pull request Jan 15, 2025

MQTT URI Design w3c/wot-binding-templates#292

Open

relu91 mentioned this pull request Jan 17, 2025

Upgrade to TypeScript 5.x #1343

Merged

erossignon force-pushed the opcua-nsu branch from 32cbe50 to b28bc5e Compare February 4, 2025 11:04

danielpeintner reviewed Feb 4, 2025

View reviewed changes

package.json Outdated Show resolved Hide resolved

erossignon force-pushed the opcua-nsu branch 2 times, most recently from d45c924 to fc725c8 Compare February 4, 2025 17:32

erossignon force-pushed the opcua-nsu branch from fc725c8 to 9354fb9 Compare February 4, 2025 17:40

opcua: handle nsu in nodeId (fixing eclipse-thingweb#1334)

520d38c

erossignon force-pushed the opcua-nsu branch from 9354fb9 to 520d38c Compare February 4, 2025 17:55

danielpeintner reviewed Feb 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

opcua: nsu in nodeId (fixing #1334) #1335

opcua: nsu in nodeId (fixing #1334) #1335

erossignon commented Nov 29, 2024

sebastiankb commented Dec 2, 2024 •

edited

Loading

danielpeintner commented Dec 3, 2024

erossignon commented Dec 3, 2024 •

edited

Loading

egekorkan commented Dec 4, 2024

danielpeintner commented Dec 4, 2024

egekorkan commented Dec 9, 2024

sebastiankb commented Jan 8, 2025 •

edited

Loading

egekorkan commented Jan 8, 2025

sebastiankb commented Jan 8, 2025

sebastiankb commented Jan 9, 2025 •

edited

Loading

barnstee commented Jan 9, 2025

relu91 commented Jan 9, 2025 •

edited

Loading

sebastiankb commented Jan 10, 2025 •

edited

Loading

sebastiankb commented Jan 15, 2025 •

edited

Loading

egekorkan commented Jan 15, 2025

danielpeintner commented Feb 3, 2025

danielpeintner left a comment •

edited

Loading

danielpeintner Feb 5, 2025

relu91 Feb 5, 2025

danielpeintner Feb 5, 2025

relu91 Feb 5, 2025

danielpeintner Feb 5, 2025

relu91 Feb 5, 2025

danielpeintner Feb 5, 2025

relu91 Feb 5, 2025

egekorkan commented Feb 6, 2025

opcua: nsu in nodeId (fixing #1334) #1335

Are you sure you want to change the base?

opcua: nsu in nodeId (fixing #1334) #1335

Conversation

erossignon commented Nov 29, 2024

sebastiankb commented Dec 2, 2024 • edited Loading

danielpeintner commented Dec 3, 2024

erossignon commented Dec 3, 2024 • edited Loading

egekorkan commented Dec 4, 2024

danielpeintner commented Dec 4, 2024

egekorkan commented Dec 9, 2024

sebastiankb commented Jan 8, 2025 • edited Loading

egekorkan commented Jan 8, 2025

sebastiankb commented Jan 8, 2025

sebastiankb commented Jan 9, 2025 • edited Loading

barnstee commented Jan 9, 2025

relu91 commented Jan 9, 2025 • edited Loading

sebastiankb commented Jan 10, 2025 • edited Loading

sebastiankb commented Jan 15, 2025 • edited Loading

egekorkan commented Jan 15, 2025

danielpeintner commented Feb 3, 2025

danielpeintner left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

egekorkan commented Feb 6, 2025

sebastiankb commented Dec 2, 2024 •

edited

Loading

erossignon commented Dec 3, 2024 •

edited

Loading

sebastiankb commented Jan 8, 2025 •

edited

Loading

sebastiankb commented Jan 9, 2025 •

edited

Loading

relu91 commented Jan 9, 2025 •

edited

Loading

sebastiankb commented Jan 10, 2025 •

edited

Loading

sebastiankb commented Jan 15, 2025 •

edited

Loading

danielpeintner left a comment •

edited

Loading