Missing EndpointUrl in telemetry format

[alevincenzi]

Describe the bug

With opcuaPublsher:2.8.0, the telemetry output (json) does not match the online documentation.

Moreover, changes in the command line option --mm has no effect on the telemetry messages.

To Reproduce

• Run a plc simulator with docker run --rm -it -p 50000:50000 -p 8086:8086 --name opcplc mcr.microsoft.com/iotedge/opc-plc:latest --pn=50000 --autoaccept --sph --sn=5 --sr=10 --st=uint --fn=5 --fr=1 --ft=uint --ctb --scn --lid --lsn --ref--gn=5
• Deploy the opcuaPublisher as explained in this tutorial
• Don't use the option --mm!
• Add routing to $upstream
• Configure the pn.json file with some nodes:

[
  {
    "EndpointUrl": "opc.tcp://10.1.0.2:50000",
    "UseSecurity": false,
    "OpcNodes": [
      { "Id": "ns=2;s=StepUp" },
      { "Id": "ns=2;s=AlternatingBoolean" },
      { "Id": "ns=2;s=RandomSignedInt32" },
      { "Id": "ns=2;s=RandomUnsignedInt32" }
    ]
  }
]

• Adapt the ip address according to the location of the plc simulator.

The telemetry messages will show up as follow

[
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=StepUp",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"Value": {
			"Value": 560934,
			"SourceTimestamp": "2021-09-24T05:56:12.202727Z"
		}
	},
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=RandomSignedInt32",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"Value": {
			"Value": -1001090625,
			"SourceTimestamp": "2021-09-24T05:56:12.2029103Z"
		}
	},
        ...
]

Expected behavior

The messages should look like those in the documentation page "https://docs.microsoft.com/en-us/azure/industrial-iot/reference-opc-publisher-telemetry-format".

The documentation page does not report which messaging model is used as default. It looks like "Samples" format but

• the EndpointUrl is missing
• the MonitoredItem is not there neither
• the node id has an extra prefix and does not match the one in pn.json

Moreover, if you run the publisher with --mm=Samples or -mm=PubSub the output does not change.

Desktop (please complete the following information):

• OS: Ubuntu Server 18
• Version iotedge 1.1.4 and opcuaPublisher 2.8.0

[cristipogacean]

Hi @alevincenzi, thank you for bringing up these topics.

Firstly, our documentation is a bit misleading, I would highly recommend you take the guthub documentation as reference and not the docs.microsoft.com. Particularly for the telemetry format, please have a look here: https://github.com/Azure/Industrial-IoT/blob/main/docs/dev-guides/telemetry-messages-format.md. The default --mm is "Samples".

If you need the endpoint url to be part of the telemetry, then you need to enable the so called "fullfeaturedmessage" using "--fm=true" argument. Default is set to false.
The MonitoredItem is no longer part of telemetry. Is this something you make use of in your application?
The NodeIds are now represented as Expanded NodeIDs in url syntax format. is this a breaking change for you?

Regarding the --mm=Samples vs -mm=PubSub, works fine for me. Would you mind giving it another try?

[alevincenzi]

Hello @cristipogacean

thank you for the reply and for pointing out the good documentation. Yes, we gave it a second try.

With the option (extracted from the deployment manifest)

\"Cmd\": [\"--autoaccept\", \"--logfile=logs/opcuaPublisher.log\", \"--publishfile=workspace/nodes.json\"]

we get messages like these

[
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Value": {
			"Value": true,
			"SourceTimestamp": "2021-09-28T15:16:58.9037588Z"
		}
	},
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Value": {
			"Value": false,
			"SourceTimestamp": "2021-09-28T15:17:03.9025818Z"
		}
	},
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Value": {
			"Value": true,
			"SourceTimestamp": "2021-09-28T15:17:08.9023353Z"
		}
	}
]

and we deduce that the defaults are "Samples" messages without the "full featured payload". They are both the default behaviour.

With the options

\"Cmd\": [\"--autoaccept\", \"--logfile=logs/opcuaPublisher.log\", \"--publishfile=workspace/nodes.json\", \"--fm=true\"]

we see

[
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"EndpointUrl": "opc.tcp://10.1.0.2:50000/",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Timestamp": "2021-09-28T15:21:19.4990399Z",
		"Value": {
			"Value": true,
			"SourceTimestamp": "2021-09-28T15:21:18.903539Z",
			"ServerTimestamp": "2021-09-28T15:21:18.9035915Z"
		},
		"SequenceNumber": 12,
		"ExtensionFields": {
			"PublisherId": "opc.tcp://10.1.0.2:50000_e873ab7d8040cb3b79e7b4a38296093f97c32da3",
			"DataSetWriterId": "opc.tcp://10.1.0.2:50000_c70079e5431fea7bfa4f9fbba7f2aad440d55efb"
		}
	},
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"EndpointUrl": "opc.tcp://10.1.0.2:50000/",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Timestamp": "2021-09-28T15:21:24.5207237Z",
		"Value": {
			"Value": false,
			"SourceTimestamp": "2021-09-28T15:21:23.9036481Z",
			"ServerTimestamp": "2021-09-28T15:21:23.9039577Z"
		},
		"SequenceNumber": 13,
		"ExtensionFields": {
			"PublisherId": "opc.tcp://10.1.0.2:50000_e873ab7d8040cb3b79e7b4a38296093f97c32da3",
			"DataSetWriterId": "opc.tcp://10.1.0.2:50000_c70079e5431fea7bfa4f9fbba7f2aad440d55efb"
		}
	}
]

which means that the full featured messages is on. So far so good.

Last, with this option

\"Cmd\": [\"--autoaccept\", \"--logfile=logs/opcuaPublisher.log\", \"--publishfile=workspace/nodes.json\", \"--fullfeaturedmessage=true\"]

we get back to messages without full payload

[
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Value": {
			"Value": false,
			"SourceTimestamp": "2021-09-28T15:24:03.903729Z"
		}
	},
	{
		"NodeId": "http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean",
		"ApplicationUri": "urn:OpcPlc:b0803e843ad2",
		"DisplayName": "AlternatingBoolean",
		"Value": {
			"Value": true,
			"SourceTimestamp": "2021-09-28T15:24:08.9040022Z"
		}
	}
]

Is there a problem in the "long version" of the command line?

This is visible also if we use the long version of the --mm option, as I mentioned above. When I opened this issue I used the short for simplicity without knowing that there was a problem behind.

Moreover, i had a look to this source code and the default is told to be "True".

Is there a problem in the description too?

Thanks

[cristipogacean]

Hello @alevincenzi,

Indeed I can replicate the issue with --fullfeaturedmessage=true. This seem to be a bug related to string lowercase comparison of the arguments name since the --FullFeaturedMessage=true works fine. As you said, the same issue might be the issue with other arguments you have been using.

We will fix this long-names issue as well as the default mismatch in the documentation of the --fullfeaturedmessage.

Thank you for taking the time to go to the bottom of this issue!

[alevincenzi]

Hello!

thank you for he update.

We'll be waiting for the next release. As for today, we are gong to use the command line options in short format.

Regarding the other open points about the payload,

The missing MonitoredItem is not a problem. I reported it just to list the differences with the documentation.

The expansion of the NodeId can be problematic for us. We use the DisplayName whenever possible and we put it in the configuration of the publisher (--pn=node.json). This way the publisher puts it in the telemetry sent as output. If the display name is not set by "our users", we use the nodeId to identify the opc node but with the 2.8.0 the nodeId does not match the one in the configuration. It changes from ns=2;s=AlternatingBoolean to http://microsoft.com/Opc/OpcPlc/#s=AlternatingBoolean and the match fails. It would be nice to keep the one in the nodes file. Can you put an option to keep the original name?

Thanks.

[cristipogacean]

fixed in main branch