View Issue Details

IDProjectCategoryView StatusLast Update
000537210000-014: PubSubSpecpublic2021-06-07 17:40
ReporterZbynek Zahradnik Assigned ToMatthias Damm  
PrioritynormalSeverityminorReproducibilityalways
Status closedResolutionfixed 
Summary0005372: Keep-alive message in JSON unclear or unfeasible
Description

In JSON DataSetMessage text, we have: "DataSetWriters may periodically provide keep-alive messages which are DataSetMessages without any Payload field.".

This needs explanation - or need to be removed.

The period mention here cannot be the normal KeepAliveTime because its minimum is PublishingInterval, and there WILL be one DataSetMessage each PublishingInterval, because there are no delta frames and no event messages in JSON.

So, can there ever be a keep-alive in JSON? Or, is it meant to provide a keep-alive that is somehow more frequent than the PublishingInterval? How would that be configured - and would it be good for anything (I doubt it)?

TagsNo tags attached.
Commit Version
Fix Due Date

Relationships

related to 0006996 closedRandy Armstrong NodeSets, XSDs and Generated Code Keep-alive message in JSON unclear or unfeasible 

Activities

Matthias Damm

2020-06-09 20:28

developer   ~0012220

KeepAlive message depend on the parameter

  • PublishingInterval (WriterGroup)
  • KeepAliveTime (WriterGroup) (The minimum value shall equal the PublishingInterval)
  • KeyFrameCount (DataSetWriter)

These three parameters are independent of the Message Mapping.

If KeyFrameCount is 0 or >1, there can be KeepAlive messages (also for JSON message mapping)

Zbynek Zahradnik

2020-06-10 06:38

developer   ~0012223

I disagree with MatthiasD's note https://opcfoundation-onlineapplications.org/mantis/view.php?id=5372#c12220 .

The part where he has it wrong is: "If KeyFrameCount is 0 or >1, there can be KeepAlive messages (also for JSON message mapping)".
That might be what the intent of the spec was, but the spec does not allow that currently.

In reality, the Delta Frames are only defined in UADP mapping. As opposed to that, for JSON, the spec does not describe that they exist, nor how they should look like. The only logical conclusion for the reader is that they do not exist in JSON - because he cannot construct them.

Matthias Damm

2020-06-10 08:20

developer   ~0012225

Technically Delta Frames exist for JSON. The spec may be silent on it.

In UADP Delta Frames are special since they contain an array of index/value pairs for the changed fields.
In JSON, the name of the field is always provided and therefore the index is not necessary.
A Delta Frame with JSON is just a regular message that does not contain all fields.

If the missing clarification for Delta Frames is the only remaining issue, this can be fixed.

Matthias Damm

2020-06-10 11:03

developer   ~0012227

Added clarification for key and delta frames in JSON encoding:

7.2.3.3 DataSetMessage
A key frame DataSetMessage or an event based DataSetMessage contains name and value for all fields of the DataSet.
A delta frame DataSetMessage contains only name and value for the changed fields.

Added in OPC 10000-14 - UA Specification Part 14 - PubSub Draft 1.05.26.docx

Zbynek Zahradnik

2020-06-11 12:46

developer   ~0012240

I think the clarification resolves 99% of the issue, thank you.

But I am reopening this issue just for one last consideration: If both delta and key frames exist in JSON, and their only distinction is that the key frame "contains all fields", it is impossible to distinguish key frames without having the metadata. You can say that this was the intent and that the subscriber always must have the metadata. But, it would be easy to resolve it e.g. by adding something MessageSubtype or MessageKind field to JSON NetworkMessage header.

Consider the following use cases where the necessity to have the metadata causes a problem:

The Delta Frames is basically a protocol-only concept to save bandwidth. Semantically, on the application level, it "should not exist": a (data-based) DataSet is is seen by the consuming application as a stream of incoming "packets" where each packet has all the fields.

If all you want to do is, for example, transform the stream with Delta/Key frames to a stream that can be consumed by the application (the one that always has all fields), you need to assemble the data from the last Key Frame and the data from subsequent Delta Frames. This means that you cannot emit any data to the application before the first Key Frame arrives. If you have explicit indication of which frame is the Key Frame (as it is in UADP), you can simply wait for a message with that indication. But with JSON, I would have no way to tell which frame is the key frame, without having to do all the stuff that relates to obtaining the metadata. I cannot, for example, write a simple logger that would log one DataSet data per row, without the metadata. I cannot even write a processor that would remove the Delta Frames and generate a stream with Key Frames only, for easy consuming by other applications, without the metadata. And what's missing is really just a single bit of information that would distinguish the Key Frames from Delta Frames.

Matthias Damm

2020-06-17 13:25

developer   ~0012385

Added MessageType to 6.3.2.3.1 DataSetMessageContentMask
Added MessageType to 7.2.3.3 DataSetMessage

Jim Luth

2020-06-17 13:26

administrator   ~0012386

Agreed to changes edited in Draft 1.05.27 in virtual F2F.

Matthias Damm

2021-05-03 07:51

developer   ~0014288

We should have an 1.04 errata for this issue

Matthias Damm

2021-06-07 15:03

developer   ~0014477

Added to errata 1.04.10
Need to clone to Nodeset for 1.04.10

Jim Luth

2021-06-07 17:40

administrator   ~0014494

Agreed to 1.04.10 Errata.

Issue History

Date Modified Username Field Change
2020-01-07 16:13 Zbynek Zahradnik New Issue
2020-05-19 18:09 Jim Luth Assigned To => Matthias Damm
2020-05-19 18:09 Jim Luth Status new => assigned
2020-06-09 20:28 Matthias Damm Status assigned => resolved
2020-06-09 20:28 Matthias Damm Resolution open => no change required
2020-06-09 20:28 Matthias Damm Note Added: 0012220
2020-06-10 06:38 Zbynek Zahradnik Status resolved => feedback
2020-06-10 06:38 Zbynek Zahradnik Resolution no change required => reopened
2020-06-10 06:38 Zbynek Zahradnik Note Added: 0012223
2020-06-10 08:20 Matthias Damm Note Added: 0012225
2020-06-10 11:03 Matthias Damm Status feedback => resolved
2020-06-10 11:03 Matthias Damm Resolution reopened => fixed
2020-06-10 11:03 Matthias Damm Note Added: 0012227
2020-06-11 12:46 Zbynek Zahradnik Status resolved => feedback
2020-06-11 12:46 Zbynek Zahradnik Resolution fixed => reopened
2020-06-11 12:46 Zbynek Zahradnik Note Added: 0012240
2020-06-17 13:25 Matthias Damm Status feedback => resolved
2020-06-17 13:25 Matthias Damm Resolution reopened => fixed
2020-06-17 13:25 Matthias Damm Note Added: 0012385
2020-06-17 13:26 Jim Luth Status resolved => closed
2020-06-17 13:26 Jim Luth Fixed in Version => 1.05
2020-06-17 13:26 Jim Luth Note Added: 0012386
2021-05-03 07:51 Matthias Damm Status closed => feedback
2021-05-03 07:51 Matthias Damm Resolution fixed => reopened
2021-05-03 07:51 Matthias Damm Note Added: 0014288
2021-06-07 15:03 Matthias Damm Status feedback => resolved
2021-06-07 15:03 Matthias Damm Resolution reopened => fixed
2021-06-07 15:03 Matthias Damm Note Added: 0014477
2021-06-07 17:38 Jim Luth Issue cloned: 0006996
2021-06-07 17:38 Jim Luth Relationship added related to 0006996
2021-06-07 17:40 Jim Luth Status resolved => closed
2021-06-07 17:40 Jim Luth Note Added: 0014494