Incremental TDs provided by constrained devices

[mmccool]

Constrained devices should be able to incrementally provide and register TDs

[mmccool]

on a 32K device we may only have a few K free to send a TD, so may have to pull it out of flash and fill in blanks (and compute the signature!)

[mmccool]

right now we are going to define profiles and reference platforms so the entire TD can fit in memory (eg 256K) for processing prior to being provided for self-description and directory registration but we should design our network interfaces so smaller devices are possible...

[farshidtz]

An alternative is to have those constrained devices only self-describe or register a minimal TD with a link (see #34 (comment)) pointing to the complete TD hosted in a directory (registered manually or by a more powerful device) or elsewhere.

[farshidtz]

Constrained devices should be able to incrementally provide and register TDs

I think this should be handled at the protocol level. HTTP requests and responses can be transmitted in chunks (see https://en.wikipedia.org/wiki/Chunked_transfer_encoding). But HTTP/2 doesn't support that and instead offers framing.

[benfrancis]

I think this should be handled at the protocol level. HTTP requests and responses can be transmitted in chunks (see https://en.wikipedia.org/wiki/Chunked_transfer_encoding).

See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

[farshidtz]

I think this should be handled at the protocol level. HTTP requests and responses can be transmitted in chunks (see https://en.wikipedia.org/wiki/Chunked_transfer_encoding).

See also: https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

@benfrancis I understand that Range requests should be initiated by the client. Do you know of any specification for a server to refuse full (normal) requests by clients and only respond to partial requests? Will returning 416 Range Not Satisfiable make sense in such situation? It is meant for an unsatisfiable range. Or 413 Payload Too Large, considering that payload is too large for the request? I am more tempted toward 416.

I'm asking because in the case of limited devices, the device has no way of returning the whole resource at once if the clients asks for it.

[benfrancis]

Do you know of any specification for a server to refuse full (normal) requests by clients and only respond to partial requests? Will returning 416 Range Not Satisfiable make sense in such situation? It is meant for an unsatisfiable range. Or 413 Payload Too Large, considering that payload is too large for the request?

Looking at the RFCs it appears that 413 means the request is too large, not that the response would be too large. 416 is really meant for invalid ranges, which is maybe a bit of a stretch. You could potentially borrow 507 from WebDAV, but probably better to just fall back to something generic like 500.

If the client doesn't know to ask for a range, then it might be better for the server to just respond with a chunked transfer encoding.

But practically speaking, if a server really can't manage to serve metadata about about its capabilities over HTTP then it's likely it can't manage to serve actual data about its state either and it should probably be using a more lightweight protocol and serialisation like CoAP + CBOR instead. If the resources are still too big then something like Block-Wise transfers might help?

[mmccool]

So this is sounding a lot like pagination, and perhaps we should deal with this issue consistently in both self-description and directory responses.

[farshidtz]

I plan to add a section under Exploration Mechanisms to recommend protocol-specific mechanisms for incremental transfer of payloads. We can refer to this section from both Self-Description and Directory sections to recommend incremental submission and retrieval. The paragraph about chunking/streaming would then be removed from Directory's Listing section.

[farshidtz]

For reference, this has been added to the spec, under exploration mechanisms as one of the use cases for ThingLink type:

A self-describing Thing with limited computational resources intends to describe itself: host a minimal TD (Thing Link) locally and references a larger one with the full details hosted at a different URL, perhaps in a directory.