Authenticated RDF (or RDF/JSON-ld on IPLD)

[nicola]

Some friends and I are working on how to bring authenticated data structures on RDF.

This consists in:

• Bringing secure links (either self-certifying or content addressed) to RDF
• Return very light proofs along with the result of simple operations that can be done in a triple store: add, update, delete, read/query/find triples

In practice, this effort is about the following:

Step 1: data model

1. normalize the rdf graph efficiently (see @msporny work)
2. if (Spec refining: Relative paths in IPLD ipld/specs#1) then replace the urls with relative paths (or (multi)hashes), otherwise build a causally linked history that can show circular links
3. hash everything (build a merkle graph)
4. Solve Clarify how the use of URI identifiers maps to IPLD graphs #155

Step 2: operations on graphs

• find optimal structures for crud proofs (updating client & server's digest - inspired by CONIKS, history trees & co)

Smallest tool that we can build:

• rdf to hashable graph
• hashable graph to IPLD

cc @flyingzumwalt, @csarven, @bergos

[flyingzumwalt]

@edsu @cmh2166 @cbeer @jcoyne @escowles @mjgiarlo @dchud @jpstroop @chadfennell

[flyingzumwalt]

cc @anarchivist

[sohkai]

I'm coming without much context, so please enlighten/ignore me if I'm totally missing the point :).

You've probably also already seen this, but to get the obvious out, @msporny has been working on Linked Data Signatures that might address a lot of Step 1.

authenticated data structures on RDF

From #149 and here, there's some talk about authenticated data structures and IPLD. Not really sure what that means; is it just talking about content addressing?

find optimal structures for crud proofs

This talk about CRUD has me really confused, but I'm probably missing out on some planned changes to the spec since I assume IPLD enforces immutability.

rdf to hashable graph

Isn't this JSON-LD?

hashable graph to IPLD

There are some incompatibilities with JSON-LD and IPLD (one being "@id" support for merkle-links), but being able to express JSON-LD as IPLD would also address this, no?

[nicola]

@sohkai

Not really sure what that means; is it just talking about content addressing?

Content-addressing authenticates the content that matches the hash.

When you have a graph and you want to add, create, search, you are doing some operations on this graph and you want to make sure that also these operations are authentic (so that no one is getting you wrong data). Basically bringing this power that content-addressing gives you further not just to retrieve the entire file, but also to do other operations.

Isn't this JSON-LD?

JSON-ld and @msporny work is great for RDF -> authenticated RDF, since (1) it gives us a nice data model (JSON) working on RDF (JSON-LD) and a normalization algorithm (rdf-normalize) so that we can have JSON objects that are normalized and easy to hash. Which is basically my Step 1.1. Now, we have to replace the URLs in existing JSON with hashes URIs (goodbye URLs), which is Step 1.2 and 1.3

(one being "@id" support for merkle-links)

@id is not a keyword in IPLD, / is!

[sohkai]

@nicola: Thanks for clarifying!

@id is not a keyword in IPLD, / is!

But this is the problem -- if you want backwards compatibility with JSON-LD (which I would assume, since there's no reason to throw out what they've done in terms of compatibilities with RDF, normalization, and context framing), these two things have to be resolved. JSON-LD doesn't really care what's in the IRIs (they could just be hashes); it only cares that it's a string in the form of one; this is where / breaks down: current JSON-LD parsers break when seeing something like:

{
   "@id": {
       "/": "<hash>"
  }
}

My pipe dream (not necessarily the most elegant approach though) with Step 1.2 would be tool that could transform IPLD to/from a syntactically-correct JSON-LD, although perhaps with non-sensical IRIs. Assuming the user wants to work in IPLD structures, they could parse the IPLD down into JSON-LD, frame it into their context, and then transform the resulting structure back to their final IPLD structure. The resulting structure is non-authenticated, but that's OK since you can authenticate the initial structure.

[nicola]

I have an intuition why we should not worry about the break and why JSON-ld and IPLD are on two different layers.

The / just defines the pathing of the physical representation, while the @id defines what is the ID of the semantic representation of a file, so you will never run in your particular examples.

The link object is more of an include/import object rather than a link. So that you can consider the @id as a link object. Not sure if this makes sense, I will write down a better differentiation between the two and how this can work soon!

[jbenet]

(I'm not following the whole discussion, but can always transform

{
"@id": { "/": "" }
}

Into

{
"@id": "fs:/ipld/"
}

And interpret it how you want.
On Wed, Aug 10, 2016 at 07:04 Nicola Greco [email protected] wrote:

I have an intuition why we should not worry about the break and why
JSON-ld and IPLD are on two different layers.

The / just defines the pathing of the physical representation, while the
@id defines what is the ID of the semantic representation of a file, so
you will never run in your particular examples.

The link object is more of an include/import object rather than a
link. So that you can consider the @id as a link object. Not sure if this
makes sense, I will write down a better differentiation between the two and
how this can work soon!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#152 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIcofrrYDxhlkmUWny_PL1Pp8f9Lt-Uks5qebAugaJpZM4JdNS4
.

[sohkai]

@nicola:

The / just defines the pathing of the physical representation, while the @id defines what is the ID of the semantic representation of a file

But in the case of IPLD and content addressing, isn't the pathing of the physical representation also the ID of the file? The @id defines a IRI where the object can be located. To avoid confusion, and I apologize for not being exact in the earlier reply, I meant something like either of these two (more fleshed out) structures:

{
    "@context": {
        "url": {
            "@type": "@id"
        }
    },
    "url": {
        "/": "<hash>"
    }
}

// Or, without the context:
{
    "url": {
        "@id": {
            "/": "<hash>"
        }
    }
}

Having the "@id" property on an object isn't really necessarily in IPLD (see #149 (comment)).

I will write down a better differentiation between the two and how this can work soon!

I think this will greatly help us both understand where we're coming from 😄!

@jbenet Is the second example you've shown a merkle-link? In the examples of what a merkle-link is (https://github.com/ipld/specs/tree/master/ipld#what-is-a-merkle-link), it says specifically that

"bar": "/ipfs/QmUmg7BZC1YP1ca66rRtWKxpXp77WgVHrnv263JtDuvs2k"

is not a link.

But along the lines of that, that's more or less what I meant when I said I wanted a transformation tool between IPLD to turn its merkle-links into regular IRIs.

[edsilv]

https://www.w3.org/TR/json-ld/#node-identifiers

To be able to externally reference nodes in a graph, it is important that nodes have an identifier. IRIs are a fundamental concept of Linked Data, for nodes to be truly linked, dereferencing the identifier should result in a representation of that node. This may allow an application to retrieve further information about a node.

So if you had some graph in memory and wanted to see what else was linked to a given node, you'd dereference the IRI in whatever way makes sense to the current application and inspect it.

In the IPLD universe, we can't include the @id because it constitutes a circular reference. However, we don't need to include it to dereference the node, because what you have in memory is the node - There is no "further information" to retrieve.

When transforming from IPLD > JSON-LD we'd take the hash of the current object and include it as an fs:/ipld/<cid> @id property in the JSON-LD serialisation. As long as it's a valid IRI we're adhering to the spec, as @jbenet says the application can interpret it however it wants. Transforming back to IPLD we'd first remove the @id property, correct?

[nicola]

quick answer:
juan comment is the right way of doing this, that is not an IPLD link, but an application level (so json-ld) link. We should use IPLD links only to include an object under a path.

quick update (not much content, but very excited to share):
The conversation that is developing right now is really great! I have been writing a set of notes on this topic in the last few months (I was originally working on bringing integrity on RDF, but we can do much more!). Today, I had a great occasion to go through them with some great linked data people that I hope will join us in making this and we laid out a simple way forward (and a simple abstraction, hopefully easy to explain) to get json-ld on IPLD the right way and tried to finally solve authentication of RDF (just step 1). I will write this down in the next few days!

[jbenet]

Exciting!
On Thu, Aug 11, 2016 at 19:06 Nicola Greco [email protected] wrote:

quick update: juan comment is the right way of doing this, that is not an
IPLD link, but an application level (so json-ld) link. We should use IPLD
links only to include an object under a path.

another quick update (not much content, but very excited to share): The
conversation that is developing right now is really great! I have been
writing a set of notes on this topic in the last few months (I was
originally working on bringing integrity on RDF, but we can do much more!).
Today, I had a great occasion to go through them with some great linked
data people that I hope will join us in making this and we lied out a
simple way forward (and a simple abstraction, hopefully easy to explain) to
get json-ld on IPLD the right way and finally solve authentication of RDF
(just step 1). I will write this down in the next few days!

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#152 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAIcoasT3KVZRnS75ipGcnusKtNk6oV-ks5qe6sOgaJpZM4JdNS4
.

[joeltg]

I will write this down in the next few days!

@nicola Did this ever happen? I'm exploring this space again for the Underlay project and would love to see your notes if you ever got them down.

[msporny]

There has been a ton of work in this space since the inception of this issue in the W3C Verifiable Claims Working Group, W3C Credentials Community Group, and W3C Digital Verification Community Group.

Further links into how this technology could help the Underlay Project (and IPFS projects in general):

• https://w3c.github.io/vc-data-model/
• https://github.com/digitalbazaar/jsonld-signatures/
• https://github.com/WebOfTrustInfo/rebooting-the-web-of-trust-spring2018/blob/master/topics-and-advance-readings/did-primer.md

[Schwartz10]

bumping @joeltg 's comment. Has any progress been made on this front? cc @nicola

[vmx]

@Schwartz10 I haven't read the full ticket, but there was activity over here which sounds related.