Docs for local "$refs"

[tryangul]

I'd love to use the library, but I can't unless I can easily get local "$refs" working.

If this is supported, it could use some documentation / examples.

See https://json-schema.org/understanding-json-schema/structuring.html#reuse for what I'm talking about.

[davishmcclurg]

I believe it should work as described in the documentation you linked. Do you have an example I can use to reproduce your issue?

[tryangul]

I greatly appreciate you taking the time to look into this. Regardless of the outcome I think adding a couple examples of using local files as refs to the documentation would be very useful.

Here's a gist with reproducible data:
https://gist.github.com/tryangul/573e4ec857f2d55ed821b0aeddb6631f

Note: the root schema is located in /json_schemas while the sub schemas are located in /json_schemas/_sub_schemas

FWIW, these schemas work with ruby-json-schema

[davishmcclurg]

It looks like the issue is your $ref URLs are relative but there's no base URL to resolve them against. I think you could get it working by defining an $id value for the root schema and parsing it out in the ref resolver. From what I can tell, the current behavior is correct according to the spec, but it's not ideal. How does this work in ruby-json-schema?

[tryangul]

This works with 0 configuration in ruby-json-schema.

Regarding the lack of $id on the root schema, I believe the spec says in this case the base URI should be the URI in which the schema was found see: http://json-schema.org/latest/json-schema-core.html#rfc.section.8.1

It is also my understanding from the spec and the definition of SHOULD here, the use case of not supplying a root $id should be supported.

I will test out providing an absolute file:/ based URI to the root component later as a suitable workaround.

[davishmcclurg]

Those spec links are helpful, thanks. In this case, we don't know the URI where the schema was found, so I think this applies:

Informatively, the initial base URI of a schema is the URI at which it was found, or a suitable substitute URI if none is known.

Not sure what a suitable substitute would be.

Do you mean that ruby-json-schema reads the schemas from the filesystem with zero config?

[tryangul]

In this case, we don't know the URI where the schema was found

Is this because json_schemer only accepts ruby hashes? Should it not also accept raw json strings and json files?

I will experiment with generating the an absolute file:/ URI and setting it as the $id on the schema hash

Do you mean that ruby-json-schema reads the schemas from the filesystem with zero config?

Yes, it reads them from the file system with 0 config. The example I provided in the gist works with the following ruby-json-schema based script

def validate_json(schema_path, json_path)
  {
    is_valid: JSON::Validator.validate(schema_path, json_path),
    errors: JSON::Validator.fully_validate(schema_path, json_path)
  }.to_json
end

puts validate_json(ARGV[0], ARGV[1])

Note: in this case, it knows where the schema was found as it accepts paths.

[davishmcclurg]

Is this because json_schemer only accepts ruby hashes? Should it not also accept raw json strings and json files?

Yeah, right now there's no way to know what the base URI should be for the schema hash. I'd rather not allow raw json, since it's pretty easy for someone to parse it into a hash themselves. I'll look into reading from files though. I'm a little wary of doing it automatically but it makes sense as an optional thing.

[tryangul]

I'd rather not allow raw json, since it's pretty easy for someone to parse it into a hash themselves.

Fair, but it seems very weird to implement a JSON spec that doesn't accept JSON.

FWIW, I got it working with the following https://gist.github.com/tryangul/07f4a46b00cf244f2a95317966fb121e

I still advocate adding something like this to the docs and or extending the ref resolver to allow for more easily setting this up.

If you're willing, I'd happily make a docs PR and we can move discussion there.

[tryangul]

Also, probably for another issue, but while this is working, the output is less than ideal

https://gist.github.com/tryangul/de71c05bf5d3553ecc8e433a113ab815

Is this related to #1 ?

[davishmcclurg]

Glad to hear you got it working. You should try out this branch to see if it behaves like you expect: https://github.com/davishmcclurg/json_schemer/compare/file

A docs PR would be great! Docs are basically nonexistent at this point.

The error output for nested schemas is not good. There's been a lot of discussion about standardizing error output somehow: json-schema-org/json-schema-spec#396 json-schema-org/json-schema-spec#643
I didn't want to implement anything more until that was done, but I haven't kept up with it very well.

[Draiken]

I have to say I'm struggling to understand how to use this gem with local files.

I have a bunch of nested schemas and can't seem to validate a simple piece of data.
I've made a small example reproducing my issue.

I've tested the same nested schema (after expanding it) and it seems to validate fine.
By reading the docs I think this should work as expected, but it still gives me (somewhat cryptic) validation errors.

Am I missing something obvious here?
Once I get this figured out I'll see if I submit a PR for some documentation around this.

[davishmcclurg]

@Draiken this looks like another case of using non-string keys (#13).
Your example works if you change it to:

pp schemer.validate({ 'action' => 'check', 'seatId' => 1 }).to_a

This seems to be one of the main issues people have. I'll think on how to fix it.

[Draiken]

Oh I wasn't aware of that.
This is unexpected behavior in my view, since this is dealing with JSON where keys are are always strings and the default behaviour of the JSON gem is to convert symbols to strings when serializing.

I'll see if I can get it working with strings.

BTW, thanks for the gem man. Too many times people just come in complaining about issues/errors but before it all, know that you have my (probably everyone's) appreciation for the work you've put into it.

[joecabezas]

I am having issues to trying to make external $ref to work, I am trying to make the $id value a relative path, but the issue is that if the $ref value is also relative it returns an error:

URI::BadURIError Exception: both URI are relative

I would like in the docs an example of a local ref working, please advice, and thanks for the hard work!

[jimeh]

Just an update for anyone else who runs into this.

If you pass in a Pathname instance pointing to the file on disk, instead of the raw JSON content when calling JSONSchemer.schema, it knows where the original schema lives on disk, and relative $ref attributes work fine.

TL;DR:

This:

JSONSchemer.schema(Pathname.new('/my/awesome/schema.json'))

Instead of:

JSONSchemer.schema(File.read('/my/awesome/schema.json'))

[Draiken]

@jimeh I couldn't get this to work. Could you give an example of how you're referencing the files?

schema = JSONSchemer.schema(Pathname.new('broadcasts.json'))

# broadcasts.json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "broadcast.json",
  "name": "Broadcast",
  "type": "object",
  "oneOf": [
    { "$ref": "file://./broadcasts/connections.json" }
  ]
}

# broadcasts/connections.json
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "connections.json",
  "name": "Connections",
  "type": "object",
  "properties": {
    "name": { "type": "string" }
  }
}

I tried using file:./path, file://./path, ./path, path and none of them worked.

[davishmcclurg]

@Draiken you may need to use an absolute path when creating your Pathname object.

[jimeh]

@Draiken try giving Pathname an absolute path like @davishmcclurg suggested. The code I've got passes in an abosolute path by doing something like this:

filename = 'foobar.json'
file_path = File.expand_path(File.join('..', 'schemas', filename), __dir__)
JSONSchemer.schema(Pathname.new(file_path))

Also, the schema refs I've got don't use any scheme:// prefixes, so it's just:

{ "$ref": "./broadcasts/connections.json" }