NDR Principles

[cabralje]

In NIEM 5.0 the scope of the NDR includes 29 guiding principles and 260 rules. For reference, the complete definitions of each principle are (of course) in Section 6 of the NDR itself.

The following principles, with one noted exception, are not enforced in any rules.

1 Keep specification to a minimum
2 Focus on rules for schemas
3 Use specific, concise rules
4 Purpose of XML schemas
6 Use XML validating parsers for content validation
8 Allow multiple schemas for XML constraints
9 Define one reference schema per namespace
11 Specify types for all constructs
13 Schema locations provided in schema documents are hints
18 Avoid displaying raw XML data
23 Reflect the real world (1 instance rule)
25 Reserve inheritance for specialization
26 Do not duplicate definitions

In NDR 6.0, I suggest that, for each of these, we either:
a. Define at least one rule, or
b. Eliminate the principle

If you believe we should keep ANY of these rules in NDR 6.0, please comment and, ideally, suggest an appropriate rule. The rule may be a manually checked rule (it doesn't have to be implementable in Schematron) but it should be stated clearly enough that an objective observer can clearly determine whether an artifact is conforming.

[cdmgtri]

Principle 2 information could go to the the Scope section. And we may want to expand this a bit. I think it would make sense to talk about reference models and message models in the NDR.

[Principle 2]
This specification SHOULD focus on specifying characteristics of schema documents, their instance documents, and their interpretation.

Section 6.1.2 Focus on rules for schemas

[TomCarlson-NTAC]

Resolved to move these into Scope sections for each NDR 6.0 document.

[cdmgtri]

[Principle 9]
Each NIEM-conformant namespace SHOULD be defined by exactly one reference schema.

Section 6.2.6 Define on reference schema per namespace

This should remain in the XML binding, but isn't needed for NIEM JSON.

[TomCarlson-NTAC]

Resolved to keep in XML binding.

[TomCarlson-NTAC]

Rule 9-89 does enforce this.

[davaya]

Both XML and JSON schemas support namespaces, and Principle 9 should apply to all bindings.

In order to accommodate less capable tooling, NIEM Open could preprocess reference schemas for all bindings to resolve cross-namespace references and publish a single merged schema in addition to the collection of per-namespace schemas. It can be argued that XML isn't normally used that way, but the same advantages and disadvantages of referencing individual schemas vs. copying them all into a single schema apply regardless of data format.

The principle should apply to NIEM JSON, but a single pre-generated (resolved) reference schema can be published in addition to the individual schemas as a convenience.

[cdmgtri]

[Principle 13]
Schema locations specified within NIEM-conformant reference schemas SHOULD be interpreted as hints and as default values by processing applications.

Section 6.2.10 Schema locations provided in schema documents are hints

I think we'll need to keep this one in the XML binding as well. It allows schema location values to be overwritten with xml catalogs.

[iamdrscott]

I once thought NIEM needed a schema assembly specification. Conformance of a message to a message specification is defined in terms of XML Schema validation, but we never say what that schema actually is. With nothing more than the XML Schema specification, it technically could be anything. But now I think the schema assembly specification is "whatever CMFTool does" :-)

[TomCarlson-NTAC]

Take out "hint"?
"Can be overwritten by an XML catalog."
Is there a rule requiring schema locations? Rule 9-90 requires a namespace on the import, but doesn't include requiring schemalocation.

Resolved for something like "Have a rule to provide a schema location for a schema import, but can have other mechanisms such as an XML catalog?" in the XML binding.

[davaya]

OASIS has namespace guidelines at http://docs.oasis-open.org/specGuidelines/ndr/namingDirectives.html#xml-namespaces, and publishes documents at their namespace URLs for TC work products. We should ask if the same service is available for Open Project work products. I assume that NIEM Open namespaces will follow the OASIS rules.

If so, the hint is the default location, so "hint" can be taken out. There is no prohibition against downloading documents from locations other than the default. This applies to all bindings -- if a merged JSON schema is produced it should still have its own namespace.

[cdmgtri]

[Principle 23]
Component definitions in NIEM-conformant schemas SHOULD reflect real-world concepts.

Section 6.5.3 Reflect the real world

This is another I think we should keep. It helps with consistency with content coming from a lot of different sources, each potentially with their own ways of organizing and grouping components.

[TomCarlson-NTAC]

Resolved keep it in in some form
Does "real-world" actually exclude much?
Upgrade to a rule? Wouldn't be able to do Schematron for it.
70/260 rules are not mechanically testable; we could narrow that down
Does it need to be reworded to better constrain this the way we want.
"General versus system-specific concepts"

[iamdrscott]

[Principle 5]
The data of a NIEM-conformant XML document provided by an XML parser SHOULD NOT be modified by the process of validating the data against an XML Schema.

6.2.2. Prohibit XML parsing from constructing values

I wonder if we still want this one. We don't really follow this principle ourselves. If we did, we would put appinfo:deprecated="false" on almost every component in the reference schemas. Or we would tell people "we don't know if this is deprecated; maybe it is and maybe it isn't". But we don't do either of those things.

[cdmgtri]

I think our use of xs:token violates this as well. We don't get validation errors on enumerations with irregular whitespace on a token type. I assume certain parsers normalize the values if needed. Although maybe that just tells us we should switch to xs:string instead of xs:token for simple type bases for NIEM 6.0.

[iamdrscott]

When converting NIEM XML to NIEM JSON or NIEM RDF, we should collapse whitespace unless the message designer put <xs:whiteSpace value="preserve"/> into his schema. So I think we need to remove Principle 5.

[TomCarlson-NTAC]

Keeping xs:token might violate it, but does so in a defined way. Going to xs:string would make the modification/normalization parser-dependent.

[TomCarlson-NTAC]

But I agree that we can remove it.

[cdmgtri]

Section 8.4 Ensure XML parsing does not construct values

...not only does an XML Schema validating parser yield data from an XML document to its caller, it determines whether the XML document is valid against an XML Schema, and also provides an augmented infoset to the caller, constructed to reflect information implied by the schema, which may not appear in the original XML document.

XML Schema allows element and attribute declarations to provide default values. When an XML document does not contain a value for a component that has a default, the XML Schema validating parser will construct a value for the component. This is done through the use of the attributes default and fixed, both of which provide default values to attributes and element content. An XML Schema validating parser that validates an XML document against a schema that uses default or fixed may yield an infoset that is augmented, constructing values in the provided XML infoset where none existed in the original XML document.

NIEM schemas should not produce constructed values in the infoset. The process of XML Schema validation against NIEM schemas should provide for marking data as valid or invalid, but should not modify original infoset data with constructed values. The XML infoset yielded by a non-validating XML parser should be the same as that yielded by an XML Schema validating parser. Turning on schema validation should not alter the data received by the caller of the parser.

The prohibition of constructed values is supported by Section 6.2.2, Prohibit XML parsing from constructing values, above. It is also supported through a prohibition on all uses of default, and most uses of fixed on attributes and elements defined by NIEM-conformant schema documents.

Relevant rules:

Rule 9-13. No use of fixed on simple type facets
Rule 9-45. No element default value
Rule 9-46. No element fixed value
Rule 9-57. No attribute default values
Rule 9-58. No fixed values for optional attributes

[iamdrscott]

[Principle 11]
NIEM-conformant schemas SHOULD NOT use or define local or anonymous components, as they adversely affect reuse.

6.2.8. Specify types for all constructs

Perhaps this should read "NIEM-conformant reference schemas". Those are the schemas that are intended for reuse. Local types in message schemas are useful, and we probably have some conformance rules for them.

[TomCarlson-NTAC]

But haven't we always hoped that folks would reuse other folks' extensions. And the API will support adding community content, which would finally enable that, which would then need components to be global and named.

[TomCarlson-NTAC]

There are rules requiring properties and types to have names.

[cdmgtri]

Rules:

• Rule 9-10, Simple type definition is top-level
• Rule 9-25, Complex type definition is top-level
• Rule 9-36, Element declaration is top-level
• Rule 9-48, Attribute declaration is top-level

[iamdrscott]

I hate to say it, but most (all?) of the rule-less principles could be moved to the Understanding the NIEM Technical Architecture project note. Some of them are already there.