Expose recipe

[peterbe]

Perhaps somewhat similar to #26 I would like more in the built .json files. In particular, I want the recipe too.

We could just take ALL the fields from the src/**/video/meta.yaml file and put them into a key like "meta" in the packaged/**/video.json. E.g.

json.load(open('video.json'))['meta']['mdn-url']

Even better would be if that (the "recipe") can be looked up too so I don't need to do:

recipe_name = json.load(open('packaged/blabla/video.json'))['meta']['recipe']
recipe = yaml.load(open(f'node_modules/stumptown/recipes/{recipe_name}.yaml'))
for section in recipe:
    ...

Meaning, I'd rather just do:

recipe = json.load(open('packaged/blabla/video.json'))['meta']['recipe']
for section in recipe:
    ...

[ddbeck]

I'm not clear on something: why would a consumer want a recipe?

[peterbe]

I'm not clear on something: why would a consumer want a recipe?

I'm using stumptown to build HTML pages. I get the video.json and next I need to get the recipe so I know exactly which order I would pluck keys out of that video.json information.

[ddbeck]

Ah! That's an interesting question. @wbamberg and I went back and forth on this a little (near the top of this comment is probably the most interesting bit). I understood Will's comment to mean that the recipe's order is incidental, rather than canonical. Whether (and what in order) to show sections is a decision for the consumer. Unless we're changing our minds on that 😄

[wbamberg]

There's an unresolved question that I've tried to avoid answering, which is something like: where's the line between stumptown as an abstract collection of bits of documentation about some web technologies, and stumptown as a concrete description of a collection of MDN pages.

So partly, a recipe describes the pieces that comprise a set of documentation on a particular thing. For example, an HTML element needs to have some BCD, and needs to have a short description, and may optionally have an interactive example. We can use the recipe here to validate that we do actually have all the bits, and in building the JSON, as a guide to including all the bits in the output. For this purpose the order of items in a recipe doesn't matter, and the recipe must account or everything in the stumptown repo.

But also, MDN wants to build web pages out of the stumptown repo and for that of course it does care about the order. And the recipe is also, at the moment, playing this role, and does present the pieces in the order that they should appear in an MDN page.

It's a bit dodgy that the recipe is playing both sides here. We could imagine it being a real problem if, for example, MDN decided it didn't want to show CSS formal syntax in its pages, but we still want to keep that content in stumptown (because, for example, VS Code wants to use it).

So perhaps at some point (and maybe now!) we should separate these two roles, and have two sets of things:

• one that describes what must/may be in the repo for a given item
• one (probably outside the stumptown repo) that tells MDN how to build web pages out of stumptown content

I think we might get into similar strangeness when we look into sidebars. Sidebars are quite specific to MDN, but they are still content. So I'm not quite sure where the specification for sidebars lives or how it relates to the stumptown content.

I understood Will's comment to mean that the recipe's order is incidental, rather than canonical. Whether (and what in order) to show sections is a decision for the consumer.

This is a bit different, or at least that was my intention. My comment there was supposed to be about the order of sections as they are written in prose.md, not as they are listed in the recipe. And about how choices we make about this may constrain the way prose.md should be written.

(Of course, if the recipe doesn't mandate an order - so for example MDN pages could generate a random order each time - then the prose.md is not allowed to assume an order. But if the recipe does mandate an order for MDN, that doesn't necessarily mean prose.md is allowed to assume an order - some other consumer may decide to drop sections or reorder them.)

(I actually think ordering sections of prose.md is a bit of a red herring. We're unlikely to want to reorder them. But we or other consumers might want to omit some sections or present the independently of the rest (e.g. an accessibility tool).)

[ddbeck]

Thanks for that discussion, Will. It's fair that there are unanswered questions (and it's fair to not necessarily answer them all just yet).

So perhaps at some point (and maybe now!) we should separate these two roles, and have two sets of things:

• one that describes what must/may be in the repo for a given item

• one (probably outside the stumptown repo) that tells MDN how to build web pages out of stumptown content

Yeah, I think this is an important step we should take sooner rather than later. If nothing else, it'll help us find the sharp edges of consuming stumptown. I expect some things to go back and forth across our boundaries as we experiment.

Until we have a second consumer, however, we won't be able to readily distinguish what's good for generating MDN pages specifically and what's good for consuming stumptown broadly. But we can ask ourselves questions like, who else is going to use the order of sections? If we can imagine consumers other than MDN, then it seems to me that we'd want to bake that information into the content structure we provide consumers (e.g., provide the section objects as an array instead of an unordered collection). To completely torture the recipe metaphor: if we're going to bake our consumers a cake, then we shouldn't make them to come into the kitchen to eat it.

[peterbe]

To me, the most important feature of the recipe is that it allows us to CI validate that nothing's missing from the content that should be there. We could just put that "hardcoded" into a script that gets run with every PR.

Stumptown should strive to be as pure as possible and focussed on the content. We could, in CI do something like this:

yarn add stumptown-cli 
stumptown-cli run content/*

Regarding order I think it's kinda crazy that that's not part of the content itself. At least for the prose sections. But having said that, it sure would be nice to have something that helps with maintaining consistency across documents.

[wbamberg]

Next you'll be telling me that if we're going to bake them a cake we shouldn't put broken glass and rusty nails in it. So ungrateful!

who else is going to use the order of sections?

This is a good point. The order of sections isn't just arbitrary. I mean, we have "short description" first, then "overview"...then it gets a bit arbitrary, but consistency is still desirable...then "see also" at the end. So perhaps listing them in the JSON in order is a good idea.

But the MDN renderer will still need something telling it how to construct a page. For one thing, you're still going to have to have something that tells the renderer how to interleave non-prose into prose - for example that in between "short description" and "overview", we should have an interactive example.

Regarding order I think it's kinda crazy that that's not part of the content itself. At least for the prose sections.

I'm not exactly sure what this means, but: we could have a system that just accepts whatever order prose sections are written in, and builds pages out of them. But as you say that means pages can get inconsistent. Or: there is a defined order that is validated for a reference page, but generally authors don't have to think about it because it's the natural order that a single prose.md file is written in, and I think that is what we are aiming for.

[wbamberg]

I think this is now obsoleted by #107.