Specify children order in frontmatter #1193

panglesd · 2024-08-30T06:43:40Z

Now that we have a convention for hierarchical pages, we need a way for the author to specify the order.

This PR does that by using the frontmatter, using this syntax:

children: child1 dir1/ child2 dir2/

The trailing / for directories is needed to distinguish them from files.

This PR is mostly ready, but:

TODO warnings:
- warn when a children field is given in the frontmatter of a non index page,
- warn when an entry of the children list is not found (should it happen during linking of the page or during the compile-index command?)
- Test those warnings
Decide what to do when a children is omitted from the list:
- ❌ do not include it in the toc (I'm in favor, this is what is implemented)?
- ✔️ Include it at the end of the list (I'm not in favor)?
- ✔️ Warn the user?
Decide what to do when there is no children list in an index page:
- ❌ Warn the user? (not in favor!)
- ✔️ Use alphabetical order on unit names (in favor, this is what the current PR does)
- ❌ Use the alphabetical order on page main title (not in favor!)
Clean up/factor the implementation of PageToc.
Fix compat CI

Julow · 2024-09-05T16:02:33Z

src/driver/odoc_driver.ml

@@ -582,6 +582,20 @@ let run libs verbose packages_dir odoc_dir odocl_dir html_dir stats nb_workers
      (fun () -> render_stats env nb_workers)
  in

+  List.iter


Is all of this just debug ? Otherwise, it seems too verbose and opaque.

Yes, this is only showing filtered logs for debugging. I'll remove it.

Julow · 2024-09-05T16:04:27Z

src/model/comment.ml

  let fm, content =
    let fm, rev_content =
      List.fold_left
        (fun (fm_acc, content_acc) doc ->
          match doc.Location_.value with
          | `Code_block (Some "meta", content, None) ->
-              (parse_frontmatter content.Location_.value :: fm_acc, content_acc)
+              (content.Location_.value :: fm_acc, content_acc)


The previous code was more logical (parse each blocks and concatenate the values rather than concatenate the blocks and parse something no one wrote) and allowed more graceful handling of errors with locations.
Anyway, that's a temporary syntax.

Indeed, we are going to have the frontmatter consistently at the top, so there won't be any distinction.
So, I suggest leaving it as is.

Julow · 2024-09-05T16:15:21Z

src/model/sidebar.ml

+
+type library = { name : string; units : Paths.Identifier.RootModule.t list }
+
+type page_hierarchy = { hierarchy_name : string; pages : toc }


I didn't expect the page hierarchy to have a name and be an extra level on top of a toc. What if this hierarchy contains an index.mld and a children order ? Is here an extra indentation with the name repeated ?

This "hierarchy_name" corresponds to the name of the library's package: the one passed in the -P option of odoc compile-index.
It is currently turned into a "{hierarchy_name}'s Pages" title:

# Package1's Pages - The best way to do foo - Guide - example - tutorial # Package2's Pages - Blazingly fast bar - Guide - example - tutorial # Library1's modules - Foo - Bar

We could possibly remove this hierarchy_name, but I would argue that it is better done in another PR (this PR is not introducing the behavior).

(Maybe:

# Pages - The best way to do foo - Guide - example - tutorial - Blazingly fast bar - Guide - example - tutorial # Libraries - library1 - Foo - Bar

)

Makes total sense! I think the hierarchy name is something we'd want in the breadcrumbs and so for clarity, we should also have it in the sidebar.

panglesd · 2024-10-03T11:37:40Z

All warnings have been implemented, and the code has been cleaned up.

Only remains a review for the implementation details, on my side this is ready to be merged.

Page don't and won't use dot references as in `page1.page2`.

Use ``` children: page1 page2 page3 ``` in the frontmatter

Children order specifies order in an index page for the current directory. It has to specify what is a page and what is a directory, as we can have: ``` doc/ index foo foo/ index bar ``` In the specification of the order in `doc/index`, we must make the difference between the foo page and the foo directory. This was not practical with references. So instead, the order specification has its own type.

Allow sequences of spaces in a row: `children: page1 page2`.

Unexpose unnecessary functions open modules use mli for model/frontmatter

Co-authored-by: Jules Aguillon <[email protected]>

Directly take children order into account. Co-authored-by: Jules Aguillon <[email protected]>

Co-authored-by: Jules Aguillon <[email protected]>

But alias it to Id. It was bringing too many values in scope (eg shadowing compare).

@jonludlam

CHANGES: ### Highlight - Hierarchical documentation (@jonludlam, @panglesd, @Julow) Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot) The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow) A new search bar that supports full-text and type-based search. ### Added - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--suppress-warnings` to the CLI to remove warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) ### Changed - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) ### Fixed - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277)

@jonludlam

CHANGES: - Hierarchical documentation (@jonludlam, @panglesd, @Julow) Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot) The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow) A new search bar that supports full-text and type-based search. - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--suppress-warnings` to the CLI to remove warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277)

panglesd force-pushed the page-order-new-take branch from 297519c to 531e8d4 Compare August 30, 2024 07:18

Julow reviewed Sep 5, 2024

View reviewed changes

panglesd and others added 22 commits October 3, 2024 15:24

Remove impossible page reference from the possible values

704b7d1

Page don't and won't use dot references as in `page1.page2`.

Frontmatter: Parse children order

bb05e60

Link frontmatter by resolving page references

14daab2

Driver: log compile-index in their own list

6c78a08

Allow specification of children order in index page

b1dcac8

Use ``` children: page1 page2 page3 ``` in the frontmatter

Children order: Add some tests

0812abe

Children order: more lax syntax

4ff50eb

Allow sequences of spaces in a row: `children: page1 page2`.

Children order: More improvements to tests

c798509

Children order: compatibility

208ee41

Children order: added changelog entry

c10fd76

Children order review: reduce diff with style improvements

3ab9782

Unexpose unnecessary functions open modules use mli for model/frontmatter

Children order: Addressing review comments

c405166

Add a warning when specifying children order in non-index pages

fa5cf0b

Co-authored-by: Jules Aguillon <[email protected]>

Children order: Refactor sidebar types

2ebdf3b

Directly take children order into account. Co-authored-by: Jules Aguillon <[email protected]>

Children order: Add omitted children at the end of the ordering

26411f7

Co-authored-by: Jules Aguillon <[email protected]>

Children order: Add a warning when a children is omitted

2118b0d

Co-authored-by: Jules Aguillon <[email protected]>

Children order: Add warnings for duplicated and unresolved entries

eeeffce

Children order: compatibility

2a0372f

Do not open Path.Identifier

e4ee1b7

But alias it to Id. It was bringing too many values in scope (eg shadowing compare).

Compatibility again

84d22e3

Children order: Remove common root using ID not URL segments

a5db29c

panglesd force-pushed the page-order-new-take branch from 03389b1 to a5db29c Compare October 3, 2024 13:26

jonludlam merged commit d8980ff into ocaml:master Oct 4, 2024
13 checks passed

jonludlam mentioned this pull request Jan 23, 2025

[new release] odoc (5 packages) (3.0.0~beta1) ocaml/opam-repository#27309

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Specify children order in frontmatter #1193

Specify children order in frontmatter #1193

panglesd commented Aug 30, 2024 •

edited

Loading

Julow Sep 5, 2024

panglesd Sep 9, 2024

Julow Sep 5, 2024

panglesd Sep 9, 2024

Julow Sep 5, 2024

panglesd Sep 9, 2024

Julow Oct 2, 2024

panglesd commented Oct 3, 2024


		type library = { name : string; units : Paths.Identifier.RootModule.t list }

		type page_hierarchy = { hierarchy_name : string; pages : toc }

Specify children order in frontmatter #1193

Specify children order in frontmatter #1193

Conversation

panglesd commented Aug 30, 2024 • edited Loading

Julow Sep 5, 2024

Choose a reason for hiding this comment

panglesd Sep 9, 2024

Choose a reason for hiding this comment

Julow Sep 5, 2024

Choose a reason for hiding this comment

panglesd Sep 9, 2024

Choose a reason for hiding this comment

Julow Sep 5, 2024

Choose a reason for hiding this comment

panglesd Sep 9, 2024

Choose a reason for hiding this comment

Julow Oct 2, 2024

Choose a reason for hiding this comment

panglesd commented Oct 3, 2024

panglesd commented Aug 30, 2024 •

edited

Loading