Sidebar and index overhaul

[panglesd]

This PR includes multiple modifications. It is best reviewed commit by commit (there is only one commit).

Trees

Odoc used to have several representations of trees: one for the page sidebar in the model, one for the document sidebar, and (in a squashed commit) one for the unit sidebar.

All trees now have the same type, making the different passes (eg model -> document for pages and units) much easier, at a small cost (the type is less tailored to the usecase, eg the payload cannot be different in leafs than in node, which was the case before in the page hierarchy).

Trees (and forests) have basic iterators defined.

The index for units

The index for the units values used to be a hashtable from ID to entry. The problem was that you cannot rebuild a sidebar from that: you lose the order between children.

The index for units now is a tree of index entries.

The sidebar for units

The sidebar for units finally shows more than just the root module.

However, it does not show the full hierarchy either, as that would be overwhelming in the case of big modules.

The sidebar (as suggested by @Julow, thanks!) shows:

• Only entries that could have had an expansion: modules, modules types, classes and class types.
• The current page (highlighted),
• The children of the current page, (highlighted differently),
• The ancestors of the current page,
• The children of the ancestors of the current page,
• Nothing else.

If you allow me, I like to use the github syntax for mathematics 😄. The sidebar has the property that it displays the smallest set $S$ that:

• Contains only modules, modules types, classes and class types.
• Contains the current page: $current_page\in S$,
• Is ancestor-closed: if $e\in S$ then $parent(e)\in S$,
• Is sibling-closed: if $e\in S$ and $parent(e)=parent(f)$, then $f\in S$

The last property is important to avoid displaying only part of the children of a parent, requiring to display some ... to show that some entries were omitted.

Organization in directories and libraries

The search/ folder and its associated odoc_search library was separated in two: the original one and the new index/ and odoc_index which contains everything that an index should contain: both the info for the sidebar and for the search index.

[panglesd]

One can have a quick look to how the sidebar feels here.

[dbuenzli]

Not sure I fully grasped the your description of the side bar. But bear in mind that predictability is paramount, so I'd avoid changing it too much depending on the context (except for the local content part of course). In fact I'm not sure that detailing contents in the GLOBAL CONTENT is a good idea. But first a few comments regardless of that idea.

Having a look at the GLOBAL CONTENT on this page:

1. You are wasting a lot of precious horizontal space by adding Package's Page and Libraries and then indenting underneath. I don't think those are needed. Also the lists should be properly classified.
2. I find the XXX's Pages/Units hard to parse and ugly. I don't think writing Units is natural, I rather have Library here: that's the name and concept I will have to use in my build system. I don't need to know that something is a page, if it's written User manual and it's clickable I expect it to be a page.
3. In general I think I'm going to peruse library module indexes more than doc pages so I'd like the library outlines first.
4. I'm don't think that detailing a top-level's page children is going to scale in practice. I think the page list in GLOBAL CONTENT should simply be the name of the toplevel pages.

TL;DR I think the outline of GLOBAL CONTENT should be simplified to:

Library `ppxlib`
|…

Library `ppxlib.ast`
|…

User manual
Other page
…

Rendered as:

<nav class = "odoc-toc odoc-global-toc">
  <ul class="odoc-modules">…</ul>
  <ul class="odoc-pages>…</ul>

Now I don't think that the GLOBAL CONTENT part should ever be different on the page of a package. It should always simply be:

1. Libraries and their toplevel modules (compilation units)
2. Toplevel pages

So that I can navigate it extremely predictably. Especially its geometry should not change depending on the page I'm perusing.

Now for detailing the content of a unit it would perhaps (needs testing) be preferable to have a new UNIT INDEX nav separately.

So basically you have there <nav>'s the global one, the local one with the topical heading outline and the one that lists the structure items by kind and in alphabetical order, maybe under a details element.

[dbuenzli]

1. Libraries and their toplevel modules (compilation units)

Also I think it would be nice to have an @namespace (or a better name) directive that you can use to indicate that the unit is mainly a namespacing module and that it's toplevel names should be spliced into the library index. For example on applying it Gg you'd get

Library Gg
| Gg
| Gg.Float
| Gg.P2
| Gg.P3 
…

Rather than:

Library Gg
| Gg

[panglesd]

Thanks for the comments!

Not sure I fully grasped the your description of the side bar.

I agree my description is unnecessarily convoluted. In short, from a page, the sidebar shows the direct children, the ancestors, and the direct children of the ancestors.

I don't think that the GLOBAL CONTENT part should ever be different on the page of a package. [...] So that I can navigate it extremely predictably. Especially its geometry should not change depending on the page I'm perusing.

That is one valid use of a sidebar. Other people like to use it to identify more easily where they are in the global hierarchy of modules, or to navigate with less clicks. In particular, the ocaml.org sidebar is popular to some people (and unpopular for others).

It is hard to decide exactly what should go in the sidebar, because of the different valid ways it can be used for. I personally think that this is a good middleground, but there is no absolute argument to validate that.

You are wasting a lot of precious horizontal space

I find the XXX's Pages/Units hard to parse and ugly.

It's true that I have not (yet) put a lot of efforts in how the sidebar is displayed, more in what it contains. I'll try to correct that now, taking your many suggestions into account.

I'd like the library outlines first

I'm not sure which order is the best, since I think most of the times the pages will take much less space and will be the first thing you want to read when you discover a library. However, I'm happy to put the libraries first to try and see how it feels.

I don't think that detailing a top-level's page children is going to scale in practice. I think the page list in GLOBAL CONTENT should simply be the name of the toplevel pages.

Yes, you are right that with many pages organized in a hierarchy (as in dune or ocaml's docs for instance), that is not going to scale. However, I think it is better to show the uncle pages. For instance, this readthedocs page show the uncle pages, and to me that is a nicer user experience than only seeing the toplevel pages.

In other words, I would like for pages to show the same things as in the doc API case: The direct children, the ancestors, and the direct children of the ancestors.

But bear in mind that predictability is paramount, so I'd avoid changing it too much depending on the context (except for the local content part of course).

I believe the sidebar is predictable enough, even though it still depending on the page. When I use it, I am not surprised something is shown/hidden. But since I know how it works, I'm not the best person to be surprised. Did something felt unpredictable when you used it?

So basically you have three <nav>'s the global one, the local one with the topical heading outline and the one that lists the structure items by kind and in alphabetical order, maybe under a details element.

If I were to add an index of structure items, I would rather add it to the "topical headings" nav, in the order given in the .mli. In GG, this would be:

Heading Floats
module Float
Heading Vectors
type m2
type m3
[...]

I would not know where to put a third nav, and I think the mli has been written in some order for a good reason. If the goal is only to jump to some value/type you know the name, a "search" in the browser is as efficient.

However, this is just my personal, untested, opinion. It might be overwhelming in some cases, and would require testings!

Also I think it would be nice to have an @namespace (or a better name) directive

That could be a good idea. Maybe @toc-expand which makes the pages's children expanded in the toc.

[dbuenzli]

That is one valid use of a sidebar. Other people like to use it to identify more easily where they are in the global hierarchy of modules, or to navigate with less clicks. In particular, the ocaml.org sidebar is popular to some people (and unpopular for others).

Well you have the breadcrumbs for that. The problem is that you can't have both. If you try to show where you are too precisely in that navigation widget then it becomes unusable for navigation (see below for explanations).

Also modules and document hierarchies should most of the time be rather shallow (at most 2-3 levels after the namespace) don't try to cater for arbitrary deep ones. The deeper you are the more anecdotic the module is likely to be and the less important it is to provide direct efficient access or the need to contextualize it in the module hierarchy; and again if you need to know where you are, look at the breadcrumbs.

For library module indexes I think you should rather try to simply follow what has bee done in the "LOCAL CONTENT" part which works extremely well now; IIRC it consistently shows at most three levels. So I would simply consistently show up to two levels of module/functor implementations (or three if @toc-expand), fully prefixed except perhaps for the namespace (because that's how you are likely to read them in the code and it makes thing less indented which makes the page layout less busy for the eyes).

However, I think it is better to show the uncle pages. For instance, this readthedocs page show the uncle pages, and to me that is a nicer user experience than only seeing the toplevel pages.

Again you have the breadcrumbs. I think if you really want this you should rather try to put them in a closed details element or for a given manual in a separate navigation element. Also bear in mind that readthedocs basically shows one linear manual so don't necessary try to copy-cat their design. You are designing something for efficiently navigating among multiple manuals and APIs.

I'm not sure which order is the best, since I think most of the times the pages will take much less space and will be the first thing you want to read when you discover a library.

I'm not sure either (i.e. I'd need to test it to come to an answer) but this is not a good argument: do not optimize for newcomers, optimize for the proficient library user. Using this as a guide I would tend to say that you want the library indexes first (in the future manuals may become more plentiful, e.g. there are at least 10 planned here). The newcomer will likely go through the landing page where a guide to the available manuals should be provided.

I believe the sidebar is predictable enough, even though it still depending on the page. When I use it, I am not surprised something is shown/hidden. But since I know how it works, I'm not the best person to be surprised. Did something felt unpredictable when you used it?

To me it is totally unusable. Since its content is constantly changing you cannot use it for navigation. When you click on something, as it expands on the destination page, the next item that was below your cursor when I clicked is no longer there, I'm lost and I may have to skip an arbitrary amount of data in order to find it again. And when I'm on an arbitrary page the fact that there may be a huge expansion doesn't allow me to quickly move among the toplevel entries and prevents me from building any muscle memory ("I know this item is just after that one").

In general on a website if, when you move from one page to another, the navigation is constantly jumping around, you don't have a very good website :–) I would really suggest you try what I mentioned above. That is for each library consistent list of up to 2 level of module implementations, list of toplevel manuals (with possibly their children in a details).

If I were to add an index of structure items, I would rather add it to the "topical headings" nav, in the order given in the .mli. In GG, this would be:

No don't do that. It becomes totally redundant with the page and is going to be unusable. The topical headings should provide a quick scannable outline that allows to jump you to a part of the page¹ without having to scroll when you land on the page. It's perfectly fine as it is now, don't touch it!

Personally I was never convinced that page-level indexes of structure items are that useful (but I'm still adamant we need global ones, at least at the package level, see #577). In any case if you'd add one then I think it should be ordered alphabetically and in a closed detail.

Footnotes

1. In fact if browsers were used for what they were supposed to be we wouldn't need it. Browsers would have UIs for interacting with the results of the HTML outline algorithm. ↩

[panglesd]

I agree that having parts moving when you click on something is not ideal. I'd hope that highlighting well the current item would alleviate the issue. For me, it is working: I don't feel I lose the ability to navigate quickly nor feel lost due to the sidebar having moving parts, in the current implementation.
For you, I understand that it is unusable. As a comparison point, does the moving parts in the readthedocs sidebar also makes it unusable for you?

I really feel the usability of the sidebar depend on the people (some people have spatial memory, some people remember the name, some people do well with high density of info, some prefer more minimalist UI, ...) and it is hard to give arguments that something is objectively better.

(In some sense, we will always have moving parts, just more or less often: if you had to scroll on the sidebar to click on an item, the sidebar's will be back on top on the next page.)

So I would simply consistently show up to two levels of module/functor implementations

I can experiment with that. I'll "publish" several libraries docs using each version of the sidebar, so we can compare.
I'm afraid that depending on the libraries, it will be too many items (eg any "standard lib replacement" library, big libraries such as core). In this case, I think the depth of the sidebar should be chosen by the library author.

[About local content displaying structure items] No don't do that.

Don't worry, I'm not doing it :)

[About displaying the uncles] Again you have the breadcrumbs.

Sometimes, having the uncles helps orienting you. In the "dune docs" example above, the breadcrumbs would only tell me I'm under "Reference Manual". However, to me it is easier with all the examples of things that fall into the "Reference manual": dune, dune-project, config, ...
Using a more appropriate example, if I'm in the module page for Bos.OS.Path the breadcrumbs tell me that I'm below the OS module. However, having the children of OS displayed makes it much clearer what exactly is the OS module about.

Back to the "readthedocs" example:

You are designing something for efficiently navigating among multiple manuals and APIs.

I'm not sure to understand. Each package gets its own sidebar, so a sidebar only show one manual, the one for the package? In a global sidebar, it seems to me that everything ends up linearized in the end, if only because things have to be shown from top to bottom.
I would think that the two usecases (odoc and readthedocs) are pretty similar. But maybe I'm missing something!
(And of course the goal is not to copy-cat the rtd design, but taking inspiration on the things that work well seem a good idea!)

[dbuenzli]

For you, I understand that it is unusable. As a comparison point, does the moving parts in the readthedocs sidebar also makes it unusable for you?

Well yes it's also totally unpredictable. I don't think readthedocs is a good design.

and it is hard to give arguments that something is objectively better.

I don't think so. For users having predictable and consistent structures for navigation is objectively better than something that is constantly changing and spatially twitching. Also bear in mind that if you are looking that on a 23'' screen the experience may be quite different on a laptop screen whose vertical size is rather short.

I'm afraid that depending on the libraries, it will be too many items (eg any "standard lib replacement" library, big libraries such as core).

I'm not so sure. For example if you take the menu on ocaml.org of core. There is actually little to be gained in hiding the modules of Core (the libraries that come after seem to be of less interest and have only one or two modules, you could also perhaps reorder them). I suspect that a core user when it comes to this page for looking up the docs of a module always ends up wasting a click to open the Core toplevel module, boy I would hate that :–) (not to mention the horribly wasted vertical space on the top).

Using a more appropriate example, if I'm in the module page for Bos.OS.Path the breadcrumbs tell me that I'm below the OS module. However, having the children of OS displayed makes it much clearer what exactly is the OS module about.

But you could also simply go in the OS module and peruse the page (or an index of modules on this page) where you will get a nice topical index. I don't think it's necessary to understand these aspects from the navigation part, they can be learned in other manners.

I think the problem of your approach is that you are trying to show everything. But having a good an efficient navigation structure is about abstracting the details, be mindful of showing the essentials that allow you to efficiently get to any other part from where you are in quick (count the clicks) and predictable (fixed structures) manners, without overwhelming users with choices and all the (sub)details of what exists (screen estate is often very limited and scrolling is annoying).

Now for a given manual with child pages you may want to have an outline with all the sibling pages but as already mentioned it doesn't have to be in the global content part and/or can be hidden in a details by default.

I'm not sure to understand. Each package gets its own sidebar, so a sidebar only show one manual, the one for the package?

I think the model should be this, in a package:

1. There is a single global sidebar.
2. There is a single package landing page index.mld.
3. There are multiple manuals. A manual being one .mld page, possibly with sub .mld pages.
4. There are multiple libraries each defining an API. An API being a set of modules, possibly with sub*-modules.

And I think the global side bars should show the following fixed structure :

Library l0
| M0
| …

Library l1
| M1
| …

Manual 0
Manual 1

With submodules shown in case of a namespacing module and in manuals possibly having the children in a details.

[panglesd]

Finish the sentence

[panglesd]

Remove

[panglesd]

Remove this file!

[panglesd]

Remove this file

[panglesd]

Thanks for your ideas.

We discussed a bit the review of this PR. It was decided to split it in smaller chunks, review and merge them: anyway, the final design will only slightly alter the code.

Once the feature is in, we will revisit the "how to prune the sidebar" question, experimenting with the various ideas. One option I would find sensible would be to give options to the driver to decide how the sidebar is pruned.

[jonludlam]

Superceded by #1244