Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add the ability to remap identifiers during HTML generation #1189

Merged
merged 5 commits into from
Aug 27, 2024
Merged

Add the ability to remap identifiers during HTML generation #1189

merged 5 commits into from
Aug 27, 2024

Conversation

jonludlam
Copy link
Member

This is to allow the publishing of 'partial docsets' - the idea being that the driver proceeds as usual up until HTML generation, and at that point only generates HTML pages for the packages you wish to publish on your site, and remap links to other packages to ocaml.org or other site.

panglesd
panglesd reviewed Aug 22, 2024
View reviewed changes
Copy link
Collaborator

@panglesd panglesd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation looks good to me. This put a lot of pressure on the targeted website (likely ocaml.org) not changing its url format!

It would be nice to be able to test it by having the odoc_driver rewrite some links to ocaml.org.

panglesd
panglesd reviewed Aug 22, 2024
View reviewed changes
src/html/link.ml Outdated
match (relative_target, anchor) with
| [], "" -> "#"
| page, "" -> String.concat "/" page
| page, anchor -> String.concat "/" page ^ "#" ^ anchor))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to ignore: We can save one line by reusing add_anchor :)

-           | page, "" -> String.concat "/" page
-           | page, anchor -> String.concat "/" page ^ "#" ^ anchor))
+           | page, _ -> add_anchor @@ String.concat "/" page))

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

@jonludlam
Add the ability to remap identifiers during HTML generation
269bfd6 
This is to allow the publishing of 'partial docsets' - the idea being
that the driver proceeds as usual up until HTML generation, and at that
point only generates HTML pages for the packages you wish to publish on
your site, and remap links to other packages to ocaml.org or other site.
@jonludlam
Update CHANGES
5f703c8
@jonludlam
Compat
452524f
@jonludlam
Improvements following review
3380fc8 
Apply @panglesd's suggestion and fix the test as 'diff' gives a different
output on Alpine.
@jonludlam
Copy link
Member Author

Implementation looks good to me. This put a lot of pressure on the targeted website (likely ocaml.org) not changing its url format!

I think we're already in that situation. We discussed leaving redirects from the old paths to the new ones when we change to the odoc3 layout.

It would be nice to be able to test it by having the odoc_driver rewrite some links to ocaml.org.

I thought about that, but right now that'd break all the source links as we don't have that on ocaml.org yet, so I think we should hold off until we've got a good way to test it.

@jonludlam jonludlam merged commit 6910541 into ocaml:master Aug 27, 2024
13 checks passed
jonludlam added a commit to jonludlam/opam-repository that referenced this pull request Jan 23, 2025
@jonludlam
[new release] odoc (6 packages) (3.0.0~beta1)
1bbe221 
CHANGES:

### Highlight

- Hierarchical documentation (@jonludlam, @panglesd, @Julow)
  Pages can now be organized in a directory tree structure.
  Relative and absolute references are added:
  `{!./other_page.label}`, `{!//other_page}`.

- Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot)
  The documentation pages and the libraries of the entire package are shown on
  the left sidebar.

- Added support for images, videos, audio and other assets
  The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images.
  The syntax for `{video...}` and `{audio...}` is the same.
  (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185)

- Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow)
  A new search bar that supports full-text and type-based search.

### Added

- Experimental driver (@jonludlam, @panglesd)
  The driver builds the documentation for a collection of Opam packages using
  the newer Odoc features. It supports linking external packages to ocaml.org
  and markdown files.
  This is experimental and will break in the future.

- Cross-package references (@panglesd, @Julow)
  Pages and modules from other packages can be referenced:
  `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`.

- Option to remap links to other packages to ocaml.org or other site.
  See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`.
  (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248)

- Option to compute occurrences of use of each identifiers
  The commands `aggregate-occurrences` and `count-occurrences` are added.
  (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206)

- Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121)
  Helps driver detecting which modules belong to which libraries.
- Added `--suppress-warnings` to the CLI to remove warnings from a unit, even
  if they end up being raised in another unit through expansion
  (@jonludlam, ocaml/odoc#1260)
- Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089)
- Navigation for the search bar : use '/' to enter search, up and down arrows to
  select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088)
- Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147)
- Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084)
- Allow referencing of polymorphic constructors in polymorphic variant type
  aliases (@panglesd, ocaml/odoc#1115)
- Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251)
  It can be disabled with a CLI option.
- Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251)
  Allows to specify the title of a page, the order of sub-pages and other
  behaviors in the sidebar.
- Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234)

### Changed

- The command line interface changed to support the new features.
  + Packages and libraries: `odoc link` must now be aware of packages and
    libraries with the `-L libname:path` and `-P pkgname:path` options. The
    module search path should still be passed with the `-I` option.
    The current package should be specified with `--current-package=pkgname`.
  + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree
    specified with `--output-dir=DIR` and the parent identifier must be
    specified with `--parent-id=PARENT`.
    The option `--source-parent-file` is removed.
  + Source code: Implementations are compiled with `compile-impl` instead of
    with `compile`. The options `--cmt=..` and `--source-name=..` are removed.
    Source code pages are generated with `html-generate-source`.
  + Assets: The commands `compile-asset`, `html-generate-asset` are added.
    The option `html-generate --asset` is removed.
  + Sidebar: The index is built using `compile-index`. The sidebar data is
    extracted from the index with `sidebar-generate` and passed to
    `html-generate --sidebar=..`.

- The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239)
  A `@tag` can now be followed by a paragraph or other elements.

- Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023)
- Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075)
- Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085)
  A newline followed by any whitespaces is normalized as one space character.
- Reduce size of `Odoc_html_frontend` when compiled to javascript
  (@EmileTrotignon, ocaml/odoc#1072)
- Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081)
- Output file paths and labels in the man and latex backends changed to avoid
  name clashes (@Julow, ocaml/odoc#1191)

### Fixed

- Fix variant constructors being hidden if they contain hidden types
  (@jonludlam, ocaml/odoc#1105)
- Fix rare assertion failure due to optional parameters
  (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001)
- Fix resolution of module synopses in {!modules} lists that require --open
  (@jonludlam, ocaml/odoc#1104}
- Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117)
- Fixed 404 links from search results (@panglesd, ocaml/odoc#1108)
- Fixed title content not being picked up across pages when rendering references
  (ocaml/odoc#1116, @panglesd)
- Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd)
- Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133)
- Fix bug where source rendering would cause odoc to fail completely if it
  encounters invalid syntax (@jonludlam ocaml/odoc#1208)
- Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268)
- Fix syntax highlighting not working for very large files
  (@jonludlam, @Julow, ocaml/odoc#1277)
jonludlam added a commit to jonludlam/opam-repository that referenced this pull request Jan 23, 2025
@jonludlam
[new release] odoc (5 packages) (3.0.0~beta1)
0ffc6db 
CHANGES:

- Hierarchical documentation (@jonludlam, @panglesd, @Julow)
  Pages can now be organized in a directory tree structure.
  Relative and absolute references are added:
  `{!./other_page.label}`, `{!//other_page}`.

- Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot)
  The documentation pages and the libraries of the entire package are shown on
  the left sidebar.

- Added support for images, videos, audio and other assets
  The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images.
  The syntax for `{video...}` and `{audio...}` is the same.
  (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185)

- Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow)
  A new search bar that supports full-text and type-based search.

- Experimental driver (@jonludlam, @panglesd)
  The driver builds the documentation for a collection of Opam packages using
  the newer Odoc features. It supports linking external packages to ocaml.org
  and markdown files.
  This is experimental and will break in the future.

- Cross-package references (@panglesd, @Julow)
  Pages and modules from other packages can be referenced:
  `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`.

- Option to remap links to other packages to ocaml.org or other site.
  See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`.
  (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248)

- Option to compute occurrences of use of each identifiers
  The commands `aggregate-occurrences` and `count-occurrences` are added.
  (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206)

- Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121)
  Helps driver detecting which modules belong to which libraries.
- Added `--suppress-warnings` to the CLI to remove warnings from a unit, even
  if they end up being raised in another unit through expansion
  (@jonludlam, ocaml/odoc#1260)
- Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089)
- Navigation for the search bar : use '/' to enter search, up and down arrows to
  select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088)
- Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147)
- Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084)
- Allow referencing of polymorphic constructors in polymorphic variant type
  aliases (@panglesd, ocaml/odoc#1115)
- Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251)
  It can be disabled with a CLI option.
- Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251)
  Allows to specify the title of a page, the order of sub-pages and other
  behaviors in the sidebar.
- Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234)

- The command line interface changed to support the new features.
  + Packages and libraries: `odoc link` must now be aware of packages and
    libraries with the `-L libname:path` and `-P pkgname:path` options. The
    module search path should still be passed with the `-I` option.
    The current package should be specified with `--current-package=pkgname`.
  + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree
    specified with `--output-dir=DIR` and the parent identifier must be
    specified with `--parent-id=PARENT`.
    The option `--source-parent-file` is removed.
  + Source code: Implementations are compiled with `compile-impl` instead of
    with `compile`. The options `--cmt=..` and `--source-name=..` are removed.
    Source code pages are generated with `html-generate-source`.
  + Assets: The commands `compile-asset`, `html-generate-asset` are added.
    The option `html-generate --asset` is removed.
  + Sidebar: The index is built using `compile-index`. The sidebar data is
    extracted from the index with `sidebar-generate` and passed to
    `html-generate --sidebar=..`.

- The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239)
  A `@tag` can now be followed by a paragraph or other elements.

- Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023)
- Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075)
- Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085)
  A newline followed by any whitespaces is normalized as one space character.
- Reduce size of `Odoc_html_frontend` when compiled to javascript
  (@EmileTrotignon, ocaml/odoc#1072)
- Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081)
- Output file paths and labels in the man and latex backends changed to avoid
  name clashes (@Julow, ocaml/odoc#1191)

- Fix variant constructors being hidden if they contain hidden types
  (@jonludlam, ocaml/odoc#1105)
- Fix rare assertion failure due to optional parameters
  (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001)
- Fix resolution of module synopses in {!modules} lists that require --open
  (@jonludlam, ocaml/odoc#1104}
- Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117)
- Fixed 404 links from search results (@panglesd, ocaml/odoc#1108)
- Fixed title content not being picked up across pages when rendering references
  (ocaml/odoc#1116, @panglesd)
- Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd)
- Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133)
- Fix bug where source rendering would cause odoc to fail completely if it
  encounters invalid syntax (@jonludlam ocaml/odoc#1208)
- Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268)
- Fix syntax highlighting not working for very large files
  (@jonludlam, @Julow, ocaml/odoc#1277)
@jonludlam jonludlam mentioned this pull request Jan 23, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@panglesd panglesd panglesd left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jonludlam @panglesd