[RFC] Define a ROOT module with toplevel module

[bobot]

Desired Behavior

When discussing #3825 we realized that dune can help with the absence of a ROOT module which contains the toplevel module before any open by being able to generate one. However the name to use as always been a problem for adding this feature to the compiler. Dune has the hability through the dune-project file to activate the feature or allow to change the name. Also what are toplevel modules is not exactly defined. Dune by having a more highlevel view can better define them.

Sought advantage of having a ROOT module:

• For ppx allow to be sure to access their runtime module.
• For user allow to access library hidden by local modules

So the choice of what is a toplevel module is important:

• toplevel module of wrapped libraries the library depend on: just add the main module of the libraries handled by dune and wrapped (what is in the name field of a library).
• module for which the cmi are accessible in libraries the library depend on: add a binding for all the cmi that are in the included directories.

Finally we need a way to indicate the toplevel name to ppx. ppxlib can help by giving a simple API which use the ROOT module if available otherwise use directly the module.

Example

TODO

[rgrinberg]

Will the ROOT module be written by the user or generated by dune?

If the module is user written, what is the difference between such a module and a module without any dependencies? E.g:

(library
 (name foo)
 (module_deps (root -> ()))

[rgrinberg]

The incentive to solve the issue of ppx + runtime libraries seems nice, but I don't quite see how this proposal addresses it. As far as I see, the ppx needs to ask dune for a ROOT module that:

• It can write the list of imports directly. E.g. module Ppx_runtime = Ppx_runtime to allow Root.Ppx_runtime.
• Or the list of imports would be generated automatically by dune from the libraries used.

[bobot]

The ROOT module would be generated by dune. The list of imports would be generated automatically by dune from the libraries used. The imports to generate are the toplevel modules I'm speaking about in the description of the issue. I'm editing the description to make clear that dune would generate the file.

[aalekseyev]

I don't see how this fixes the ppx runtime dependency problem.
It's equally easy for the user to write module ROOT = MY_ROOT as it is to shadow a ppx runtime dep.
In fact the former is probably easier.
And the ability to shadow things by open or include means that we can't get away with a per-project "forbidden" module name, either.

It does help with "renaming" library deps, though, so that part seems fine.

By the way, I couldn't understand the section where you say "So the choice of what is a toplevel module is important" followed by bullet-points. Maybe you could expand each bullet-point a bit?

[rgrinberg]

It's equally easy for the user to write module ROOT = MY_ROOT as it is to shadow a ppx runtime dep.
In fact the former is probably easier.
And the ability to shadow things by open or include means that we can't get away with a per-project "forbidden" module name, either.

Perhaps just having a convention that this module cannot be shadowed is good enough? We could even call it something like Root_ to make it clearer that it isn't a normal module name. This isn't very elegant, but I like better than having nothing.

[bobot]

If we want to be cautious we can write a ppx which check that we never have module Root_ =. Of course this check doesn't preserve against an include or open of a module defined outside the dune project which define the Root_ module (we can also look at the cmi but it starts to be crazy).

And at the end the feature in dune can be just an experiment for a futur feature in the compiler. For that direction we can fix the name of the root module: OCAML_ROOT_NAMESPACE or ROOT__.

Ok I'm expanding the bullet point.

[aalekseyev]

Oh, got it (I still had to re-read it, like, 3 times though).
So if we depend on a wrapped library that has:

a.cmi
a__X.cmi
a__Y.cmi

you're asking if we should only have module A = A, or to also include module A__X = A__X...

I think definitely the former. The only supported way to access wrapped libraries is through their public renaming module, so that's the only thing we should expose.

[bobot]

I agree that it is better for wrapped library. But it means we do it only for wrapped dune libraries. Or for non-dune libraries we can do the latter. Sorry for the complicated explanation.

[aalekseyev]

I'm not very familiar with non-wrapped libraries, but I expect that for them we'll want to indeed have one alias per .cmi they expose.
Basically, whatever the public interface of the library is, we should create an alias for it.

[aalekseyev]

Here's another problem with having the ROOT module: the user can open it, or include it, or module type of it, and then we lose the property that adding a useless dependency is a no-op.

Consider code like this:

module A = struct 
  let x = 8
end
open ROOT
let () = printf "%d" A.x

What the code prints may depend on whether or not you depend on a library A. The problem becomes potentially more confusing if you consider how implicit_transitive_deps can make you depend on things you've never heard about.

[bobot]

Interesting, forbidding open and include of the module should be also checked at least until no implicit_transitive_deps is the default.

[bobot]

I'm moving to postponing this issue, except if it is found urgent by some.

[rgrinberg]

I'd be interested in exploring this proposal in the short term to get an idea if #3825 is going to be necessary.

@bobot @aalekseyev I'd like to use #3825 or something similar in the short term, but I'm not against implementing this proposal or even (module_deps ..). Or do you think that #3825 would be harmless to have if this proposal was accepted later?

[bobot]

#3825 can be put behind an experimental flag, and is harmless.

The problem with this ROOT proposal for me is choosing what is the set of toplevel libraries. If a ppx use Re, should the ppx use ROOT.Re or ROOT.MYRUNTIMELIBRARY.Re (the first one will not work in implicit_transitive_deps in a user code that doesn't depend on Re if we don't consider it to be a toplevel). For the ppx case I wonder if we should not aim at more: in a perfect world the ppx generated code would be able to access cmi that the developer code could not. The handling of cmi by the compiler is not very fine grain. Adding in the compiler an extension point that shortcut the lookup could be interesting for dune and ppxlib module FOO = {% ocaml.load_module ".../foo.cmi" %}.

In conclusion the ppx case shouldn't be considered in this proposal, as I did at first, I believe we should focus on the developer case.

[rgrinberg]

I agree that we should shelve the ppx use case for now. How about I quickly turn #3825 into:

(library
 (root_module ROOT))

The machinery to do all of this is basically done in #3825. I think this is a little more elegant than my proposal.

I'll also mention that I discussed this proposal with Jeremie briefly, and he's fine with it, but strongly against introducing any reserved identifiers that aren't chosen explicitly by the user.

[bobot]

I'm ok with that.

[rgrinberg]

I'm afraid that root_module field is not sufficient on its own. The problem arises when we depend on an unwrapped library:

(library
 (libraries foo.bar) ;; foo.bar is unwrapped with modules A, B
 (root_module ROOT))

What name do we choose for foo.bar? It seems like we cannot get away from introducing individual names for the libraries. At which point, my original proposal seems simpler.

[bobot]

Can we do it only for dune wrapped library? This possible constraint arose in the conversation.

[rgrinberg]

We can, but I think that silently skipping unwrapped libraries is bad. At least the other approach can be adapted to support unwrapped libraries.

[bobot]

In case of dune unwrapped library, can we just add the modules? So in your example A, B. In the unwrap case the module A and B should not hide other top-level module.

[rgrinberg]

Yeah, I suppose that would be fine and not too surprising for the user.

@aalekseyev do you agree?

[rgrinberg]

This was implemented