[merlin] Handle filenames with dot-separated sections (ocaml#4264)

* Split filename on `.` and use the first part for config matching
* Illustrate usage with some tests
* Add some documentation about naming preprocessed files.

Signed-off-by: Ulysse Gérard <[email protected]>

doc/usage.rst

@@ -280,8 +280,8 @@ There's a few aliases that dune automatically creates for the user
   defined in that directory.
 
 * ``check`` - This alias will build the minimal set of targets required for
-  tooling support. Essentially, this is ``.cmi``, ``.cmt``, ``.cmti``, and
-  ``.merlin`` files.
+  tooling support. Essentially, this is ``.cmi``, ``.cmt``, ``.cmti`` files and
+  Merlin configurations.
 
 Variables for artifacts
 -----------------------
@@ -542,3 +542,107 @@ The `--prefix` directory should be used to specify the destination.
 If you are using plugins that depends on installed libraries which are not
 dependencies of the executables -- so libraries that need to be loaded at
 runtime -- you must copy the libraries manually to the destination directory.
+
+Since version 2.8 Dune does not promote ``.merlin`` files to the source
+directories any more. Instead these configurations are stored in the `_build`
+folder and Merlin communicates directly with Dune to obtain its configuration
+via the `ocaml-merlin` subcommand. The Merlin configuration is now stanza
+specific allowing finer control. The following commands are not needed for
+normal use of Dune and Merlin but can provide insightful informations when
+debugging or configuring non-standard projects.
+
+Printing the configuration
+--------------------------
+
+It is possible to manually query the generated configuration for debugging
+purposes:
+
+::
+
+    $ dune ocaml-merlin --dump-config
+
+This command will print the distinct configuration of each module present in the
+current directory. This directory must be in a Dune workspace and the project
+must be already built. The configuration will be encoded as a s-expressions, which
+are used to communicate with Merlin.
+
+Printing an approximated ``.merlin``
+------------------------------------
+
+It is also possible to print the configuration of the current folder in the
+Merlin configuration syntax by running the following command:
+
+::
+
+    $ dune ocaml dump-dot-merlin > .merlin
+
+In that case only one configuration will be printed which is the result of a
+coarse merge of the configurations of the various modules present in the current
+folder. This folder must be in a Dune workspace and the project must be already
+built. Preprocessing directives and other flags will be commented out and must
+be un-commented afterward. This feature does not aim at writing exact or correct
+``.merlin`` files, its sole purpose is to lessen the burden of writing the
+configuration from scratch.
+
+Both these commands also support an optional path to specify the target
+directory. This directory must be in a Dune workspace and the project must have
+already been built.
+=======
+
+Querying Merlin configuration
+=============================
+
+Since version 2.8 Dune does not promote ``.merlin`` files to the source
+directories any more. Instead these configurations are stored in the `_build`
+folder and Merlin communicates directly with Dune to obtain its configuration
+via the `ocaml-merlin` subcommand. The Merlin configuration is now stanza
+specific allowing finer control. The following commands are not needed for
+normal use of Dune and Merlin but can provide insightful informations when
+debugging or configuring non-standard projects.
+
+Printing the configuration
+--------------------------
+
+It is possible to manually query the generated configuration for debugging
+purposes:
+
+::
+
+    $ dune ocaml-merlin --dump-config
+
+This command will print the distinct configuration of each module present in the
+current directory. This directory must be in a Dune workspace and the project
+must be already built. The configuration will be encoded as a s-expressions, which
+are used to communicate with Merlin.
+
+Printing an approximated ``.merlin``
+------------------------------------
+
+It is also possible to print the configuration of the current folder in the
+Merlin configuration syntax by running the following command:
+
+::
+
+    $ dune ocaml dump-dot-merlin > .merlin
+
+In that case only one configuration will be printed which is the result of a
+coarse merge of the configurations of the various modules present in the current
+folder. This folder must be in a Dune workspace and the project must be already
+built. Preprocessing directives and other flags will be commented out and must
+be un-commented afterward. This feature does not aim at writing exact or correct
+``.merlin`` files, its sole purpose is to lessen the burden of writing the
+configuration from scratch.
+
+Non-standard filenames
+----------------------
+
+Merlin configuration loading is based on filename. That means that if you have
+files which are preprocessed by custom rules before they are built, they should
+respect the following naming convention: the unprocessed file should start with
+the name of the resulting processed file followed by a dot and then the rest
+does not matter. Only the name before the first dot will be used by Dune to
+match with available configurations.
+
+For example, if you use the ``cppo`` preprocessor to generate the file
+``real_module_name.ml`` then the source file could be named
+``real_module_name.cppo.ml``.

src/dune_rules/merlin.ml

@@ -79,7 +79,14 @@ module Processed = struct
          [ stdlib_dir; exclude_query_dir; obj_dirs; src_dirs; flags; suffixes ])
 
   let get { modules; pp_config; config } ~filename =
-    let fname = Filename.remove_extension filename |> String.lowercase in
+    (* We only match the first part of the filename : foo.ml -> foo foo.cppo.ml
+       -> foo *)
+    let fname =
+      String.lsplit2 filename ~on:'.'
+      |> Option.map ~f:fst
+      |> Option.value ~default:filename
+      |> String.lowercase
+    in
     List.find_opt modules ~f:(fun name ->
         let fname' = Module_name.to_string name |> String.lowercase in
         String.equal fname fname')

test/blackbox-tests/test-cases/merlin/server.t/dune

@@ -11,3 +11,8 @@
  (name main)
  (modules main lib2)
  (libraries mylib mylib3))
+
+(executable
+ (name not-a-module-name)
+ (modules not-a-module-name)
+ (flags :standard -w -24))

test/blackbox-tests/test-cases/merlin/server.t/not-a-module-name.ml

@@ -0,0 +1 @@
+print_endline "I do not bear a valid module name."

test/blackbox-tests/test-cases/merlin/server.t/run.t

@@ -16,3 +16,28 @@
   > (4:File${#FILE}:$FILE)
   > EOF
   ((?:STDLIB?:OPAM_PREFIX/lib/ocaml)(?:EXCLUDE_QUERY_DIR)(?:B?:$TESTCASE_ROOT/_build/default/.mylib.objs/byte)(?:B?:$TESTCASE_ROOT/_build/default/.mylib3.objs/byte)(?:S?:$TESTCASE_ROOT)(?:FLG(?:-open?:Mylib?:-w?:@1..3@5..28@30..39@43@46..47@49..57@61..62-?:-strict-sequence?:-strict-formats?:-short-paths?:-keep-locs)))
+
+If a file has a name of the kind `module_name.xx.xxx.ml/i`
+we consider it as ``module_name.ml/i`
+This can be useful when some build scripts perform custom
+preprocessing and copy files around.
+  $ FILE=lib3.foobar.ml
+  $ dune ocaml-merlin <<EOF | sed -E "s/[[:digit:]]+:/\?:/g" | sed 's#'$(opam config var prefix)'#OPAM_PREFIX#'
+  > (4:File${#FILE}:$FILE)
+  > EOF
+  ((?:STDLIB?:OPAM_PREFIX/lib/ocaml)(?:EXCLUDE_QUERY_DIR)(?:B?:$TESTCASE_ROOT/_build/default/.mylib.objs/byte)(?:B?:$TESTCASE_ROOT/_build/default/.mylib3.objs/byte)(?:S?:$TESTCASE_ROOT)(?:FLG(?:-open?:Mylib?:-w?:@1..3@5..28@30..39@43@46..47@49..57@61..62-?:-strict-sequence?:-strict-formats?:-short-paths?:-keep-locs)))
+
+If a directory has no configuration the configuration of its parent is used
+This can be useful when some build scripts copy files from subdirectories.
+  $ FILE=some_sub_dir/lib3.foobar.ml
+  $ dune ocaml-merlin <<EOF | sed -E "s/[[:digit:]]+:/\?:/g" | sed 's#'$(opam config var prefix)'#OPAM_PREFIX#'
+  > (4:File${#FILE}:$FILE)
+  > EOF
+  ((?:STDLIB?:OPAM_PREFIX/lib/ocaml)(?:EXCLUDE_QUERY_DIR)(?:B?:$TESTCASE_ROOT/_build/default/.mylib.objs/byte)(?:B?:$TESTCASE_ROOT/_build/default/.mylib3.objs/byte)(?:S?:$TESTCASE_ROOT)(?:FLG(?:-open?:Mylib?:-w?:@1..3@5..28@30..39@43@46..47@49..57@61..62-?:-strict-sequence?:-strict-formats?:-short-paths?:-keep-locs)))
+
+Test of an valid invalid module name
+  $ FILE=not-a-module-name.ml
+  $ dune ocaml-merlin <<EOF | sed -E "s/[[:digit:]]+:/\?:/g" | sed 's#'$(opam config var prefix)'#OPAM_PREFIX#'
+  > (4:File${#FILE}:$FILE)
+  > EOF
+  ((?:STDLIB?:OPAM_PREFIX/lib/ocaml)(?:EXCLUDE_QUERY_DIR)(?:B?:$TESTCASE_ROOT/_build/default/.not-a-module-name.eobjs/byte)(?:S?:$TESTCASE_ROOT)(?:FLG(?:-w?:@1..3@5..28@30..39@43@46..47@49..57@61..62-?:-strict-sequence?:-strict-formats?:-short-paths?:-keep-locs?:-w?:-24)))