[Wait for OcamlLSP] Disable promotion of `.merlin` files and work on `ocaml-merlin` server #3554

voodoos · 2020-06-15T14:38:24Z

This PR should not be merged before ~~the release of Merlin 3.4.0 and~~ an appropriate update of ocaml-lsp-server.

What's in this PR

Dune will not promote .merlin files anymore, it is incompatible with Merlin < 3.4.0
There is now one merlin file per stanza generated in a .merlin-conf/ directory
The configuration is now directly mlarshalled in the melin config files
Update the test-suite accordingly

voodoos · 2020-07-10T11:03:37Z

The new CSEXP format of .merlin-conf files need some ad-hoc sanitization in the test. However because this format might change again in a near future it might not be worth it to tidy this process.

ghost · 2020-07-13T16:14:09Z

I guess so, though the current tests seems fine. Perhaps we could add a tool to the csexp repo to manipulate canonical s-expressions files. For instance to convert a csexp file into a human readable s-expression.

src/dune/merlin.ml

Signed-off-by: Ulysse Gérard <[email protected]>

This reverts commit b332ce9. Signed-off-by: Rudi Grinberg <[email protected]>

necessary for handling warnings Signed-off-by: Rudi Grinberg <[email protected]>

Signed-off-by: Ulysse Gérard <[email protected]>

…compatibility Signed-off-by: Ulysse Gérard <[email protected]>

Signed-off-by: Ulysse Gérard <[email protected]>

ghost · 2020-11-30T17:15:56Z

🎉

yannl35133 · 2020-12-04T23:05:12Z

Did you merge this to master before ocaml-lsp['s master] was patched to account for the change?

I have pinned both ocaml-lsp (I did when it was necessary, I didn't remove it) and dune (the absence of newline at the end of builds is annoying, why is this not patched into a release yet?).
So dune definitely doesn't promote .merlin files anymore, and ocaml-lsp tells us to do dune build to fix this, so at least this message isn't up-to-date (it also doesn't register types defined in other files, so I think it just doesn't work).

What do you advise me to do to fix this? Should I file an issue against ocaml-lsp?

rgrinberg · 2020-12-04T23:45:06Z

Did you merge this to master before ocaml-lsp['s master] was patched to account for the change?

I've verified this PR for lsp 1.3.0.

the absence of newline at the end of builds is annoying, why is this not patched into a release yet?

I wasn't aware of the pressing need. Moreover, I was quite busy with other tasks, and I did not feel obligated to report to randoms on github about it. Shall we switch to a less entitled tone to deal with this issue?

What do you advise me to do to fix this? Should I file an issue against ocaml-lsp?

I'd suggest to describe your environment in as much detail as possible. Starting with the commit hashes of dune and ocamllsp that you're using.

yannl35133 · 2020-12-05T00:12:54Z

I wasn't aware of the pressing need. Moreover, I was quite busy with other tasks, and I did not feel obligated to report to randoms on github about it. Shall we switch to a less entitled tone to deal with this issue?

Really sorry about the construction, I meant that as an honest question : I must have vastly underestimated the amount of work needed for a release and overestimated the number of people working on this. Sorry again.

I'd suggest to describe your environment in as much detail as possible. Starting with the commit hashes of dune and ocamllsp that you're using.

Thanks for your help. If I'm not mistaken about pinning in opam, they are at the latest versions in master (which is why I don't understand why it doesn't work for me)

$ dune --version
2.7.0-370-g5a2e49be6
$ ocamllsp --version
1.3.0-38-g23546b370

rgrinberg · 2020-12-05T00:16:03Z

Do you see a process like this running:

  501 71166 71165   0  4:01pm ??         0:00.36 dune ocaml-merlin --no-print-directory

It should be launched ocamllsp to get the configuration from dune.

yannl35133 · 2020-12-05T00:42:52Z

I do, and now I understand the problem.
Because dune has so many dependencies, I pinned it in another switch so that I don't recompile everything at every update.
So I use this dune through an alias, which does not promote the .merlin, and then ocaml-lsp calls the v2.7.1 dune which does not handle the new behaviour.

Now on how to fix this.
I guess I can do a build with the v2.7.1 dune and go on with the pinned dune afterwards.
Do you see a better alternative? Do you have a [very] rough estimate on when this will be released (so that I go back on the released version of dune)?

rgrinberg · 2020-12-05T01:04:04Z

I plan on making a release next week

yannl35133 · 2020-12-05T01:06:42Z

Ok, thank you very much for your help and also your work.

@mrmr1993

…ne-action-plugin, dune-private-libs and dune-glob (2.8.0) CHANGES: - `dune rules` accepts aliases and other non-path rules (ocaml/dune#4063, @mrmr1993) - Action `(diff reference test_result)` now accept `reference` to be absent and in that case consider that the reference is empty. Then running `dune promote` will create the reference file. (ocaml/dune#3795, @bobot) - Ignore special files (BLK, CHR, FIFO, SOCKET), (ocaml/dune#3570, fixes ocaml/dune#3124, ocaml/dune#3546, @ejgallego) - Experimental: Simplify loading of additional files (data or code) at runtime in programs by introducing specific installation sites. In particular it allow to define plugins to be installed in these sites. (ocaml/dune#3104, ocaml/dune#3794, fixes ocaml/dune#1185, @bobot) - Move all temporary files created by dune to run actions to a single directory and make sure that actions executed by dune also use this directory by setting `TMPDIR` (or `TEMP` on Windows). (ocaml/dune#3691, fixes ocaml/dune#3422, @rgrinberg) - Fix bootstrap script with custom configuration. (ocaml/dune#3757, fixes ocaml/dune#3774, @marsam) - Add the `executable` field to `inline_tests` to customize the compilation flags of the test runner executable (ocaml/dune#3747, fixes ocaml/dune#3679, @lubegasimon) - Add `(enabled_if ...)` to `(copy_files ...)` (ocaml/dune#3756, @nojb) - Make sure Dune cleans up the status line before exiting (ocaml/dune#3767, fixes ocaml/dune#3737, @alan-j-hu) - Add `{gitlab,bitbucket}` as options for defining project sources with `source` stanza `(source (<host> user/repo))` in the `dune-project` file. (ocaml/dune#3813, @rgrinberg) - Fix generation of `META` and `dune-package` files when some targets (byte, native, dynlink) are disabled. Previously, dune would generate all archives for regardless of settings. (ocaml/dune#3829, ocaml/dune#4041, @rgrinberg) - Do not run ocamldep to for single module executables & libraries. The dependency graph for such artifacts is trivial (ocaml/dune#3847, @rgrinberg) - Fix cram tests inside vendored directories not being interpreted correctly. (ocaml/dune#3860, fixes ocaml/dune#3843, @rgrinberg) - Add `package` field to private libraries. This allows such libraries to be installed and to be usable by other public libraries in the same project (ocaml/dune#3655, fixes ocaml/dune#1017, @rgrinberg) - Fix the `%{make}` variable on Windows by only checking for a `gmake` binary on UNIX-like systems as a unrelated `gmake` binary might exist on Windows. (ocaml/dune#3853, @kit-ty-kate) - Fix `$ dune install` modifying the build directory. This made the build directory unusable when `$ sudo dune install` modified permissions. (fix ocaml/dune#3857, @rgrinberg) - Fix handling of aliases given on the command line (using the `@` and `@@` syntax) so as to correctly handle relative paths. (ocaml/dune#3874, fixes ocaml/dune#3850, @nojb) - Allow link time code generation to be used in preprocessing executable. This makes it possible to use the build info module inside the preprocessor. (ocaml/dune#3848, fix ocaml/dune#3848, @rgrinberg) - Correctly call `git ls-tree` so unicode files are not quoted, this fixes problems with `dune subst` in the presence of unicode files. Fixes ocaml/dune#3219 (ocaml/dune#3879, @ejgallego) - `dune subst` now accepts common command-line arguments such as `--debug-backtraces` (ocaml/dune#3878, @ejgallego) - `dune describe` now also includes information about executables in addition to that of libraries. (ocaml/dune#3892, ocaml/dune#3895, @nojb) - instrumentation backends can now receive arguments via `(instrumentation (backend <name> <args>))`. (ocaml/dune#3906, ocaml/dune#3932, @nojb) - Tweak auto-formatting of `dune` files to improve readability. (ocaml/dune#3928, @nojb) - Add a switch argument to opam when context is not default. (ocaml/dune#3951, @tmattio) - Avoid pager when running `$ git diff` (ocaml/dune#3912, @AltGr) - Add `(root_module ..)` field to libraries & executables. This makes it possible to use library dependencies shadowed by local modules (ocaml/dune#3825, @rgrinberg) - Allow `(formatting ...)` field in `(env ...)` stanza to set per-directory formatting specification. (ocaml/dune#3942, @nojb) - [coq] In `coq.theory`, `:standard` for the `flags` field now uses the flags set in `env` profile flags (ocaml/dune#3931 , @ejgallego @rgrinberg) - [coq] Add `-q` flag to `:standard` `coqc` flags , fixes ocaml/dune#3924, (ocaml/dune#3931 , @ejgallego) - Add support for Coq's native compute compilation mode (@ejgallego, ocaml/dune#3210) - Add a `SUFFIX` directive in `.merlin` files for each dialect with no preprocessing, to let merlin know of additional file extensions (ocaml/dune#3977, @vouillon) - Stop promoting `.merlin` files. Write per-stanza Merlin configurations in binary form. Add a new subcommand `dune ocaml-merlin` that Merlin can use to query the configuration files. The `allow_approximate_merlin` option is now useless and deprecated. Dune now conflicts with `merlin < 3.4.0` and `ocaml-lsp-server < 1.3.0` (ocaml/dune#3554, @voodoos) - Configurator: fix a bug introduced in 2.6.0 where the configurator V1 API doesn't work at all when used outside of dune. (ocaml/dune#4046, @aalekseyev) - Fix `libexec` and `libexec-private` variables. In cross-compilation settings, they now point to the file in the host context. (ocaml/dune#4058, fixes ocaml/dune#4057, @TheLortex) - When running `$ dune subst`, use project metadata as a fallback when package metadata is missing. We also generate a warning when `(name ..)` is missing in `dune-project` files to avoid failures in production builds. - Remove support for passing `-nodynlink` for executables. It was bypassed in most cases and not correct in other cases in particular on arm32. (ocaml/dune#4085, fixes ocaml/dune#4069, fixes ocaml/dune#2527, @emillon) - Generate archive rules compatible with 4.12. Dune longer attempt to generate an archive file if it's unnecessary (ocaml/dune#3973, fixes ocaml/dune#3766, @rgrinberg) - Fix generated Merlin configurations when multiple preprocessors are defined for different modules in the same folder. (ocaml/dune#4092, fixes ocaml/dune#2596, ocaml/dune#1212 and ocaml/dune#3409, @voodoos) - Add the option `use_standard_c_and_cxx_flags` to `dune-project` that 1. disables the unconditional use of the `ocamlc_cflags` and `ocamlc_cppflags` from `ocamlc -config` in C compiler calls, these flags will be present in the `:standard` set instead; and 2. enables the detection of the C compiler family and populates the `:standard` set of flags with common default values when building CXX stubs. (ocaml/dune#3875, ocaml/dune#3802, fix ocaml/dune#3718 and ocaml/dune#3528, @voodoos)

Merlin files no longer get generated (ocaml/dune#3554); It's quite nice actually, but I think it's also valuable to keep a simple and lightweight way of extracting basic info about the project structure, such as what library a module belongs to, or what modules are open at compilation time. This usually follows from directory structures, but that's not a requirement. So, following the "simplest approximation" policy this project adheres to, this patch adds trivial parsing of `dune` files, just to infer what library a module belongs to. This in turn enables the search engine to know about the corresponding open module, and helps get proper cross-module references.

voodoos mentioned this pull request Jun 16, 2020

External config readers ocaml/merlin#1123

Merged

1 task

voodoos changed the title ~~[Wait for Merlin] Disable promotion of .merlin files~~ [Wait for Merlin] Disable promotion of .merlin files and work on ocaml-merlin server Jul 8, 2020

voodoos force-pushed the merlin-abs-paths branch 7 times, most recently from cea5c87 to c9427c4 Compare July 10, 2020 10:48

voodoos marked this pull request as ready for review July 15, 2020 13:23

voodoos requested review from aalekseyev, emillon, nojb and rgrinberg as code owners July 15, 2020 13:23

voodoos force-pushed the merlin-abs-paths branch from c9427c4 to 6bb07cd Compare July 15, 2020 13:57

ghost reviewed Jul 15, 2020

View reviewed changes

src/dune/merlin.ml Outdated Show resolved Hide resolved

voodoos mentioned this pull request Aug 7, 2020

Disable merlin config server until files stop being generated #3688

Merged

voodoos force-pushed the merlin-abs-paths branch 2 times, most recently from 51b579f to 1c4533c Compare August 10, 2020 14:27

voodoos force-pushed the merlin-abs-paths branch from 1c4533c to 1d3a530 Compare August 31, 2020 14:16

voodoos force-pushed the merlin-abs-paths branch from 1d3a530 to 3faff18 Compare September 9, 2020 15:06

voodoos force-pushed the merlin-abs-paths branch 7 times, most recently from 9748133 to 93b07d7 Compare September 24, 2020 16:26

voodoos and others added 16 commits November 30, 2020 15:58

Add test for merlin approx deprecation

7343503

Signed-off-by: Ulysse Gérard <[email protected]>

Comment + shorter sprintf

e7b50e3

Signed-off-by: Ulysse Gérard <[email protected]>

[WIP] Test vendoring causes Memo cycle

70a4172

Signed-off-by: Ulysse Gérard <[email protected]>

Revert "[WIP] Test vendoring causes Memo cycle"

61477b3

This reverts commit b332ce9. Signed-off-by: Rudi Grinberg <[email protected]>

Add dir_status when creating dune_project

5b47e1d

necessary for handling warnings Signed-off-by: Rudi Grinberg <[email protected]>

Add test for no warning in vendor

c5229a2

Signed-off-by: Ulysse Gérard <[email protected]>

Remove redundant dune-project in tests

58395f3

Signed-off-by: Ulysse Gérard <[email protected]>

Move merlin "ident" generation to Lib.Compile.make

df1994a

Signed-off-by: Ulysse Gérard <[email protected]>

Move ident gen to merlin module and use polymorphic variant for type …

d3db2bf

…compatibility Signed-off-by: Ulysse Gérard <[email protected]>

Delete useless test and improve the one for warnings

03beecb

Signed-off-by: Ulysse Gérard <[email protected]>

Move Merlin ident type to a separate module

3076689

Signed-off-by: Ulysse Gérard <[email protected]>

Refactor merlin_exists_name and mov to Merlin_ident

f148734

Signed-off-by: Ulysse Gérard <[email protected]>

Move merlin_file_path to Merlin_ident

c84e53e

Signed-off-by: Ulysse Gérard <[email protected]>

Multiple Merlin_server tweaks

ef45bba

Signed-off-by: Ulysse Gérard <[email protected]>

Add Changelog entry

1c14767

Signed-off-by: Ulysse Gérard <[email protected]>

Update tests

83550c1

Signed-off-by: Ulysse Gérard <[email protected]>

voodoos force-pushed the merlin-abs-paths branch from fa2be03 to 83550c1 Compare November 30, 2020 15:19

Edit changes

9dc5e54

Signed-off-by: Ulysse Gérard <[email protected]>

voodoos merged commit 78707ff into ocaml:master Nov 30, 2020

rgrinberg mentioned this pull request Jan 13, 2021

[new release] dune-build-info, dune, dune-configurator, dune-site, dune-action-plugin, dune-private-libs and dune-glob (2.8.0) ocaml/opam-repository#17967

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Wait for OcamlLSP] Disable promotion of `.merlin` files and work on `ocaml-merlin` server #3554

[Wait for OcamlLSP] Disable promotion of `.merlin` files and work on `ocaml-merlin` server #3554

voodoos commented Jun 15, 2020 •

edited

Loading

voodoos commented Jul 10, 2020

ghost commented Jul 13, 2020

ghost commented Nov 30, 2020

yannl35133 commented Dec 4, 2020

rgrinberg commented Dec 4, 2020

yannl35133 commented Dec 5, 2020

rgrinberg commented Dec 5, 2020

yannl35133 commented Dec 5, 2020

rgrinberg commented Dec 5, 2020

yannl35133 commented Dec 5, 2020

[Wait for OcamlLSP] Disable promotion of .merlin files and work on ocaml-merlin server #3554

[Wait for OcamlLSP] Disable promotion of .merlin files and work on ocaml-merlin server #3554

Conversation

voodoos commented Jun 15, 2020 • edited Loading

What's in this PR

voodoos commented Jul 10, 2020

ghost commented Jul 13, 2020

ghost commented Nov 30, 2020

yannl35133 commented Dec 4, 2020

rgrinberg commented Dec 4, 2020

yannl35133 commented Dec 5, 2020

rgrinberg commented Dec 5, 2020

yannl35133 commented Dec 5, 2020

rgrinberg commented Dec 5, 2020

yannl35133 commented Dec 5, 2020

[Wait for OcamlLSP] Disable promotion of `.merlin` files and work on `ocaml-merlin` server #3554

[Wait for OcamlLSP] Disable promotion of `.merlin` files and work on `ocaml-merlin` server #3554

voodoos commented Jun 15, 2020 •

edited

Loading