Failed to parse file, not adding version and locations information #4389

toots · 2021-03-23T15:44:06Z

Sorry for the avalanche of bug reports y'all, I hope that they are useful..

This one seems like a bug, at least it looks like I'm doing everything as documented.

I'm using the new sites feature to install our local library files:

dune-project:

(lang dune 2.8)
(using menhir 2.1)
(using dune_site 0.1)

(package
  (name liquidsoap)
  (version 2.0.0)
  (sites (share libs)))

libs/dune:

(install
 (section
  (site
   (liquidsoap libs)))
 (files
  audio.liq
  deprecations.liq
...))

When installing, I see this error:

Installing /Users/toots/.opam/4.10.2/lib/liquidsoap/dune-package
File "_build/install/default/lib/liquidsoap/dune-package", line 1, characters 0-0:
Warning: Failed to parse file, not adding version and locations information.

The content of dune-package is automatically generated seems okay:

dune-package:

(lang dune 2.8)
(name liquidsoap)
(version 2.0.0)
(sections
 (share
  /Users/toots/sources/savonet/liquidsoap/_build/install/default/share/liquidsoap))
(sites (libs share))

The installed binary segfaults, I believe because it cannot find the location information at runtime.

The text was updated successfully, but these errors were encountered:

bobot · 2021-03-23T16:24:11Z

I'm really thankful for all the testing this experimental feature get. I'm just sorry to not have time before April to fix them.

The segfaulting is especially bad, normally the worst case is an exception. Could you run gdb on the resulting binary in order to perhaps pinpoint where it segfault?

ghost · 2021-03-23T17:23:26Z

Sorry for the avalanche of bug reports y'all, I hope that they are useful..

It's definitely welcome :) It's better if we know about bugs than if we don't.

toots · 2021-03-24T00:24:51Z

I'm really thankful for all the testing this experimental feature get. I'm just sorry to not have time before April to fix them.

The segfaulting is especially bad, normally the worst case is an exception. Could you run gdb on the resulting binary in order to perhaps pinpoint where it segfault?

I can definitely confirm that:

The executable has the same md5 when stashed in _build/install/default/bin and when installed
It works when called via dune exec
It segfaults when ran from the target folder after doing dune install
It does not segfault when complied with the dune-site support.

Unfortunately, the segfault seems a little light:

* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGTTOU
  * frame #0: 0x0000000192507510 libsystem_kernel.dylib`__wait4_nocancel + 8
    frame #1: 0x000000019247967c libsystem_c.dylib`system + 428
    frame #2: 0x00000001005a14f4 liquidsoap`caml_sys_system_command(command=<unavailable>) at sys.c:444:12 [opt]
    frame #3: 0x00000001005aeaf8 liquidsoap`caml_c_call + 28
    frame #4: 0x000000010020b778 liquidsoap`camlUtils__print_strings_inner_2681 + 656
    frame #5: 0x000000010001208c liquidsoap`camlDune__exe__Main__parse_1156 + 204
    frame #6: 0x00000001000122f0 liquidsoap`camlDune__exe__Main__fun_1984 + 64
    frame #7: 0x00000001004ab3d0 liquidsoap`camlDtools__Dtools_impl__exec_768 + 232
    frame #8: 0x00000001004de28c liquidsoap`camlStdlib__list__iter_258 + 44
    frame #9: 0x00000001004ab414 liquidsoap`camlDtools__Dtools_impl__exec_768 + 300
    frame #10: 0x000000010000f908 liquidsoap`camlDune__exe__Liquidsoap__code_begin + 32
    frame #11: 0x000000010000888c liquidsoap`caml_program + 15660
    frame #12: 0x00000001005aeb70 liquidsoap`caml_start_program + 108
    frame #13: 0x000000010058b6e4 liquidsoap`caml_startup_common(argv=0x000000010020b778, pooling=<unavailable>) at startup_nat.c:162:10 [opt]
    frame #14: 0x000000010058b74c liquidsoap`caml_main [inlined] caml_startup_exn(argv=<unavailable>) at startup_nat.c:167:10 [opt]
    frame #15: 0x000000010058b744 liquidsoap`caml_main [inlined] caml_startup(argv=<unavailable>) at startup_nat.c:172 [opt]
    frame #16: 0x000000010058b744 liquidsoap`caml_main(argv=<unavailable>) at startup_nat.c:179 [opt]
    frame #17: 0x000000010058b7ac liquidsoap`main(argc=<unavailable>, argv=<unavailable>) at main.c:44:3 [opt]
    frame #18: 0x0000000192550f34 libdyld.dylib`start + 4

toots · 2021-03-24T13:32:03Z

Also, interestingly, the binary works when installed via opam but shows no locations for the given site.

bobot · 2021-03-24T13:41:07Z

There is an open bug with site and opam install (which use the *.install file), the binary is not substituted #4212 .

rgrinberg · 2021-09-07T04:54:12Z

@toots do you still get a segfault? #4212 is now fixed.

toots · 2021-09-07T14:15:22Z

I'm afraid I still do:

% dune --version
2.9.0

# This was installed via dune install:
% /Users/toots/.opam/4.12.0/bin/liquidsoap --help
zsh: killed     /Users/toots/.opam/4.12.0/bin/liquidsoap --help

% lldb /Users/toots/.opam/4.12.0/bin/liquidsoap
(lldb) target create "/Users/toots/.opam/4.12.0/bin/liquidsoap"
r --hepCurrent executable set to '/Users/toots/.opam/4.12.0/bin/liquidsoap' (arm64).
(lldb) r --help
Process 36513 launched: '/Users/toots/.opam/4.12.0/bin/liquidsoap' (arm64)
Process 36513 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGTTOU
    frame #0: 0x0000000180db05a0 libsystem_kernel.dylib`__wait4_nocancel + 8
libsystem_kernel.dylib`__wait4_nocancel:
->  0x180db05a0 <+8>:  b.lo   0x180db05c0               ; <+40>
    0x180db05a4 <+12>: pacibsp
    0x180db05a8 <+16>: stp    x29, x30, [sp, #-0x10]!
    0x180db05ac <+20>: mov    x29, sp
Target 0: (liquidsoap) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGTTOU
  * frame #0: 0x0000000180db05a0 libsystem_kernel.dylib`__wait4_nocancel + 8
    frame #1: 0x0000000180d24580 libsystem_c.dylib`system + 428
    frame #2: 0x000000010069cf68 liquidsoap`caml_sys_system_command(command=<unavailable>) at sys.c:477:12 [opt]
    frame #3: 0x00000001006ac558 liquidsoap`caml_c_call + 28
    frame #4: 0x000000010029a0a4 liquidsoap`camlUtils__print_strings_inner_2738 + 660
    frame #5: 0x0000000100011f94 liquidsoap`camlDune__exe__Main__parse_1168 + 204
    frame #6: 0x0000000100012238 liquidsoap`camlDune__exe__Main__fun_2008 + 64
    frame #7: 0x000000010054bd54 liquidsoap`camlDtools__Dtools_impl__exec_786 + 228
    frame #8: 0x000000010057f4d4 liquidsoap`camlStdlib__list__iter_261 + 44
    frame #9: 0x000000010054bd94 liquidsoap`camlDtools__Dtools_impl__exec_786 + 292
    frame #10: 0x000000010000f7e0 liquidsoap`camlDune__exe__Liquidsoap__code_begin + 32
    frame #11: 0x000000010000741c liquidsoap`caml_program + 19236
    frame #12: 0x00000001006ac5d0 liquidsoap`caml_start_program + 108
    frame #13: 0x0000000100686f8c liquidsoap`caml_startup_common(argv=0x000000010029a0a4, pooling=<unavailable>) at startup_nat.c:158:10 [opt]
    frame #14: 0x0000000100686ff0 liquidsoap`caml_main [inlined] caml_startup_exn(argv=<unavailable>) at startup_nat.c:163:10 [opt]
    frame #15: 0x0000000100686fe8 liquidsoap`caml_main [inlined] caml_startup(argv=<unavailable>) at startup_nat.c:168 [opt]
    frame #16: 0x0000000100686fe8 liquidsoap`caml_main(argv=<unavailable>) at startup_nat.c:175 [opt]
    frame #17: 0x0000000100687050 liquidsoap`main(argc=<unavailable>, argv=<unavailable>) at main.c:41:3 [opt]
    frame #18: 0x0000000180dfd450 libdyld.dylib`start + 4
(lldb)

The binary works when installed with opam but shows no location.

Basically, still the same issue here. Thanks for checking in!

strub · 2021-11-17T16:11:41Z

Hi,

I exactly have the same issue here with EasyCrypt (but the segfault). With dune version2.9.1, dune install outputs the following:

Installing /Users/strub/Projects/EasyCrypt/_opam/lib/easycrypt/dune-package
File "_build/install/default/lib/easycrypt/dune-package", line 1, characters 0-0:
Warning: Failed to parse file, not adding version and locations information.

Looking at /Users/strub/Projects/EasyCrypt/_opam/lib/easycrypt/dune-package, I have:

(lang dune 2.9)
(name easycrypt)
(sections
 (lib
  /Users/strub/Projects/EasyCrypt/1.0/_build/install/default/lib/easycrypt))
(sites (theories lib))

The file seems well-formed, but the value for lib is invalid (I would expect something like /Users/strub/Projects/EasyCrypt/_opam/lib/easycrypt/). This is not surprising given the error above.

strub · 2021-11-17T17:12:03Z

Ok, the fix seems pretty straightforward. I'll create a PR s.t. it can be discussed. Moreover, I find weird that the ordering of pairs in sites in the dune-package file are swapped w.r.t. their counterparts in dune-project.

bobot · 2021-11-18T08:49:03Z

Thanks a lot for lokking into it. Do you understand why a segfault occurs?

strub · 2021-11-18T08:50:40Z

Thanks a lot for lokking into it. Do you understand why a segfault occurs?

No. This might be a totally unrelated issue. In my use case, I do not have a segfault.

Signed-off-by: Pierre-Yves Strub <[email protected]>

Signed-off-by: François Bobot <[email protected]>

- dune-site should replace paths in "sections", not "sites" - paths' substitution should be done in all cases (not only when a version is added to dune-package) fix #4389 Signed-off-by: Pierre-Yves Strub <[email protected]>

strub · 2021-12-09T18:40:45Z

So, I think this issue can be reopened. I have users on Mac that are experiencing their binary being killed by the system (only the promoted binary, not the one in _build). I think this might be related to that issue: #4135.

If the promoted binary is signed again with codesign -s - path-to-binary.exe, then everything works as expected. Anyway, think this means that either dune should codesign promoted binaries or stop to rewrite them on the fly.

As a side question, why dune rewrites the binary when promoted?

bobot · 2021-12-10T09:05:55Z

As a side question, why dune rewrites the binary when promoted?

It does this for updating the information about ocamlpath, sites location. The binaries in _build can be installed or promoted, when installed the location are the one of the install path, when promoted the one of _build. Moreover not using the final path in the generated OCaml module allows the cache to work better. @ejgallego as another proposition: keeping the information in a file next to the binary.

Do you really need promotion? Does a script that does exec dune exec --no-build --root $(dirname $0) -- mytool $@ would be acceptable? Making sites work with promotion added complications but is really not satisfactory. Perhaps the automatic generation of such script would be better.

Anyway, think this means that either dune should codesign promoted binaries or stop to rewrite them on the fly.

I would be happy to remove the rewriting on the fly, but the other solutions have also major drawbacks. Discussing alternative is interesting, still in the meantime codesigning the promoted binaries seems a simple fix.

Thank you a lot for raising this problem of codesigning! But I don't know if this issue is related to the one at the source of this issue.

- dune-site should replace paths in "sections", not "sites" - paths' substitution should be done in all cases (not only when a version is added to dune-package) fix ocaml#4389 Signed-off-by: Pierre-Yves Strub <[email protected]>

Signed-off-by: Pierre-Yves Strub <[email protected]>

Signed-off-by: François Bobot <[email protected]>

- dune-site should replace paths in "sections", not "sites" - paths' substitution should be done in all cases (not only when a version is added to dune-package) fix ocaml#4389 Signed-off-by: Pierre-Yves Strub <[email protected]>

Signed-off-by: Pierre-Yves Strub <[email protected]>

Signed-off-by: François Bobot <[email protected]>

- dune-site should replace paths in "sections", not "sites" - paths' substitution should be done in all cases (not only when a version is added to dune-package) fix #4389 Signed-off-by: Pierre-Yves Strub <[email protected]>

@Kakadu

CHANGES: - Fix missing -linkall flag when linking library dune-sites.plugin ( ocaml/dune#4348, @Kakadu, @bobot, reported by @Kakadu) - No longer reference deprecated Toploop functions when using dune files in OCaml syntax. (ocaml/dune#4834, fixes ocaml/dune#4830, @nojb) - Use the stag format API to be compatible with OCaml 5.0 (ocaml/dune#5351, @emillon). - Fix post-processing of dune-package (fix ocaml/dune#4389, @strub)

strub mentioned this issue Nov 17, 2021

dune-site should replace paths in "sections", not "sites" #5192

Merged

bobot self-assigned this Nov 18, 2021

bobot closed this as completed in #5192 Nov 20, 2021

bobot pushed a commit that referenced this issue Nov 20, 2021

Test case for bug #4389

ba0f1b2

Signed-off-by: Pierre-Yves Strub <[email protected]>

bobot added a commit that referenced this issue Nov 20, 2021

Test case for #4389 without build info

a76e31f

Signed-off-by: François Bobot <[email protected]>

rgrinberg reopened this Dec 10, 2021

ejgallego pushed a commit to ejgallego/dune that referenced this issue Jan 21, 2022

Test case for bug ocaml#4389

91a389d

Signed-off-by: Pierre-Yves Strub <[email protected]>

ejgallego pushed a commit to ejgallego/dune that referenced this issue Jan 21, 2022

Test case for ocaml#4389 without build info

9525164

Signed-off-by: François Bobot <[email protected]>

ejgallego pushed a commit that referenced this issue Jan 22, 2022

Test case for bug #4389

70d33c2

Signed-off-by: Pierre-Yves Strub <[email protected]>

ejgallego pushed a commit that referenced this issue Jan 22, 2022

Test case for #4389 without build info

e267196

Signed-off-by: François Bobot <[email protected]>

ejgallego mentioned this issue Jan 23, 2022

[new release] dune (2.9.2) ocaml/opam-repository#20521

Merged

kit-ty-kate closed this as completed in ocaml/opam-repository#20521 Jan 23, 2022

bobot mentioned this issue May 3, 2022

For Apple M1, binary substitution breaks codesign #5650

Closed

toots mentioned this issue May 21, 2022

Incorrect value when installing a relocatable binary with dune-site #5749

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failed to parse file, not adding version and locations information #4389

Failed to parse file, not adding version and locations information #4389

toots commented Mar 23, 2021

bobot commented Mar 23, 2021

ghost commented Mar 23, 2021

toots commented Mar 24, 2021 •

edited

Loading

toots commented Mar 24, 2021

bobot commented Mar 24, 2021

rgrinberg commented Sep 7, 2021

toots commented Sep 7, 2021 •

edited

Loading

strub commented Nov 17, 2021 •

edited

Loading

strub commented Nov 17, 2021 •

edited

Loading

bobot commented Nov 18, 2021

strub commented Nov 18, 2021

strub commented Dec 9, 2021

bobot commented Dec 10, 2021

Failed to parse file, not adding version and locations information #4389

Failed to parse file, not adding version and locations information #4389

Comments

toots commented Mar 23, 2021

bobot commented Mar 23, 2021

ghost commented Mar 23, 2021

toots commented Mar 24, 2021 • edited Loading

toots commented Mar 24, 2021

bobot commented Mar 24, 2021

rgrinberg commented Sep 7, 2021

toots commented Sep 7, 2021 • edited Loading

strub commented Nov 17, 2021 • edited Loading

strub commented Nov 17, 2021 • edited Loading

bobot commented Nov 18, 2021

strub commented Nov 18, 2021

strub commented Dec 9, 2021

bobot commented Dec 10, 2021

toots commented Mar 24, 2021 •

edited

Loading

toots commented Sep 7, 2021 •

edited

Loading

strub commented Nov 17, 2021 •

edited

Loading

strub commented Nov 17, 2021 •

edited

Loading