fix(x-compilation): find host ppx dependencies in the host context #7415

anmonteiro · 2023-03-26T04:23:58Z

While researching the cross-compilation story to fix #6191, I found about this failing test case from #4155 that never got fixed

cc @TheLortex

Alizter · 2023-03-26T05:02:51Z

Would this mean you could have two opam switches and use the ppx from one when compiling packages depending on the other?

anmonteiro · 2023-03-26T05:33:35Z

I'm not too familiar with how contexts and opam switches are related, but I think there's a relation. If they're 1:1 I suspect that should be possible.

anmonteiro · 2023-03-26T20:26:30Z

I'm not too familiar with how contexts and opam switches are related, but I think there's a relation. If they're 1:1 I suspect that should be possible.

Reading the docs and the code a little more, it seems like this is in fact the case if you're setting up your cross compilation environment in OPAM. I think it might be even a bit more restrictive: you can only depend on PPXes in the other switch.

Alizter · 2023-03-26T20:48:24Z

That still seems OK to me. I wouldn't consider ppx a "runtime dependency" i.e. something that needs to be around when the code is executing but rather a "build time dependency". Modifying source code should be independent of platform, so whether this is performed in the target context or the host context doesn't really matter.

anmonteiro · 2023-03-26T20:54:52Z

Note that this doesn't exclude PPXes from being cross-compiled. This change just interprets the library dependencies in (pps ... ) in the host context.

Alizter · 2023-03-26T20:58:22Z

@anmonteiro Could you add a note about that in the cross-compilation section? i.e. that deps from pps are interpreted in the host?

anmonteiro · 2023-03-26T21:05:46Z

I believe this is already covered, it's just that it's broken. From https://dune.readthedocs.io/en/stable/cross-compilation.html#how-does-it-work :

One consequence of this is that all preprocessing (PPX or otherwise) will be done using binaries built in the default context.

src/dune_rules/lib.ml

test/blackbox-tests/test-cases/ppx-cross-context-issue.t/run.t

src/dune_rules/scope.ml

anmonteiro · 2023-03-27T04:10:46Z

I tested this PR in my cross-compilation setup and everything seems to work as expected 👍

rgrinberg

LGTM. Don't forget a CHANGES entry

rgrinberg · 2023-03-27T05:20:48Z

src/dune_rules/scope.ml

+          let host =
+            let host =
+              Memo.Lazy.create @@ fun () ->
+              let+ installed_libs = Lib.DB.installed ~host:None host_context in


One day we should allow host contexts to have their host contexts as well.

Signed-off-by: Antonio Nuno Monteiro <[email protected]>

anmonteiro · 2023-03-27T05:29:28Z

Rebased on main and added a CHANGES entry

Alizter · 2023-03-27T05:30:42Z

Just to be clear about the terminology, since this always confuses me, are we saying:

Host platform (where we are targetting)
Build platform (where we are building)

anmonteiro · 2023-03-27T05:34:29Z

In my understanding, it's actually a triplet: build, host and target. This is how I think about it:

Autotools	Dune
Build	Host
Host	---
Target	Target

TheLortex · 2023-03-28T07:35:34Z

That's great, thank you for the fix @anmonteiro !

…ntext (ocaml#7415)" This reverts commit 16a7e88.

…ntext (ocaml#7415)" This reverts commit 16a7e88. Signed-off-by: Etienne Millon <[email protected]>

* Revert "fix(x-compilation): delay evaluation of `ppx_runtime_deps` until context is known" This reverts commit ab74a71. Signed-off-by: Etienne Millon <[email protected]> * Revert "test(x-compilation): demonstrate overlap check failure with ppx_runtime_libraries" This reverts commit 096fc97. Signed-off-by: Etienne Millon <[email protected]> * Revert "fix(x-compilation): find host ppx dependencies in the host context (#7415)" This reverts commit 16a7e88. Signed-off-by: Etienne Millon <[email protected]> * Changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Etienne Millon <[email protected]>

* Revert "fix(x-compilation): delay evaluation of `ppx_runtime_deps` until context is known" This reverts commit ab74a71. Signed-off-by: Etienne Millon <[email protected]> * Revert "test(x-compilation): demonstrate overlap check failure with ppx_runtime_libraries" This reverts commit 096fc97. Signed-off-by: Etienne Millon <[email protected]> * Revert "fix(x-compilation): find host ppx dependencies in the host context (ocaml#7415)" This reverts commit 16a7e88. Signed-off-by: Etienne Millon <[email protected]> * Changelog Signed-off-by: Etienne Millon <[email protected]> --------- Signed-off-by: Etienne Millon <[email protected]>

* Revert "fix(x-compilation): delay evaluation of `ppx_runtime_deps` until context is known" This reverts commit ab74a71. * Revert "test(x-compilation): demonstrate overlap check failure with ppx_runtime_libraries" This reverts commit 096fc97. * Revert "fix(x-compilation): find host ppx dependencies in the host context (#7415)" This reverts commit 16a7e88. * Changelog --------- Signed-off-by: Etienne Millon <[email protected]>

@Alizter

CHANGES: - Fix a crash when using a version of Coq < 8.13 due to the native compiler config variable being missing. We now explicitly default to `(mode vo)` for these older versions of Coq. (ocaml/dune#7847, fixes ocaml/dune#7846, @Alizter) - Duplicate installed Coq theories are now allowed with the first appearing in COQPATH being preferred. This is inline with Coq's loadpath semantics. This fixes an issue with install layouts based on COQPATH such as those found in nixpkgs. (ocaml/dune#7790, @Alizter) - Revert ocaml/dune#7415 and ocaml/dune#7450 (Resolve `ppx_runtime_libraries` in the target context when cross compiling) (ocaml/dune#7887, fixes ocaml/dune#7875, @emillon)

This reverts commit 886df09.

This reverts commit 886df09. Signed-off-by: Antonio Nuno Monteiro <[email protected]>

This reverts commit b566e50.

…7890)"" This reverts commit 4323a7c.

…ocaml#7890)""" This reverts commit d7ee814.

anmonteiro changed the title ~~fix: find host ppx dependencies in the host context~~ fix(x-compilation): find host ppx dependencies in the host context Mar 26, 2023

anmonteiro force-pushed the anmonteiro/with-host-db-arg branch 3 times, most recently from e783536 to 312f778 Compare March 26, 2023 04:48

anmonteiro force-pushed the anmonteiro/with-host-db-arg branch from 312f778 to 9f8c810 Compare March 26, 2023 20:15

rgrinberg linked an issue Mar 27, 2023 that may be closed by this pull request

ppxs are built in the target context in cross-compilation settings #4156

Open

rgrinberg added this to the 3.8.0 milestone Mar 27, 2023

rgrinberg reviewed Mar 27, 2023

View reviewed changes

src/dune_rules/lib.ml Outdated Show resolved Hide resolved

test/blackbox-tests/test-cases/ppx-cross-context-issue.t/run.t Outdated Show resolved Hide resolved

anmonteiro force-pushed the anmonteiro/with-host-db-arg branch 2 times, most recently from a3adfed to 10e71cc Compare March 27, 2023 01:53

rgrinberg reviewed Mar 27, 2023

View reviewed changes

src/dune_rules/scope.ml Outdated Show resolved Hide resolved

anmonteiro force-pushed the anmonteiro/with-host-db-arg branch from 10e71cc to 688354f Compare March 27, 2023 03:19

rgrinberg reviewed Mar 27, 2023

View reviewed changes

src/dune_rules/scope.ml Show resolved Hide resolved

anmonteiro force-pushed the anmonteiro/with-host-db-arg branch from 688354f to 91ced91 Compare March 27, 2023 03:55

rgrinberg approved these changes Mar 27, 2023

View reviewed changes

fix(x-compilation): find host ppx dependencies in the host context

9dc3cc7

Signed-off-by: Antonio Nuno Monteiro <[email protected]>

anmonteiro force-pushed the anmonteiro/with-host-db-arg branch from 91ced91 to 9dc3cc7 Compare March 27, 2023 05:29

rgrinberg merged commit 16a7e88 into ocaml:main Mar 27, 2023

anmonteiro added the cross-compilation label Mar 27, 2023

anmonteiro added a commit to anmonteiro/dune that referenced this pull request Mar 30, 2023

Revert "fix(x-compilation): find host ppx dependencies in the host co…

995a110

…ntext (ocaml#7415)" This reverts commit 16a7e88.

anmonteiro mentioned this pull request Mar 30, 2023

test(x-compilation): demonstrate overlap check failure with ppx_runtime_libraries #7450

Merged

gridbugs added a commit to gridbugs/dune that referenced this pull request Apr 13, 2023

Revert "fix(x-compilation): find host ppx dependencies in the host co…

72d9cce

…ntext (ocaml#7415)" This reverts commit 16a7e88.

gridbugs mentioned this pull request Apr 14, 2023

Can't resolve ppxlib.traverse when ppxlib is vendored #7543

Closed

gridbugs added a commit to gridbugs/dune that referenced this pull request Apr 14, 2023

Revert "fix(x-compilation): find host ppx dependencies in the host co…

8390612

…ntext (ocaml#7415)" This reverts commit 16a7e88.

anmonteiro mentioned this pull request Apr 14, 2023

test: demonstrate regression depending on a ppx by its private name #7553

Merged

dinosaure mentioned this pull request Jun 5, 2023

3.8.0 regression: Error: Library "ppx_deriving_runtime" not found. #7875

Closed

emillon mentioned this pull request Jun 5, 2023

Revert #7415 and #7450 #7887

Merged

emillon added a commit to emillon/dune that referenced this pull request Jun 5, 2023

Revert "fix(x-compilation): find host ppx dependencies in the host co…

25d17ef

…ntext (ocaml#7415)" This reverts commit 16a7e88.

emillon added a commit to emillon/dune that referenced this pull request Jun 5, 2023

Revert "fix(x-compilation): find host ppx dependencies in the host co…

f55a205

…ntext (ocaml#7415)" This reverts commit 16a7e88. Signed-off-by: Etienne Millon <[email protected]>

emillon added a commit to emillon/dune that referenced this pull request Jun 5, 2023

Revert "fix(x-compilation): find host ppx dependencies in the host co…

b0bedbf

…ntext (ocaml#7415)" This reverts commit 16a7e88. Signed-off-by: Etienne Millon <[email protected]>

emillon mentioned this pull request Jun 5, 2023

[3.8] backport #7887 #7890

Merged

emillon mentioned this pull request Jun 5, 2023

[new release] dune (15 packages) (3.8.1) ocaml/opam-repository#23884

Merged

anmonteiro added a commit to anmonteiro/dune that referenced this pull request Jun 5, 2023

Revert "Revert ocaml#7415 and ocaml#7450 (ocaml#7887)"

1969321

This reverts commit 886df09.

anmonteiro added a commit to anmonteiro/dune that referenced this pull request Jun 5, 2023

Revert "Revert ocaml#7415 and ocaml#7450 (ocaml#7887)"

6282041

This reverts commit 886df09. Signed-off-by: Antonio Nuno Monteiro <[email protected]>

anmonteiro added a commit to anmonteiro/dune that referenced this pull request Jun 9, 2023

Revert "Revert ocaml#7415 and ocaml#7450 (ocaml#7887)"

a1ac1e2

This reverts commit 886df09. Signed-off-by: Antonio Nuno Monteiro <[email protected]>

emillon added a commit to emillon/dune that referenced this pull request Jun 27, 2023

Revert "Revert ocaml#7415 and ocaml#7450 (ocaml#7887) (ocaml#7890)"

4323a7c

This reverts commit b566e50.

emillon added a commit to emillon/dune that referenced this pull request Jun 28, 2023

Revert "Revert "Revert ocaml#7415 and ocaml#7450 (ocaml#7887) (ocaml#…

d7ee814

…7890)"" This reverts commit 4323a7c.

emillon added a commit to emillon/dune that referenced this pull request Jun 28, 2023

Revert "Revert "Revert "Revert ocaml#7415 and ocaml#7450 (ocaml#7887) (…

73bd375

…ocaml#7890)""" This reverts commit d7ee814.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(x-compilation): find host ppx dependencies in the host context #7415

fix(x-compilation): find host ppx dependencies in the host context #7415

anmonteiro commented Mar 26, 2023 •

edited

Loading

Alizter commented Mar 26, 2023

anmonteiro commented Mar 26, 2023

anmonteiro commented Mar 26, 2023

Alizter commented Mar 26, 2023

anmonteiro commented Mar 26, 2023

Alizter commented Mar 26, 2023

anmonteiro commented Mar 26, 2023 •

edited

Loading

anmonteiro commented Mar 27, 2023

rgrinberg left a comment

rgrinberg Mar 27, 2023

anmonteiro commented Mar 27, 2023

Alizter commented Mar 27, 2023

anmonteiro commented Mar 27, 2023

TheLortex commented Mar 28, 2023

fix(x-compilation): find host ppx dependencies in the host context #7415

fix(x-compilation): find host ppx dependencies in the host context #7415

Conversation

anmonteiro commented Mar 26, 2023 • edited Loading

Alizter commented Mar 26, 2023

anmonteiro commented Mar 26, 2023

anmonteiro commented Mar 26, 2023

Alizter commented Mar 26, 2023

anmonteiro commented Mar 26, 2023

Alizter commented Mar 26, 2023

anmonteiro commented Mar 26, 2023 • edited Loading

anmonteiro commented Mar 27, 2023

rgrinberg left a comment

Choose a reason for hiding this comment

rgrinberg Mar 27, 2023

Choose a reason for hiding this comment

anmonteiro commented Mar 27, 2023

Alizter commented Mar 27, 2023

anmonteiro commented Mar 27, 2023

TheLortex commented Mar 28, 2023

anmonteiro commented Mar 26, 2023 •

edited

Loading

anmonteiro commented Mar 26, 2023 •

edited

Loading