Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to prevent exposing modules from a haskell_library? #152

Closed
iphydf opened this issue Feb 21, 2018 · 15 comments
Closed

How to prevent exposing modules from a haskell_library? #152

iphydf opened this issue Feb 21, 2018 · 15 comments
Assignees
Labels
P2 major: an upcoming release type: feature request

Comments

@iphydf
Copy link

iphydf commented Feb 21, 2018

The aeson package has a private Data.Aeson.Compat, in cabal "other-modules". If exposed, it clashes with the aeson-compat package exposing the same module name.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 21, 2018

Good question, I've been thinking about this myself. We have to check how this is achieved in Cabal.

@Fuuzetsu
Copy link
Collaborator

Fuuzetsu commented Feb 22, 2018

You do it at ghc-pkg register time. Currently we expose everything by default. See http://downloads.haskell.org/~ghc/latest/docs/html/users_guide/packages.html#installedpackageinfo-a-package-specification .

I don't know if we want to replicate what cabal does. In my experience hidden modules are a pain for the developers. The default stack new command these days produces package.yaml which does not list modules at all and generates a cabal file with all modules exposed. There's always the PackageImports extension to disambiguate modules exposed by multiple packages.

Fixing this should be pretty easy with a new attribute, the question is more of whether we want to. Probably.

Edit: To answer the OP: you currently can't do it, it'd be a new feature.

@Fuuzetsu Fuuzetsu added type: feature request P2 major: an upcoming release labels Feb 22, 2018
@mrkkrp
Copy link
Member

mrkkrp commented Feb 22, 2018

In my experience hidden modules are a pain for the developers.

My favorite feature! Love to hide things that should not be visible. This, combined with not-exporting everything on module boundaries, smart constructors, etc. one of the reasons I like Haskell. Although opinions may differ, of course.

Being able to control what is public and what is hidden It is so essential.

@mboes
Copy link
Member

mboes commented Feb 22, 2018

In my experience hidden modules are a pain for the developers.

My favorite feature!

I'm surprised @qnikst hasn't chimed in yet?

@qnikst
Copy link

qnikst commented Feb 22, 2018

I have not seen that yet. Hidden modules are PITA for all library users (I'm not sure if haskell_rules are indended to be used for building public libraries though); because all the time library authours are trying to overthink their users and very often that fails, and require users to push PR and unhide modules or just clone the code.

@Fuuzetsu
Copy link
Collaborator

@mrkkrp Use .Internal modules. You can even hide them in Haddock documentation (please don't). Library authors do not know better than the users about all the use-cases. If you have external knowledge about something and you know that it preserves library invariants, why force the user to jump through the hoops of the API? I just opened yet another "please don't hide internals" ticket yesterday.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 22, 2018

Use .Internal modules.

Then we start to rely on convention for writing common code "don't import internal modules" instead of forcing something reliably (which I like to do very much). If we're to code using conventions, then why use types, etc., it's the same thing. You probably want to see internals to do some hacking and get things done. But then types may also seem inconvenient, as in "without this type-mess I'd get this done much easier and quicker".

because all the time library authours are trying to overthink their users and very often that fails, and require users to push PR and unhide modules

Doesn't happen much to my libraries. When it happens I'd rather start thinking what's wrong with public API, why it's not powerful enough, etc.

Library authors do not know better than the users about all the use-cases

We must talk to users to rectify that. That's what have been happening with Megaparsec for a long time, I got dozens of feature requests. Probably if I wasn't so stubborn those users would quietly hack something for themselves using access to internals and all others would end up with a less powerful library.

Anyway I see your point from "getting things done" perspective, maybe I'll end up agreeing after some time.

@mrkkrp
Copy link
Member

mrkkrp commented Feb 22, 2018

A better answer is probably "it depends", but the ability to hide things is by itself must be available, for those who know when to use it (let's assume library authors are not completely detached from their users).

@iphydf
Copy link
Author

iphydf commented Feb 22, 2018

I'm not sure if haskell_rules are indended to be used for building public libraries though

I really hope so, because if I can't build public libraries (I assume you meant hackage packages) that are dependencies of my programs, I can't use it, and probably rules_haskell would fill a very small niche (everything in-house, only depending on base).

If we can convince aeson to rename their internal Compat module to Internal.Compat, I'm happy too. I don't know how often this clash happens, but I only wrote BUILD files for 93 packages so far, so I don't know what's still coming down the line. If you don't want to support hiding modules at all, but are willing to maintain an aeson fork that's compatible with aeson-compat and rules_haskell, I can use that. If you know of a reasonable (non-viral) workaround that I can apply to my external BUILD files for aeson and/or aeson-compat, I'm also happy.

I guess one workaround that I could try, in case this issue won't be solved on the rules_haskell side, is to apply a patch to the aeson sources before building. This would involve making a copy of every source file except Compat.hs, and copy Compat.hs to Internal/Compat.hs, and then doing s/Compat/Internal.Compat/g on all source files with genrules.

@mboes
Copy link
Member

mboes commented Feb 22, 2018

This was discussed offline among a few engineers and the consensus seems to be that while we're not keen to encourage the use of hidden modules, it's useful for people building Hackage packages so let's do it.

Incidentally, the way I've been dealing with Hackage packages is to simply offload building them to Nix, and hence get the benefit of the Nix cache today without relying with Bazel's remote caching (which is still quite new). That's what rules_nixpkgs helps with. Other users (including yourself) have taken the approach of building everything in Bazel. Time will tell what best practices we should recommend here. It's quite unclear at this point.

@iphydf
Copy link
Author

iphydf commented Feb 22, 2018

FWIW, bazel remote caching is awesome, but only if your build is hermetic (nix is hermetic, bazel doesn't have to be). For Haskell, remote caching could be less interesting at the moment, depending on how rules_haskell is implemented.

@mboes
Copy link
Member

mboes commented Feb 27, 2018

I only wrote BUILD files for 93 packages so far, so I don't know what's still coming down the line

@iphydf was curious - what are you using rules_haskell for currently? We'll be publishing a blog post in the next couple days about the project. It'll be helpful to hear about what people have already been doing with it. :)

@iphydf
Copy link
Author

iphydf commented Feb 27, 2018

https://github.com/TokTok/toktok-stack

This is our "software stack", containing build rules for all dependencies of all our projects. We use it for rapid development, especially for integration of projects. Each individual project keeps its own language-specific build system, but we additionally have an integration build using Bazel. Right now, it's focussed mostly on building on Linux. Some of the projects also build under OSX and FreeBSD. A possible future goal is to get cross-compilation working so we can build Android and Raspberry Pi binaries.

We used to have a Makefile at the root of toktok-stack that declares inter-project dependencies and rebuilds projects when needed. This was unreliable and slow. Unreliable because the dependencies had to be maintained manually and they would sometimes change or break. Slow, because when a dependency changed, the entire project had to be rebuilt, because the makefile didn't contain fine-grained dependency information.

With rules_haskell, I'm not done yet, as you know I move forward each time you fix an issue I report. The goal is to integrate our Haskell projects into this stack. The next big thing is the github-tools project, which is the backend to the project's pull requests page (currently CSS is broken, the backend also serves html). Then, we also have Haskell bindings to c-toxcore, with a test and a "groupbot" binary. That already works today. After that, we want to build hs-toxcore, a Haskell reimplementation of c-toxcore. Finally, c-toxcore-hs is a C export of hs-toxcore providing the same API as c-toxcore, to be used as drop-in replacement. All of this will be integrated in the Bazel build. Exporting a C API will be an interesting use case for rules_haskell, as it's currently a hand-written Makefile.

This was a rough overview. Anything else you're interested in?

@mboes
Copy link
Member

mboes commented Feb 27, 2018

Wow, that's awesome. Thanks for the overview. :) Any objection to be cited in the "current users" section of the post?

@iphydf
Copy link
Author

iphydf commented Feb 27, 2018

No objections :).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 major: an upcoming release type: feature request
Projects
None yet
Development

No branches or pull requests

5 participants