Meta: Shift from dependence on implementations to interfaces

[Gozala]

Why

We can reduce amount of coordination required across libraries by coding against interfaces as opposed to specific implementations. That way libraries that just consume / read do not need to depend on library providing implementation instead they can just consume provided values and type checker will ensure that we do pass compatible implementation.

Code that produces values would still need to import library providing an implementation, but those could live at the edges where input is received from the wire or through user events.

Current plan

• PeerId as an interface
• CID as an interface
• Multiaddr as an interface

[rvagg]

@Gozala in terms of implementation, this is going to mean we have a big fat interfaces library we all depend on? Or do each of these pieces get their own library we install as a dependency?

[achingbrain]

this is going to mean we have a big fat interfaces library we all depend on?

Please, no! Too many of these become dumping grounds for weird or redundant code and dependencies you don't need have a way of creeping in. Small, focussed modules are generally better.

The flip side is that lots of small repos are an utter pain to maintain and coordinate releases from where necessary.

I created https://github.com/ipfs/js-ipfs-interfaces last month which uses npm workspaces to mitigate this - it's only got a couple of interfaces in there at the moment but it's trivial to add more and releasing one or all of them is trivial with the tooling.

[Gozala]

@Gozala in terms of implementation, this is going to mean we have a big fat interfaces library we all depend on? Or do each of these pieces get their own library we install as a dependency?

I wrote more concrete proposal for peer-id but it reflects my general thinking across other pieces as well libp2p/js-libp2p#955 (comment)

I do not think doing it pre library is ideal especially if they'll end up depend on one another. One fat library is also not ideal as it can become dumping ground for things as @achingbrain mentioned. I think js-multiformats makes good balance between two by clustering interdependent pieces in convenient bundle.

this is going to mean we have a big fat interfaces library we all depend on?

Please, no! Too many of these become dumping grounds for weird or redundant code and dependencies you don't need have a way of creeping in. Small, focused modules are generally better.

I think avoiding dumping grounds can be accomplished simply by having a PR template with right checklist that reminds both reviewer and proposer of what makes sense to make part of the group.

I find breaking up larger packages into smaller ones to be not too distracting. Once you see that subset of the larger package is what you need to pull in, it is pretty straight forward to refactor it into separate piece and just re-export from the larger one.

It is also worth considering that just interface defs don't add any runtime overhead so putting several blocks together isn't nearly as bad as when doing it with utility libs.

I created https://github.com/ipfs/js-ipfs-interfaces last month which uses npm workspaces to mitigate this - it's only got a couple of interfaces in there at the moment but it's trivial to add more and releasing one or all of them is trivial with the tooling.

This may be a good compromise, but I would personally put all the libp2p interfaces into libp2p-interfaces as described in the comment linked above. I would also do something along those lines for multiformats. Not sure about ipfs though, as I think IPFS could few more layers e.g. dag-service.

[rvagg]

I'm becoming more and more of a dependency-scold over time so that tension exists for me - these pure type dependencies (not devdeps) in JS projects is a bother when they don't get used in the execution path. But on the other hand, the ability to iterate in js-multiformats is also a pain when we're so tightly coupled to the same set of types--we get out of sync really easily and need to do these huge rolling updates to get things back into sync. So I'm keen to see a careful balancing between those two things. It sounds like that's roughly what you're trying to propose here and I'm interested to see a concrete proposal. Probably keeping things as minimal as absolutely possible is going to solve some of the problems witnessed in the larger *-type libraries. So minimal that it should be rare that we need to even touch the type libraries, that would help with churn.

[achingbrain]

Closing this as it's largely been done. The exception is the CID class but there's further discussion in the multiformats repo.

See: multiformats/js-multiformats#203