Partial packages per target framework

[nvborisenko]

NuGet Product(s) Involved

Other/NA

The Elevator Pitch

Most existing nuget packages support multi target frameworks. These packages contain lib/net5.0 and lib/net6.0 inside (for example).

Usually user is targeted to only one framework (let's say net6.0), so, most probably, he needs to download only lib/net6.0. I want to say, that there are some parts in nuget package that are not consumed by user deterministically.

What if nuget restore downloads/caches only necessary parts of a consumed package. Am I crazy?

Additional Context and Details

No response

[kartheekp-ms]

Good idea. Having this information per target framework also enables package authors to take informed decisions such as deprecating support for older tfms based on usage stats NuGet/NuGetGallery#3450. Currently Client does the compatibility checks based on project's target frameworks and if we proceed in this route then the responsibility is transferred to the Server side. NuGet also has package signing functionality. NuGet Client verifies package integrity and authenticity during restore.

If I am not mistaken, there must be an issue for this feature request already but couldn't find in my search so far.

[nkolev92]

NuGet packages ship as a zip file, as such partial download are impossible. Signed package verification is a core NuGet feature that simply wouldn't work as is, if we were to somehow start doing partial downloads.

Finally, multiple frameworks within a project actually a heavily used feature.

[fowl2]

I think the idea would be to produce multiple files for the same "package", each of which could be signed. Perhaps it could end up being implemented with a meta or nested package design and some UI enhancements to group them together.

Edit, related: #11004

[zivkan]

This issue proposes a solution, but what's the problem that you'd like to be solved by this request?

If the problem is the size of NuGet's global packages folder, #4980 is probably a better solution. There's already a nuget.config setting to have NuGet update a timestamp on a package file every restore, which cleanup tools could use to clean only packages that haven't been used recently. I was hoping that someone in the community would create a cleanup tool that uses this, other development ecosystems have a more lively community than .NET does, so I'm a bit disappointed. Maybe I'll release a sample CLI tool myself, so we can experiment with different cleanup configurations.

If the concern is large downloads on slow internet connections, I'm curious how much of a problem this really is. Using NuGet Insights, here are the zip sizes in KB for all packages (all versions), and then just the latest version of each package id, by "percentile"

type	10	25	50	90	95	99
all versions	704	907	1160	2464	3278	5848
latest version	542	728	978	1823	2370	4357

So, less than 1% of all packages on nuget.org at larger than around 5 megabytes. I understand that internet speeds vary a lot both across countries and also within countries. I grew up in Australia, and before I left, my mum's house had ~8Mb/s ADSL2+, and my own house was closer to 4Mb/s. But new technologies are available now and my mum gets around 26Mb/s now. Looking at https://en.wikipedia.org/wiki/List_of_sovereign_states_by_Internet_connection_speeds, the 100th country's average speed is over 30Mb/s in 3/4 data sources. The 4th data source lists it at only 3.2Mb/s. Unfortunately this is "average", so basically 50th percentile. It would be very interesting to know what the slowest 25% and 10% are.

Assuming zero latency, this means that any of these internet connections, even the slowest, can download most of the packages on nuget.org in less than a second. The average internet connection can download a package in theoretically a tenth of a second.

However, latency is not zero. A few years ago, I tested changing NuGet's graph resolver to download only the json metadata from HTTP servers, rather than the full nupkg, and then download the nupkgs only once the graph had been resolved and package versions selected. On the NuGet.Client repo, where there were a reasonable number of packages downloaded that were not selected, there was basically no performance difference. Testing orleans and orchardcore where there were fewer (or zero?) non-selected packages downloaded, my test NuGet was slower, for overall restore performance. Admittedly, this was on a very fast internet connection, so maybe slow internet connections could have a benefit.

So, earlier prototypes found that increasing HTTP requests in order to reduce download size had a negative impact, not positive. With this issue's suggestion, when a solution has projects using 2 or more TFMs, then it would need to download multiple nupkgs, and therefore this latency problem is relevant.

Overall, speaking only for myself, not the whole team, looking at average internet speed data, and average package size data (at least for packages on nuget.org), I'm not convinced that this suggestion is worthwhile investigating. It would be a very major change, and therefore take a huge amount of effort to implement, will require server API changes, and therefore all server implementations will have to adapt, so a high cost not just to the NuGet client and nuget.org teams, but many other teams as well. Therefore, given the high cost to implement, it needs a large benefit to be worthwhile, which I just don't see.