Add Ubuntu package scripts

[alexreinking]

This PR would add long-awaited Ubuntu package scripts!

Ubuntu

The Ubuntu package is split into runtime, development, and documentation components. Development naturally depends on runtime. The packages have standard names: libhalideV, libhalideV-dev, and libhalideV-doc where V is the major version (also the SOVERSION). Naturally, they link to system LLVM.

I had created a PPA we can use but apparently launchpad wants source debs instead of binary debs. These have a weird, special format that's poorly documented and---once unpacked---passes through nearly a dozen bespoke debian packaging tools to compile our sources from scratch and feed them through their packaging pipeline. So that's not happening when we can just publish our perfectly good CPack-generated debs on a private APT repo.

GitHub Actions

There is testing for the Ubuntu packages. We create a GHA VM with developer tools for building and packaging Halide, then save the packages to scratch storage before creating a second, fresh VM with a minimal set of tools. The second VM installs the packages and then tests a very basic pipeline in a wide variety of build scenarios to test that our CMake package works as expected and without any special setup. These tests are located in test/integration.

To-dos

• Verify Ubuntu package dependencies
  • Hear back on the CMake Discourse about dpkg-shlibdeps:
• GHA testing
  • Test Ubuntu package
    • Test JIT with shared/static
    • Test AOT with shared/static
    • Test AOT error when attempting to use an autoscheduler and static linking
    • Test JIT with both shared and static
    • Test AOT cross-compile
• Update buildbots to call TGZ / ZIP packaging scripts by their new names.

Fixes #5285

[alexreinking]

By the way, I think that GHA would be the ideal place to test these packages if we decide only to support them on common hosts (ie. i686 and/or amd64). It should build Halide in one step, save the binary packages as an artifact, spawn a new, clean VM, and then try installing the package and building / running the apps in a second step.

[steven-johnson]

By the way, I think that GHA would be the ideal place to test these packages if we decide only to support them on common hosts (ie. i686 and/or amd64).

OK for now, though I predict that it will be desirable to have aarch64 support for Nuget (at least) in the medium-term future.

[steven-johnson]

I have created a PPA we can use here: https://launchpad.net/~alexreinking/+archive/ubuntu/halide (we might want a more-official one, but the setup involves creating GPG keys and tying it to an email address, so that was difficult to do without coordination).

Definitely worth looking into a location we'd be happy(er) to support longterm, but that can wait till everything else is done. (Something at halide-lang.org seems the obvious option.)

[steven-johnson]

LGTM.

[alexreinking]

The changelist was getting crazy, so I completely redid the history so you can understand the different parts of this PR better. Some things—like the spelling fixes and #! in autotune_loop.sh—could be split into a separate PR if that would make reviewing this easier. Those changes were made, however, because lintian discovered them.

[alexreinking]

Running into this with my cross-compiling efforts. It would be nice if we had some MinGW compatibility again... https://metricpanda.com/rival-fortress-update-45-dealing-with-__chkstk-__chkstk_ms-when-cross-compiling-for-windows/

[steven-johnson]

It would be nice if we had some MinGW compatibility again

Gonna get a hard no on that from me. MinGW is a big pain to support and appears to be an environment rapidly losing mindshare.

[alexreinking]

Okay, this is finally ready!

I've moved the Nuget stuff into a separate branch (cpack/nuget) since I'm stuck reverse engineering the package format. This is largely undocumented territory, unfortunately.

I'm cross compiling from amd64 Linux to aarch64 Linux and testing with Qemu.

There will probably be a follow-up PR to this where I address the extra lintian checks, write some man pages, etc. but this is at a point where I'm comfortable merging it, using the Ubuntu packages myself, and (finally) cutting Halide 12.

[steven-johnson]

Is this for packaging Halide 11 or Halide 12? (Halide 12 will compile against LLVM11, but should probably preferentially be packaged with LLVM12.)

[alexreinking]

Ubuntu 20.04 LTS doesn't package LLVM 12, but it does come with LLVM 11. Maybe it would work to set up LLVM's apt repo, but I think this work can be done in a separate PR.

[alexreinking]

It might also be worth backporting these changes to v11 to provide Halide 11 + LLVM 11 packages on Ubuntu 20.04.