coreos-meta-translator: add metal4k support

[jlebon]

For coreos/fedora-coreos-tracker#385.

[jlebon]

This is what it looks like with this (nevermind the bogus URLs):

$ jq .architectures.x86_64.media.metal release.json
{
  "artifacts": {
    "raw.xz": {
      "disk": {
        "location": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal.x86_64.raw.xz",
        "signature": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal.x86_64.raw.xz.sig",
        "sha256": "057b0f0005f7971fa129e92cf45f2562294b413341bc8711e3fc16b20a6030e1"
      }
    },
    "4k": {
      "raw.xz": {
        "disk": {
          "location": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal4k.x86_64.raw.xz",
          "signature": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal4k.x86_64.raw.xz.sig",
          "sha256": "a5aeb7fc02b4d35d69edafa070287842aa24305f19f3b05774116faea6277128"
        }
      }
    }
  }
}

[bgilbert]

If your example output is correct, this changes the schema. Currently, platforms (metal) have formats (raw.xz) which have artifacts (disk). 4k adds another level. That's potentially okay for release metadata, but we'd have to translate for stream metadata anyway, since I don't think we can break the schema there.

[jlebon]

OK right, let's just go with 4k.raw.xz. Updated!

Here's a sample of the stream metadata (again, don't mind the raw vs raw.xz and other missing artifacts and bad URLs here):

{
  "stream": "bodhi-updates",
  "metadata": {
    "last-modified": "2020-03-20T15:06:59Z"
  },
  "architectures": {
    "x86_64": {
      "artifacts": {
        "metal": {
          "release": "31.20200319.dev.2",
          "formats": {
            "4k.raw": {
              "disk": {
                "location": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal4k.x86_64.raw",
                "signature": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal4k.x86_64.raw.sig",
                "sha256": "a5aeb7fc02b4d35d69edafa070287842aa24305f19f3b05774116faea6277128"
              }
            },
            "raw": {
              "disk": {
                "location": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal.x86_64.raw",
                "signature": "https://builds.coreos.fedoraproject.org/prod/streams/bodhi-updates/builds/31.20200319.dev.2/x86_64/fedora-coreos-31.20200319.dev.2-metal.x86_64.raw.sig",
                "sha256": "057b0f0005f7971fa129e92cf45f2562294b413341bc8711e3fc16b20a6030e1"
              }
            }
          }
        }
      }
    }
  }
}

(I had to use coreos/fedora-coreos-stream-generator#8 to test this)

[dustymabe]

man this is kind of ugly if we didn't have existing schema to consider what would be the "ideal" structure?

[bgilbert]

I think this is pretty good, actually. We won't be able to anticipate every image variant we'll need for some Weird Unfortunate Reason. When building the initial stream metadata design, I assumed that we'd publish those as additional formats, as we're doing here.

[dustymabe]

yeah I think we're probably doing the best we can. Right now the 'format' has been an actual format (other than pxe) whereas this is just something we made up.

Architecture  Platform   Format
x86_64        aliyun     qcow2.xz
x86_64        aws        vmdk.xz
x86_64        azure      vhd.xz
x86_64        exoscale   qcow2.xz
x86_64        gcp        tar.gz
x86_64        metal      iso
x86_64        metal      pxe
x86_64        metal      raw.xz
x86_64        metal      4k.raw.xz
x86_64        openstack  qcow2.xz
x86_64        qemu       qcow2.xz
x86_64        vmware     ova

all in all this LGTM, but I wish there was a better way.

[jlebon]

I think the fact that we call it "format" instead of just "extension" makes this more palatable. In that context, 4k vs 512b is indeed part of the format.

Anyway, agreed this is not optimal, but I don't think it's too bad. (And remember the majority of users will be downloading through getfedora, which will have a proper description of what it actually is.)