Zip64 field size

[01mf02]

In

fq/format/zip/zip.go

Line 241 in cc522b1

zi.diskNumberWhereFileStarts = d.FieldU32("disk_number_where_file_starts")

, the "diskNumberWhereFileStarts" is decoded as 32-bit integer, whereas the field in zip64ExtendedInformation holds a 64-bit integer. At least on Wikipedia, the size is given as 32 bits.

[wader]

You mean that the diskNumberWhereFileStarts member in the zip64ExtendedInformation struct is 64 bit? atm all integers are 64 bit and are stored quite wastefully, in the case of unsigned integer it will use https://github.com/wader/fq/blob/master/pkg/scalar/scalar_gen.go#L1324. The idea at some point has been to use the Scalarable interface (maybe renamed) to somehow switch between implementation if value has a mapping, description etc to save on memory.

[01mf02]

Ah, I understand! I missed that all other numeric fields were also parsed to uint64. Sorry for the noise.

[wader]

No worries! it's not pretty but not sure if there is a nicer way to express it with the type system. Maybe bit size integer alises but would be tricky to range limit them in a nice way 🤔

[01mf02]

I'm not sure whether this can be done a lot better. I currently use something like the following:

pub enum Val {
    Bool(bool),
    U8(u8),
    U16(u16),
    U32(u32),
    U64(u64),
    // ...
}

Here, because there is at least one variant that is 64 bits wide (namely U64), the whole Val takes at least 64 bits of storage. (And even more than that, because it also has to store which variant the Val is.)
So even if you say that you store numbers "quite wastefully", I think that my approach is just as wasteful as yours. ;)

[wader]

Aha yes :) for fq currently each decoded value is a "Scalar" struct to store optional mapping, description etc. That one i would like to turn into an interface instead so it that it could store just an 64 bit number in the case of a integer that has no mapping or description, or maybe mapping etc could be done using the interface, not sure yet.

About integer types i think it could have made sense for fq if go could express bit-sized types, U1, U2, and so on.