"ktx create" produces invalid file when using --generatemipmap flag

[soadzoor]

Steps to reproduce:
I have an 8k PNG texture, which I'd like to convert to KTX2 for use in a WebGL application.

This is the command I'm using:
ktx create --format R8G8B8_SRGB --generate-mipmap --assign-oetf srgb --encode basis-lz --clevel 2 --qlevel 128 earth_color_8K.png earth_color_8K.ktx2

After validation with
ktx validate earth_color_8k.ktx2

It says the following error:

Validation failed

error-4001: Invalid Level Index. Indices must be sorted from the largest level to the smallest level.
    Indexes for level 11 with byteLength 3 and level 12 with byteLength 4 are incorrectly ordered.
warning-5004: Non-conformant texture file accepted by libktx.
    KTX 2.0 file does not conform to the specification but it is currently accepted by libktx.

Logging out the info with ktx info shows me this:

Validation failed

error-4001: Invalid Level Index. Indices must be sorted from the largest level to the smallest level.
    Indexes for level 11 with byteLength 3 and level 12 with byteLength 4 are incorrectly ordered.
warning-5004: Non-conformant texture file accepted by libktx.
    KTX 2.0 file does not conform to the specification but it is currently accepted by libktx.

Header

identifier: «KTX 20»\r\n\x1A\n
vkFormat: VK_FORMAT_UNDEFINED
typeSize: 1
pixelWidth: 8192
pixelHeight: 8192
pixelDepth: 0
layerCount: 0
faceCount: 1
levelCount: 14
supercompressionScheme: KTX_SS_BASIS_LZ
dataFormatDescriptor.byteOffset: 0x1a0
dataFormatDescriptor.byteLength: 44
keyValueData.byteOffset: 0x1cc
keyValueData.byteLength: 100
supercompressionGlobalData.byteOffset: 0x230
supercompressionGlobalData.byteLength: 12376

Level Index

Level0.byteOffset: 0x1020d3
Level0.byteLength: 2926688
Level0.uncompressedByteLength: 0
Level1.byteOffset: 0x48f8b
Level1.byteLength: 758088
Level1.uncompressedByteLength: 0
Level2.byteOffset: 0x17120
Level2.byteLength: 204395
Level2.uncompressedByteLength: 0
Level3.byteOffset: 0x9031
Level3.byteLength: 57583
Level3.uncompressedByteLength: 0
Level4.byteOffset: 0x4ecb
Level4.byteLength: 16742
Level4.uncompressedByteLength: 0
Level5.byteOffset: 0x3b41
Level5.byteLength: 5002
Level5.uncompressedByteLength: 0
Level6.byteOffset: 0x3537
Level6.byteLength: 1546
Level6.uncompressedByteLength: 0
Level7.byteOffset: 0x335b
Level7.byteLength: 476
Level7.uncompressedByteLength: 0
Level8.byteOffset: 0x32c8
Level8.byteLength: 147
Level8.uncompressedByteLength: 0
Level9.byteOffset: 0x329e
Level9.byteLength: 42
Level9.uncompressedByteLength: 0
Level10.byteOffset: 0x3293
Level10.byteLength: 11
Level10.uncompressedByteLength: 0
Level11.byteOffset: 0x3290
Level11.byteLength: 3
Level11.uncompressedByteLength: 0
Level12.byteOffset: 0x328c
Level12.byteLength: 4
Level12.uncompressedByteLength: 0
Level13.byteOffset: 0x3288
Level13.byteLength: 4
Level13.uncompressedByteLength: 0

Data Format Descriptor

DFD total bytes: 44
Vendor ID: KHR_DF_VENDORID_KHRONOS
Descriptor type: KHR_DF_KHR_DESCRIPTORTYPE_BASICFORMAT
Version: KHR_DF_VERSIONNUMBER_1_3
Descriptor block size: 40
Flags: 0x0 (KHR_DF_FLAG_ALPHA_STRAIGHT)
Transfer: KHR_DF_TRANSFER_SRGB
Primaries: KHR_DF_PRIMARIES_BT709
Model: KHR_DF_MODEL_ETC1S
Dimensions: 4, 4, 1, 1
Plane bytes: 0, 0, 0, 0, 0, 0, 0, 0
Sample 0:
    Qualifiers: 0x0 ()
    Channel Type: 0x0 (KHR_DF_CHANNEL_ETC1S_RGB)
    Length: 64 bits Offset: 0
    Position: 0, 0, 0, 0
    Lower: 0x00000000
    Upper: 0xffffffff

Key/Value Data

KTXwriter: ktx create v4.3.2~6 / libktx v4.3.2~1
KTXwriterScParams: --clevel 2 --qlevel 128

Basis Supercompression Global Data

endpointCount: 2352
selectorCount: 2725
endpointsByteLength: 4513
selectorsByteLength: 6321
tablesByteLength: 1242
extendedByteLength: 0

imageFlags: 0
rgbSliceByteLength: 2926688
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 758088
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 204395
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 57583
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 16742
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 5002
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 1546
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 476
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 147
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 42
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 11
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 3
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 4
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

imageFlags: 0
rgbSliceByteLength: 4
rgbSliceByteOffset: 0
alphaSliceByteLength: 0
alphaSliceByteOffset: 0

I'm attaching the original PNG file here:
earth_color_8K_2.png.zip

And the generated KTX2 file here:
earth_color_8K_2.ktx2.zip

(Actually the original files were 16K resolution, but I resized the image so I can upload it here. The problem doesn't happen below 8K resolution with this image, as far as I tested (4k, 2k, 1k), and it doesn't happen with other 16k and 8k images, either...)

[MarkCallow]

It looks like the validator is using only the byte length to determine level ordering and BasisLZ compression is resulting in level 11 being smaller than level 12. It is not entirely surprising as the actual data is in the global dictionaries in the supercompression global data. The level data is references to those dictionaries.

@aqnuep I think this is one for you. For some reason your handle is no longer appearing in the list of possible assignees even though you still have write permission to the repo.

[aqnuep]

We'll take a look at this. IIRC we should have had some special cases to deal with supercompressions so that we check the uncompressed byte lengths, but there could be some corner case here for BasisLZ that isn't handled as expected.

[MarkCallow]

@aqnuep please look at this soon. I am about to prepare the next release.

[aqnuep]

I didn't forget about this, but I'm not sure when we'll get the change to deal with this. I'll try to give this one a priority.