Making nuget package with aar file always has first resource file corrupted

[Basewq]

Android application type

.NET Android (net7.0-android, net8.0-android, etc.)

Affected platform version

VS 2022 17.9.6, Android Manifest Version: 34.0.52/8.0.100

Description

I am trying to make a nuget library in .net8-android that include android resource files to a nuget package (packaged into the aar file), but the first resource file in the aar can't be extracted (always the first file no matter the content).
If I try to extract all contents of the aar with 7z to inspect, every file extracts fine except the first resource file. In 7z, the error only says
Headers Error: res\values\file1.xml
(This will then just 'extract' as an empty file)

Attached is the simplest example
AndroidLibTest.zip
For a quick glance, below is a screenshot of the project layout:

I have seen other libraries that have aar files, and all contents can be extracted without issue in 7z, so I don't think this is a 7z issue.

Steps to Reproduce

1. See zip attachment above for the project.
2. Run dotnet pack, or Pack via Visual Studio.
3. Extract the nuget package with 7z (or whatever)
4. Try extract AndroidNugetTest.1.0.0\lib\net8.0-android34.0\AndroidNugetTest.aar

Did you find any workaround?

Not tested, but maybe make a dummy/sacrificial resource file that's always the first one to ensure it's the only one corrupted?

Relevant log output

No response

[dellis1972]

Interesting unzip -lvt reports the same issue

Archive:  bin/Release/net8.0-android/AndroidNugetTest.aar
    testing: res/values/file1.xml    res/values/file1.xml:  unknown compression method
    testing: res/values/file2.xml     OK
    testing: .net/__res_name_case_map.txt   OK

@grendello could this be our illusive libzipsharp bug?

[grendello]

@dellis1972 yep, I'd say this is our heisenbug...

[dellis1972]

@grendello

Interestingly this test fails consistently on mac. I altered the ZipArchive.Create and ZipArchive.Open methods to allow for the addition of a strictConsistencyChecks arg (and the flag to libzip). This fails 100% of the time on the ZipArchive.Open after the archive has been written.

using (var stream = File.Create ("test-archive-write.zip"))
			using (var zip = ZipArchive.Create (stream, strictConsistencyChecks:true)) {
				foreach (var entry in zip) {
					Console.Write (entry.FullName);
				}
				ZipEntry e;
				e = zip.AddFile (filePath, "/path/ZipTestCopy1.exe");
				filePath = Path.GetFullPath ("file2.txt");
				e = zip.AddFile (filePath, "/path/ZipTestCopy2.exe");

				foreach (var entry in zip) {
					Console.Write (entry.FullName);
				}

				zip.Close ();
			}

			using (var zip = ZipArchive.Open ("test-archive-write.zip", FileMode.Open, strictConsistencyChecks: true)) {
				foreach (var entry in zip) {
					Console.Write (entry.FullName);
				}
			}

[grendello]

It's a good thing that we finally have a reliable repro :)

The attached zip is indeed broken in a weird way:

$ zipinfo -vvv AndroidLibTest.zip | less

Archive:  AndroidLibTest.zip
There is no zipfile comment.

End-of-central-directory record:
-------------------------------

  Zip archive file size:                      2343 (0000000000000927h)
  Actual end-cent-dir record offset:          2321 (0000000000000911h)
  Expected end-cent-dir record offset:        2321 (0000000000000911h)
  (based on the length of the central directory and its expected offset)

  This zipfile constitutes the sole disk of a single-part archive; its
  central directory contains 8 entries.
  The central directory is 998 (00000000000003E6h) bytes long,
  and its (expected) offset in bytes from the beginning of the zipfile
  is 1323 (000000000000052Bh).

Central directory entry #1:
---------------------------

  AndroidLibTest/

  offset of local header from start of archive:   0
                                                  (0000000000000000h) bytes
  file system or operating system of origin:      MS-DOS, OS/2 or NT FAT
  version of encoding software:                   6.3
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT
  minimum software version required to extract:   2.0
  compression method:                             none (stored)
  file security status:                           not encrypted
  extended local header:                          no
  file last modified on (DOS date/time):          2024 May 28 22:01:18
  32-bit CRC value (hex):                         00000000
  compressed size:                                0 bytes
  uncompressed size:                              0 bytes
  length of filename:                             15 characters
  length of extra field:                          36 bytes
  length of file comment:                         0 characters
  disk number on which file begins:               disk 1
  apparent file type:                             binary
  non-MSDOS external file attributes:             000000 hex
  MS-DOS file attributes (10 hex):                dir 

  The central-directory extra field contains:
  - A subfield with ID 0x000a (PKWARE Win32) and 32 data bytes.  The first
    20 are:   00 00 00 00 01 00 18 00 3d c9 cb fb e5 b0 da 01 3d c9 cb fb.

  There is no file comment.

Central directory entry #2:
---------------------------

  There are an extra -36 bytes preceding this file.

  AndroidLibTest/AndroidLibTest.sln

  offset of local header from start of archive:   45
                                                  (000000000000002Dh) bytes
  file system or operating system of origin:      MS-DOS, OS/2 or NT FAT
  version of encoding software:                   6.3
  minimum file system compatibility required:     MS-DOS, OS/2 or NT FAT
  minimum software version required to extract:   2.0
  compression method:                             deflated
  compression sub-type (deflation):               normal
  file security status:                           not encrypted
  extended local header:                          no
  file last modified on (DOS date/time):          2024 May 28 22:00:02
  32-bit CRC value (hex):                         1c24a670
  compressed size:                                446 bytes
  uncompressed size:                              1154 bytes
  length of filename:                             33 characters
  length of extra field:                          36 bytes
  length of file comment:                         0 characters
  disk number on which file begins:               disk 1
  apparent file type:                             binary
  non-MSDOS external file attributes:             000000 hex
  MS-DOS file attributes (20 hex):                arc 

  The central-directory extra field contains:
  - A subfield with ID 0x000a (PKWARE Win32) and 32 data bytes.  The first
    20 are:   00 00 00 00 01 00 18 00 d9 c2 04 ce e5 b0 da 01 82 20 01 fa.

  There is no file comment.

Note that the 2nd entry is negatively offset by the first entry's size of extra fields/records. It might be that we don't handle copying of extra fields properly.

[Basewq]

@grendello Hi, I think you misunderstood which zip file has a problem.

The AndroidLibTest.zip isn't the problem, that's just me attaching the project to help reproduce the issue - you still need to build/pack the project to investigate the generated .aar file inside the nuget package that has the problem.

Actually, if you're seeing an issue with this zip file, you might have a separate issue since that zip shouldn't have any issues at all.

[dellis1972]

@Basewq this issue report has surfaced a long standing intermitent issue with zip files in our build system. This only seemed to happen on our Azure Pipelines builds and we were never able to reproduce the issue outside of this environment.

We suspect there is something happening. I can see the issue with the .aar and have managed to reproduce something similar locally. We are now investigating, but at this point we have no idea what is causing it.

[Basewq]

@dellis1972 Thanks for the info.

A minor followup to my issue. It turns out while the .aar does appear corrupted to 7z (and Windows), it seems within the xamarin build system it's able to extract out the first file (eg. when I consume the nuget package, its content can be found in the obj folder). I also think the Android app can manage to extract the file, so I think these systems have a bit of error tolerance on extracting the file (or 7z or Windows doesn't properly understand the compression used).

[dellis1972]

@Basewq the good news is we think we have found the issue see dotnet/android-libzipsharp#142.

Not sure which release we will be able to get this into yet though.