[Release Tooling] Add functionality to build dynamic frameworks

[ncooke3]

#no-changelog

[ncooke3]

I manually verified by diffing the directory structure that the refactored code paths for static xcframeworks produces unchanged artifacts.

[ncooke3]

1h 21m 35s for the dynamic linking job. Huge speedup!

[ncooke3]

Dynamic Firebase.zip has the same directory structure of the static Firebase.zip (which makes sense since the resources bundles are moved within the static frameworks)

[ncooke3]

Promising results with the zip looking good and ABTesting x dynamic linking test looking good as well. Will now try to test against all QS.

[ncooke3]

Based on #12893, there was one fix needed for the check_firestore_symbols job that I made in c2e84d2 (we don't need to run it on the dynamically linked artifact yet). All of the QS tests passed which was great.

[paulb777]

Looks great! Would you see if you can find why the dynamic build is significantly smaller? Is is consistent across all the libraries or just a few particular ones?

[ncooke3]

The savings are spread out among the top-level directories.

I did some spot checking with comparing the executables (.a vs .dylib) using nm -g.

Example:

diff <(nm -g Firebase-dynamic/FirebaseAnalytics/FirebaseCoreInternal.xcframework/ios-arm64/FirebaseCoreInternal.framework/FirebaseCoreInternal | sort) <(nm -g Firebase-static/FirebaseAnalytics/FirebaseCoreInternal.xcframework/ios-arm64/FirebaseCoreInternal.framework/FirebaseCoreInternal | sort)

The output is pretty hard to parse, but I did look for patterns of symbols being in both outputs.

Size calculated using du -sh /path/to/dir

Directory	Dynamic Size	Static Size	Notes
FirebaseABTesting	1.8M	1.5M
FirebaseAnalytics	62M	68M
FirebaseAnalyticsOnDeviceConversion	8.6M	8.6M	Uses same build for both.
FirebaseAppCheck	7.5M	10M
FirebaseAppDistribution	756K	824K
FirebaseAuth	19M	35M
FirebaseCrashlytics	33M	49M
FirebaseDatabase	28M	44M
FirebaseDynamicLinks	1.2M	1.8M
FirebaseFirestore	421M	665M
FirebaseFunctions	27M	29M
FirebaseInAppMessaging	5.4M	7.6M
FirebaseMLModelDownloader	138M	183M
FirebaseMessaging	11M	20M
FirebasePerformance	44M	61M
FirebaseRemoteConfig	21M	26M
FirebaseStorage	26M	33M
Google-Mobile-Ads-SDK	26M	26M	Uses same build for both.
GoogleSignIn	13M	17M

[ncooke3]

@andrewheard, can you think of anything in particular to look for?

[andrewheard]

@andrewheard, can you think of anything in particular to look for?

Unfortunately I'm not sure of anything in particular to look for in addition to what you did with nm.

[andrewheard]

This LGTM. Since the static builds remain the default, this gives us a chance to test out the dynamic zip locally without affecting the next release. (I'm not expecting anything wrong since the quickstarts worked.)

[ncooke3]

Sharing a script I used to measure sizes, and readings of the Firestore directory. Notice how interop/extension SDKs take up more for the dynamic case.

➜  Downloads cat ./script.sh                                
#!/bin/bash

# Loop through each sub-directory excluding files and hidden directories
for dir in $1/*; do
  # Check if it's a directory (not a file or hidden)
  if [[ -d "$dir" && ! -h "$dir" ]]; then
    # Get the size of the directory using du with human-readable format (-h) and silent mode (-s)
    size=$(du -sh "$dir" | cut -f 1)
    # Print the directory name and size
    echo "$dir: $size"
  fi
done
➜  Downloads ./script.sh Firebase-dynamic/FirebaseFirestore/
Firebase-dynamic/FirebaseFirestore//FirebaseAppCheckInterop.xcframework: 1.3M
Firebase-dynamic/FirebaseFirestore//FirebaseCoreExtension.xcframework: 1.6M
Firebase-dynamic/FirebaseFirestore//FirebaseFirestore.xcframework: 8.5M
Firebase-dynamic/FirebaseFirestore//FirebaseFirestoreInternal.xcframework:  44M
Firebase-dynamic/FirebaseFirestore//FirebaseSharedSwift.xcframework:  10M
Firebase-dynamic/FirebaseFirestore//absl.xcframework:  29M
Firebase-dynamic/FirebaseFirestore//grpc.xcframework: 183M
Firebase-dynamic/FirebaseFirestore//grpcpp.xcframework:  98M
Firebase-dynamic/FirebaseFirestore//leveldb.xcframework: 4.4M
Firebase-dynamic/FirebaseFirestore//openssl_grpc.xcframework:  41M
➜  Downloads ./script.sh Firebase-static/FirebaseFirestore/ 
Firebase-static/FirebaseFirestore//FirebaseAppCheckInterop.xcframework: 312K
Firebase-static/FirebaseFirestore//FirebaseCoreExtension.xcframework: 664K
Firebase-static/FirebaseFirestore//FirebaseFirestore.xcframework:  11M
Firebase-static/FirebaseFirestore//FirebaseFirestoreInternal.xcframework: 112M
Firebase-static/FirebaseFirestore//FirebaseSharedSwift.xcframework:  12M
Firebase-static/FirebaseFirestore//absl.xcframework:  37M
Firebase-static/FirebaseFirestore//grpc.xcframework: 329M
Firebase-static/FirebaseFirestore//grpcpp.xcframework: 103M
Firebase-static/FirebaseFirestore//leveldb.xcframework: 6.8M
Firebase-static/FirebaseFirestore//openssl_grpc.xcframework:  53M

[paulb777]

Thanks. For static to dynamic increases, grpc and FirebaseFirestoreInternal are huge. Would be interesting to break those down to the individual .o files.