Reduce binary size

[mappu]

The ideas from #8 and #9 can also help to reduce binary size.

Ideas:

• Use -ffunction-sections -fdata-sections in CFLAGS, and -Wl,--gc-sections in LDFLAGS to allow stronger dead code elimination across language boundaries
  • Proposal: Improve dead code ilimination when using CGO golang/go#38135
  • This causes each function to have segment alignment. If a large proportion of the Miqt C ABI functions are used, all the extra padding in the binary could create a net negative effect. Only section alignment, not segment alignment.
• Attempt to build with gccgo
  • Some reports online that this can produce a significant size reduction (although with adding external dynamic dependencies)
• Refactor marshaling code to call helper functions instead of inlining it in every function wrapper
• Ability to disable unwanted Qt classes at compile time?
  • Maybe with a build tag
  • Similar idea to qtminimal
• Ability to disable subclassing support for unneeded classes?
  • The comment in Support subclassing #42 (comment) shows that subclassing support did increase the binary size noticably

[ddkwork]

It's 17-21mb after removing cgo, compilation speed and binary size are very impressive, but I'm not very familiar with cpp classes and templates. I will continue to remove cgo when I have enough experience, I tried to manually fix the c abi file generated by miqt and compiled it to dynamic dll, it was 17mb, the outer code of the go project was about 2-3mb, it added up to about 20mb, on par with flutter.

However, the main repository needs to carry a lot of c abi dynamic libraries resulting in a larger repository, this is another success story

https://github.com/richardwilkes/unison/tree/main/internal%2Fskia

[arnetheduck]

FWIW in the nim bindings, -flto removes almost all overhead since it ends up including only the wrappers that actually get used - this approach of course depends on the underlying C++ compiler - in a way, it's similar to sections but more powerful.

Another idea might be to follow the upstream modules more closely - ie miqt lumps together QtCore/widgets/etc which might not be necessary for all applications - recycling the upstream qt module organisation has other benefits too since the bindings will be more in tune with available documentation etc.

[mappu]

I recall trying -flto, but the Go compiler doesn't do anything with it, so it had no effect overall. It would probably work very well with gccgo or gollvm/llgo. I'll try that, it could be a recommendation for people to make smaller release builds with a different compiler.

There is probably only limited use cases for using a language binding of Qt Core without gui/widgets. Almost everything in Core has some sort of native equivalent in Go anyway. I can maybe imagine extending Miqt to bind some custom library that uses Core types without gui/widgets, but that's hypothetical at this stage. (I'm not sure if Qml or Quick depend on Gui.)

There are also some difficult circular dependencies in the Qt headers between package types - at least QtMultimedia and QtMultimediaWidgets, etc.

[mappu]

• gccgo does not yet implement go 1.18 language features
• TinyGo is an independent LLVM-based Go compiler, that could be capable of dead code elimination across Go/C/C++ boundaries. So far, it is missing some CGO features cgo: invalid #cgo line: CXXFLAGS, pkg-config tinygo-org/tinygo#4765

[foresto]

The helloworld6 example yields a 118 MiB binary when compiled with go build -gccgoflags "-s -w" on Debian Bookworm. I'm using gccgo, so it's a dynamically linked binary.

Is this normal? I think I was expecting closer to 10 MiB.

$ apt list gccgo-go libqt6core6 gccgo-go
gccgo-go/stable,now 2:1.19~1 amd64 [installed]
libqt6core6/stable,now 6.4.2+dfsg-10 amd64 [installed,auto-removable]

[mappu]

@foresto For helloworld6 on master with golang-go, the unstripped size was 98 MiB. With -ldflags "-s -w" it reduced to 33 MiB; then with upx on default settings it became 7 MiB, with upx --lzma it became 4.8 MiB.

Your 118 MiB is close to the unstripped size from golang-go so I think the -gccgoflags "-s -w" is not taking effect.

Actually I'm happy to see gccgo working, i'll try to see if other flags can help.

EDIT:

A default build with gccgo took 17m and produced a 379MiB binary. 😱 Building with -gccgoflags "-s -w" reduced the size to 118MiB as you say.

I think this is a limitation in gccgo. I also tried -Os, -ffunction-sections -fdata-sections, -Wl,--gc-sections with no further success.

[foresto]

Thanks; I got results similar to yours with golang-go. I'll plan to use that instead of gccgo, at least until the latter improves in this area.

I should clarify that the RAM footprint is probably more important than the binary file size (and is equally large in my helloworld tests). UPX wouldn't help with this; in fact, I think it would make things worse, because:

• The whole image must be decompressed into RAM, not paged in as needed.
• The decompressed image cannot be shared by multiple instances.
• It has a habit of getting binaries flagged by malware scanners.
• It leads to slower launch speed.

I wonder why both compilers produce such large binaries when they're all dynamically linked to the Qt libs, and why gccgo is so absurdly bad.

Does MIQT use reflection features? I have read suggestions that some of them can interfere with Go's dead code elimination. (Maybe visible here.)

[mappu]

I tried gccgo with -flto but also wasn't able to achieve any smaller size (maybe I used it incorrectly).

There's no Go reflection used in Miqt. But, I do agree this is almost certainly a problem of insufficient dead code elimination.

[arnetheduck]

I tried gccgo with -flto but also wasn't able to achieve any smaller size (maybe I used it incorrectly).

this would rely on the go compiler emitting lto objects as well, which perhaps it doesn't.

one more thing that's been on my mind is __attribute__ ((visibility ("hidden"))) - this would require some tweaks (for shared or not) - this would help "normal" c applications with DCE at least.