-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Build GTK3 support for linux x86-64 to maximize compatibility #1795
Conversation
@laeubi and/or @HannesWell How to test this? Can I push to a branch of SWT so I can run builds on https://ci.eclipse.org/releng/job/eclipse.platform.swt/? If so, any objections to me merging eclipse-platform/eclipse.platform.releng.aggregator#2802 now so that I can start to test this? Also, assuming that is the way to test, can you check out if always push new natives is ok? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, assuming that is the way to test, can you check out if always push new natives is ok?
In general the policy is that all SWT native binaries have to be build on project infrastructure and to ensure this we usually don't accept changes in the binaries directly from PRs but just receive the code changes and only rebuild on master what is eventually added to the git/p2-repo.
In order to test the newly built binaries you can just download the built artifacts from Jenkins. E.g. for the second build:
https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1795/2/artifact/eclipse.platform.swt/binaries/org.eclipse.swt.gtk.linux.x86_64/target/
I've just created a PR to clean up the archived artifacts a bit: #1799
In the past there have been discussions about how to stream-line this process, especially as the required build-resources are currently rare (I hope this changes in the future: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4231).
But I think if this is changed, it should be a separate, dedicated PR.
For all the other changes I need a bit more time to review. I intend to do it tomorrow.
I will remove that commit then and apply your ternary change. |
@jonahgraham If you change any of the C files or build scripts a little bit (e.g. add a temporary comment) then the "build natives" workflow should kick in and build things with your changed containers. What I have done for GTK4 back then was that I first checked the build is running, and then look at the build output (e.g. did the GTK4 binaries produced) and finally that all the so files are attached as desired. If that works (and the tests not go completely wild) there is a good chance that the change is good to be submited and a committer can then run a build with forced binary generation. |
Alternatively you can just start a build via With #1799 being submitted the build native fragments are now prominently displayed as the only Build artifacts in the corresponding Jenkins job and can be downloaded and further inspected and tests. The current build is waiting for the Y-build tests to complete and will therefore need some more time, but the previous build failed: For example for the master you can see it already: I have to leave now for a few hours but, when back I'll review this in detail and make a few more suggestions for enhancements. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Below I have made a few suggestions for making this change a bit more compact and one required change to fix the pipeline failure.
With that applied the build succeeds:
https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1795/7/
You can obtain the newly built native jars from that runs Build-artifacts
section on the overview page or just visit it directly at:
https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1795/7/artifact/eclipse.platform.swt/binaries/
Is the "that" in this comment that you changed the Jenkinsfile via "Replay" feauture? Because AFAICT run 6 failed and run 7 succeeded, but they were both built from same SHA1 but there is a diff in the replay https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1795/7/replay/diff Not a feature I had been familiar with until now. Thanks for the demo. |
Yes but this only works for committers and especially when testing different approaches it is a bit inconvenient, but of course many things work (including replay). |
Is this still in the draft state? Would be great to have it before M3. |
Yes "that" was meant to be the changes suggested/described before. Should have been more clear.
Yes that's right. I just wanted to show an alternative that IMO suitable if one just wants try a change only ones or twice.
@jonahgraham said in #1795 (comment) that he want's to enhance this a bit further but in general this looks good to me and I assume we can land this very soon (after the commits have been squashed). One can already test the built native fragments from: |
I hope it is completely ready for review very soon, well before M3. I have just pushed a squashed version that also includes the checking of dependencies. Assuming it builds correctly I will mark as ready for review soon. I did some restructuring compared to the previous commit. The force-pushed compare link renders those changes well. The updated commit message is: Build GTK3 support for linux x86-64 to maximize compatibility Recently all SWT builds for Linux x86-64 have been done on a new Debian image. This leads to requiring such a new distro to be able to run SWT successfully. This change builds GTK3 and common code on an older debian container to increase the range of versions we support. Only GTK4's key library, libswt-pi4-gtk*.so is shipped from a build on the new debian container. We continue to build all of the natives on latest debian so we benefit from compiler warnings/errors and other such tooling. As part of the build all .so files are checked to ensure their dependencies are as expected. In particular glibc version is checked for all symbols and all libraries are checked. The version of glibc 2.14 was chosen as that is what the effective requirement has been for a while for SWT (at least since Eclipse 4.26/2022-12) Fixes #1791 |
I have verified that the build contains the correct info and I open this up as Ready for review. gtk3 + gtk4 build outputKey parts extracted from https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-1795/14/pipeline-console/ gtk4
gtk3
As you can see from the above build output
To verify that the produced build worked as expected this is what I did on an Xubuntu 24.04 machine
The This is the full command I used in step 7 + 14:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the update, the change looks good to me.
I just have a remark about a small readability improvement that gave me hard time to understand the change. Please see below.
I have also verified that all so-files in the linux.x86_64
fragment except for libswt-pi4-gtk-*.so
only reference GLIBC_ 2.14 or below.
@jonahgraham I have just pushed a commit that includes the suggested change to your branch. If you are fine with it, please squash it into your commit and I would perfectly fine with this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just have a remark about a small readability improvement that gave me hard time to understand the change. Please see below.
@jonahgraham I have just pushed a commit that includes the suggested change to your branch. If you are fine with it, please squash it into your commit and I would perfectly fine with this.
I like the change - I'll squash and push momentarily.
Recently all SWT builds for Linux x86-64 have been done on a new Debian image. This leads to requiring such a new distro to be able to run SWT successfully. This change builds GTK3 and common code on an older debian container to increase the range of versions we support. Only GTK4's key library, libswt-pi4-gtk*.so is shipped from a build on the new debian container. We continue to build all of the natives on latest debian so we benefit from compiler warnings/errors and other such tooling. As part of the build all .so files are checked to ensure their dependencies are as expected. In particular glibc version is checked for all symbols and all libraries are checked. The version of glibc 2.14 was chosen as that is what the effective requirement has been for a while for SWT (at least since Eclipse 4.26/2022-12) Fixes eclipse-platform#1791
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the change - I'll squash and push momentarily.
Great and thank you for this change!
The master build successfully rebuilt the native libraries and tonights I-build will contain it. |
Just in time for my Friday's weekly product build on latest SDK nightly that will run on RHEL7 & 9. |
Thanks for all the focus on this issue. Glad it got done for this release! |
🥳 The I-build worksI tested this change + the corresponding change eclipse-equinox/equinox#834 with I20250206-1800. The test suites have not completed running yet. While the SWT unit tests passed in this PR, please help me identify if anything looks like it regressed in the integration build. Prep stepsCompleted on my Xubuntu 24.04 machine:
On that Eclipse, I ran each of the following commands and verified that Eclipse started as expected: GTK3 runsRuns on Xubuntu 24.04:
Runs on Debian 11 - these are the commands that were failing without this fix:
|
I fear there was a problem with the build of the binaries / lfs. I can't pull from master locally, I see git lfs errors like
I also don't see build results for anything but Linux on https://download.eclipse.org/eclipse/downloads/drops4/I20250206-1800/ |
OK, that is strange. I was able to pull from master from RH9 but not from RH7. After pulling / pushing to local RH9 repo I was able to pull from local on RH7. |
Awesome. I can also test it next Wednesday on a Linux workstation.
They are still running: https://ci.eclipse.org/releng/job/AutomatedTests/ |
On RH7 I have git version 2.36.5, on RH9 2.39.2.3, so they seem to have fixed something in git... I was able to build platform on RH7 now & start IDE. Test results follow, but I believe we should be fine regarding older GLIBC now. Many thanks Jonah, Hannes & everyone involved! |
I can confirm the latest I-Build now runs on my RHEL 8 development environment. Thanks! |
Build GTK3 support for linux x86-64 to maximize compatibility
Recently all SWT builds for Linux x86-64 have been done on a new Debian image. This leads to requiring such a new distro to be able to run SWT successfully.
This change builds GTK3 and common code on an older debian container to increase the range of versions we support. Only GTK4's key library, libswt-pi4-gtk*.so is shipped from a build on the new debian container.
We continue to build all of the natives on latest debian so we benefit from compiler warnings/errors and other such tooling.
As part of the build all .so files are checked to ensure their dependencies are as expected. In particular glibc version is checked for all symbols and all libraries are checked.
The version of glibc 2.14 was chosen as that is what the effective requirement has been for a while for SWT (at least since Eclipse 4.26/2022-12)
Fixes #1791
The way I have tested this concept locally is to run the build.sh twice on my local machine using the two docker containers mentioned in runOnNativeBuildAgent of the Jenkinsfile*:
cd /scratch/eclipse/oomph/swt-master/git/eclipse.platform.swt/
java -Dws=gtk build-scripts/CollectSources.java -nativeSources 'target/natives-build-temp/gtk'
cd 'target/natives-build-temp/gtk'
docker run --rm -it -v/scratch:/scratch -w$PWD -e SWT_JAVA_HOME=/scratch/java/jdk-17.0.13+11 -e OUTPUT_DIR=/scratch/eclipse/oomph/swt-master/git/eclipse.platform.swt/binaries/org.eclipse.swt.gtk.linux.x86_64 swtnativebuild ./build.sh -gtk4 install
docker run --rm -it -v/scratch:/scratch -w$PWD -e SWT_JAVA_HOME=/scratch/java/jdk-17.0.13+11 -e OUTPUT_DIR=/scratch/eclipse/oomph/swt-master/git/eclipse.platform.swt/binaries/org.eclipse.swt.gtk.linux.x86_64 swtgtk3nativebuild ./build.sh -gtk3 install
* Note that I actually built them manually with USER 1000 at the end of the file so I can share directories with the container. There are more details in the step-by-step in eclipse-platform/eclipse.platform.releng.aggregator#2802 (comment)