Publish Maven Build To Central #625
Comments
|
@wimjongman Do you or anyone you know have experiencing setting this up? What we would like is to be able to automatically deploy/publish the Runtime to Maven Central. To date, we have been doing this by hand, it would be great if we could do this in the easiest way possible. @SteveSchafer-Innovent |
|
I have tried once and failed... However, it seems doable with some focus. https://docs.github.com/en/actions/guides/publishing-java-packages-with-maven |
|
Thanks Wim,
We will take a look in the upcoming days using those links.
Scott
…On Thu, Apr 29, 2021 at 1:46 PM Wim Jongman ***@***.***> wrote:
I have tried once and failed...
However, it seems doable with some focus.
https://docs.github.com/en/actions/guides/publishing-java-packages-with-maven
https://dzone.com/articles/publish-your-artifacts-to-maven-central
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#625 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKF2BOJ6UWXBORNJMMBOLLTLGSO5ANCNFSM4ZYBQNVA>
.
--
Scott Rosenbaum
Innovent Solutions, Inc.
612 220 6006 cell
|
|
Hello there, We were able to build the BIRT runtime locally and will probably integrate the runtime by using the build jars Can we help you, in order to get a public build release on Maven Central? |
Yes, that would be great. You can create PR's referencing this issue. |
|
What are the release problems? |
|
We could setup a release workflow, where a release is uploaded to Github first We have done it for our project as well: https://github.com/minova-afis/aero.minova.core.application.system/blob/master/.github/workflows/release.yml The only thing needed to my knowledge are a username and an access token for Github, |
|
@splitcells, @pipebaum can we re-start working on this? |
|
@wimjongman It is not known, what is needed or if a Maven Central release is desired. |
|
Hi, I'm not sure what your problem is, but for sure a Maven Central release is desired by us, see #863 |
|
This is what is currently inside the maven repo [1]. Our official build is in Jenkins [2]. [1] https://mvnrepository.com/artifact/org.eclipse.birt |
|
Which one is the release workflow or what are the commands used for release? |
|
The birt-master job creates all the artifacts. The artifacts are then uploaded here [1] We used to have a process to make milestones and release candidates but we are going to skip that for now. Finally, there will be a job that will move the artifacts from the downloads/snapshots and update-site/snapshots to a final release location. Then there could be another job that uploads these artifacts to maven central. I guess the artifacts to upload are the bundles in update-site/snapshots/plugins |
|
You mentioned earlier, that you tried something? That way, we have a common process to start from. |
That was for a different project and a long while ago. |
|
I can take this on and I'd like to publish 4.9 manually to make sure all the requirements are met before automating it. Maybe have automation a goal for 4.10. Do we want to publish just the runtime or everything? |
|
+1 to doing a manual build for now and get the automated build into 8.10
I think that we should just publish the Runtime, if people want to use the
design elements, I think they would use a different path.
Scott
…On Wed, Mar 16, 2022 at 3:36 PM Steve Schafer ***@***.***> wrote:
I can take this on and I'd like to publish 8.9 manually to make sure all
the requirements are met before automating it. Maybe have automation a goal
for 8.10.
Do we want to publish just the runtime or everything?
—
Reply to this email directly, view it on GitHub
<#625 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKF2BN4YYU5TJBPBBGQG7TVAJA4JANCNFSM4ZYBQNVA>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Scott Rosenbaum
Innovent Solutions, Inc.
612 220 6006 cell
|
|
Thanks, Steve! I'm not sure what is meant with the Runtime. This is what is currently on maven: |
|
If that's of any help, this is what I'm interested with: https://mvnrepository.com/artifact/org.eclipse.birt.runtime/viewservlets What I'm looking for is a security fix: #863 |
|
Quite happy to see BIRT is alive again :) We are stuck on 4.4.2 and would love to upgrade 4.9.0 but need maven artifacts in central. BTW, the 4.4.2 runtime artifact loads lots of dependencies, better if most of the data connectors are made separate artifacts. |
|
Thanks, Amit. We are looking forward to your patches ;) |
|
Also quite interested in seeing this published to Maven. Would appreciate both the runtime and also viewservlets. Super excited to see activity in this project again! |
|
birt.war is part of the runtime. |
When I think of runtime I think of what's built into build/birt-packages/birt-runtime/target and also what we could download called "birt runtime". It includes all the classes you need to be able to generate BIRT reports in your own application. It also includes the web app and the war file. I think when people want a maven repo, it's so they can conveniently include the necessary classes in their own app. I'll need to find out if there's a way to build the runtime independently or construct a pom that will do that. That will be the root and then everything it depends on will need to be also published to central. All the pom's will need to meet the maven requirements including <licenses>, <scm>, and PGP signatures. Since this will involved a lot of changes to pom.xml files and 4.9 is now frozen I'll probably need to publish from my fork in order to publish 4.9. Once we get 4.9 successfully published we can make sure 4.10 is ready to go. https://maven.apache.org/repository/guide-central-repository-upload.html When I published 4.8 under the Innovent name I pushed all the jar files to Sonatype OSSRH. There is also a mechanism for running your own repository manager. That might be overkill for us but let me know what you think. |
|
Sounds good Steve. I don't think we need our own repo manager, but you be the judge. I am happy to release 4.10 early, there is no reason to postpone a release when we have some great new stuff to offer. Here is a link to the platform publish wiki: https://wiki.eclipse.org/Platform-releng/Publish_to_Maven_Central |
|
I can publish to the group com.innoventsolutions, but in order to publish to org.eclipse or any sub-groups, I would need the sonatype jira credentials of whoever owns org.eclipse. |
|
It would be quite better if it was under org.eclipse. |
What commands are you using? Sry, for the noobish question. |
|
@splitcells, although I have published to central before I haven't published using maven so I'm just as much a noob as you are in that area, but I can say that any modifications to the pom files will be part of the repo. My comments about group ownership came from reading https://central.sonatype.org/publish/requirements/coordinates/#choose-your-coordinates. |
|
I recently had to work on the integration of an updated BiRT dependency as the solution of @siepkes broken some of my transitive dependencies for logging (no offense, at least you shared something that was extremely valuable). You'll find it below. https://gist.github.com/Marks13/94fd7de500297e9c6b9830a5f167e5c1 |
|
Can this issue be resolved anytime sooner? |
|
@m-thirumal Yes, if you provide a solution to it. If you have read this issue in detail, you should have seen that there is no easy solution. You should not expect a solution soon. A solution seems to depend on understanding and cleaning up the build process, besides others, AFAIK. The build process is "historical grown", meaning that is a complicated mess. |
It no longer is thanks to Ed @merks 💞 |
|
There have been some discussions with folks potentially willing to provide sponsorship to align with their wishes. If wishes were fishes… |
We have added adaptation of BIRT for maven project. For our aims it was quite enough. May be it will helpful for your projects too |
|
I periodically check this issue and noticed that the milestones were modified: does it means that 4.17 is going to be in maven central? |
|
If you work on it, then yes. |
|
I understand. Thank you! |
|
Looks like @SteveSchafer-Innovent got this partially working back in 2022. I see that 4.9.0 was published (manually?) to maven central. It's not very clearly outlined in this issue what is the work remaining to get this automation in place for each release. |
|
@wimjongman please see previous comment |
|
I'm just watching this silently and I don't know anything about it, but AFAIK ATM @merks is working hard to simplify the build process. I believe that the Maven build has been very complex manual work (and that nobody is able to do this) because the build process is a mess. As soon as Ed has simplified and cleaned up the build process to a point when at least it can be comprehensed by experienced Java/Eclipse developers with some good will, maybe someone can tackle this Maven issue, but before that, it hardly makes sense. And BTW "pinging" issues does not help anyone IMHO. We are well aware that the lacking availability of BIRT in the Maven repository is a major issue. If you want to speed up development, maybe you can ask Ed how you can help? |
|
If you read my comment, I was trying to get an idea of what is the work remaining. Thank you for your explanation.
How is this done other than asking him here? |
|
Maybe I'm the one to blame for "pinging"? I did in the context of a milestone change in the issue: with the intention of understanding if this milestone change means that 4.17 was expected to be in maven central, or if it was just issues/milestones maintenance... My apologies if that was considered unhelpful or rude; that was not the intention. |
|
Please go ahead and keep pinging the issue. However, we require someone from the community with good experience in this matter to help here.
@hvbtup Sorry I just read this :) A bit of mixed advice from our side... ;) |
|
Supporting BIRT's release engineering is already a lot of work and cleaning up the existing mess is a daunting task: No one funds this effort, the on-going releng effort, nor even the bug fixes and helpful discussion answers. I do the releng work primarily to help the committers of the project. I really admire our committers! Adding publishing to Maven Central to this workload literally just creates yet more work for me. I have other work for which I'm actually compensated, e.g., publishing the Platform to Maven Central, which is a prerequisite to BIRT doing so. As you might imagine, the burden of yet more free work is uncompelling at best. I think it's really time for all the folks who benefit from all this free goodness, including all the free goodness provided by the folks improving the BIRT implementation, fixing the bugs, and answering the question to step up and support the effort with more than just wishes and emojis. I expect the companies you work for collectively make more money in a day than I make in a year. So get involved in the IDE WG and then we talk about priorities: https://eclipseide.org/membership/ No one there currently considers BIRT a priority... Attached is an outline that I wrote to describe what I believe needs to be done and how it should be done: Consider that it's possible sponsor your wishes and that this will prove much more effective than pings and emojis. |
|
Scott's original issue asked if we could publish the BIRT (non-osgi) runtime. That's all I think most people care about. The runtime basically consists of a fat jar and a lot of other jars. The easy and ugly way to do it is to publish all the jars as new artifacts and make the fat jar dependent on all the other jars. That's what I did for the innovent 4.8 runtime. People can then include the fat jar as a dependency on their pom to get a working BIRT runtime in their project. Yes most of those jars are redundant to artifacts already in maven, but there's a lot of redundancy in maven anyway, so I'm not sure that's so bad. Recently I started to think about how to eliminate those redundancies and I came up with the idea of using the maven REST API to search maven in a kind-of automated way. Unfortunately, it's sometimes hard to find an existing artifact on maven even if it already exists, if all you have to go in is the name of a jar file which doesn't really match the maven coordinates. So I was able to write a script that found a lot of the artifacts using simple logic but then I had to add a lot of exceptions and was ultimately able to locate maybe 90% of the artifacts. Then I had to resolve any doubt that what I found in maven was REALLY the same as my jar file. In the end it turned out to be a lot of work and would take a lot of maintenance, so probably isn't worth doing. I'm sure Ed Merk's approach is better but it requires a deeper understanding of P2 and Orbit than I currently have (although learning new things is always good). Maybe a hybrid approach would work. I have a few weeks during which I can spend time on this before I get busy with other stuff and would be happy to do some research/learning. @merks, can you direct me to where I can learn more about "CBI p2 Aggregator" which you mentioned in your PDF? |
I think you are right. This is all that's needed to execute BIRT reports and that's what probably > 99% of people need. |
|
Some remarks on @merks analysis in BIRT-Details PDF:
+1
+1
Not sure.
+1 for the idea of moving this into BIRT's coordinate space.
Probably most users of BIRT only use Apache Derby for the builtin demo database. Regarding the proposals:
+1 Regarding the technical details about how to achieve these goals:
+1
From my POV: yes. |
|
I'm not sure we can say definitely what most people care about. Certainly no one cares enough to do the work themselves nor to sponsor the work being done. As they say, what goes around, comes around. (I see very little coming around). In any case, if folks are happy with a dump, why not just dump the one entire zip and call it done?
If folks are happy to have an exploded repackaged version of birt-runtime*.zip, that can be done external to the build, repackaging the build results. There are currently 141 additional jars. It feels like an abomination to republish those, but that's just my personal feeling. Moreover, in our increasingly security-conscious world, I expect that folks will want to know exactly where the artifacts come from and what exactly is in them (to be sure that nothing untoward and been stuffed into them while migrating from one coordinate to another). So I fully expect the results of republishing to be a major hassle for consumers, but that's just my personal expectation. Also note that what is in the runtime is really poorly defined. These 141 libraries effectively include the transitive dependencies of the Eclipse workbench. That can't really be necessary can it? That really ought to be cleaned up before we dump things to Maven Central. The cleanup tasks for BIRT appears to be never ending. 😱 Note that almost all the current Orbit dependencies have links to their Maven origins as generated on this summary page: https://download.eclipse.org/tools/orbit/simrel/orbit-aggregation/table.html I'm trying to get WTP to use purely artifacts from the above repository, but that's proving a bit of a challenge. Unfortunately BIRT depends on WTP (though the runtime does not) so some of the older artifacts leak in through WTP. (I do have PRs to fix all those issues in WTP). Note that Orbit has a bunch of Maven crawlers and analyzers that are used to keep Orbit up-to-date as new dependencies are published to Maven Central. Like for BIRT, there's an Oomph setup for that if you want to have a look at that: https://github.com/eclipse-orbit/orbit-simrel/blob/main/CONTRIBUTING.md Finding things on Maven Central is a challenge at best. Maven Central is a midden heap in which the crown jewels are stored. It's especially challenging to find artifacts based on the Java packages they provide. That was a big problem for me when cleaning up the mess that was Orbit. So I prototyped an index/crawler that lets me do test search like this.
Unfortunately Apache has a habit of publishing garbage artifacts, even artifacts without sources. So for Derby, it's a significantly challenge and workload to even produce a functional bundle (in my copious free spare time). Basically I'll need to do get the original build results and repackage them: https://dlcdn.apache.org/db/derby/db-derby-10.16.1.1/ And then I'm not even sure how to test that what I produce is fully functional. 😢 Here is the repository where the CBI Aggregator is maintained (by me): https://github.com/eclipse-cbi/p2repo-aggregator Documentation links are there. Some of the newer features of the aggregator (e.g., dependency mapping of Java package dependencies, import to Maven publishing) are not yet documented. So much to do... All the Platform's infrastructure for publishing to Maven Central is in this folder: The jobs driven by these definitions and scripts are here: https://ci.eclipse.org/releng/view/Publish%20to%20Maven/ One aspect of publishing to Maven central, at least via the Nexus servers we use from Eclipse is that one must include a source jar and a javadoc jar for every jar published. A significant amount of the ugly scripting in the above is associated with this gotcha. It's also all too easy to publish results with broken dependencies, so the Platform's processes carefully test that the Maven dependencies all actually resolve. Note that the Platform does not republish any 3rd party libraries. The DataTools project is in maintenance purely by my personal efforts, as noted above. It's also used by EMF (ODA) and by Web Tools Project. The following are my personal, politically-incorrect opinions. I think it's great that our awesome committers provide the community with regular quality releases, helpful answers, and so on. That helps to set high expectations. The community's wish and hope for publishing to Maven Central is completely understandable. But I can assure everyone that for a project has complex, and with as many dependencies, as BIRT, this is an ocean boiling exercise that is even for me, with decades of experience with this entire technology stack and with a decade of experience with the build infrastructure, a daunting task. But hey, if someone wants to do the work, doesn't consume an unbounded amount of my personal time to help with that work, and doesn't permanently increase my maintenance workload going forward, then that's all find and good. Just be warned that this is an effort where a month of work is likely to be insufficient. Failing that effort, what the community cares about should be reflected in community's actions and therein lies the biggest shortcoming. What goes around, comes around. |
|
A summary note to myself and perhaps for everyone about the current state of this case:
|
and others
Currently, there is no automated feature to publish the BIRT-Runtime to Maven. Wouldn't it be nice to fix this.
The text was updated successfully, but these errors were encountered: