ARM support status? #2035
Comments
|
I've used the operator on x86_64 and wanted to see how a deployed the operator in a hybrid arm64/x86_64 cluster would work out as I ported containers. I have to hot-patch the deployments with a nodeSelector. And that sort of works. Where I get hosed is with jobs, because they are immutable. Running on arm64 would be sweet. |
|
It would be beneficial for small environments such as raspberry pi based clusters to support arm64. |
|
I would be willing to take a stab at a PR for this if someone could outline the approach. In particular, I'm thinking I'd just need to add a |
|
Does the underlying https://github.com/zalando/spilo image need to be made arm64 compatible first? |
|
https://github.com/CrunchyData/postgres-operator/blob/v4.7.1/Makefile#L142 That amd64 is a problem there I believe. |
I also have a multi-arch cluster. I probably have 95% of things working truly multi-arch, the rest using NodeSelector as above. In cases like these, where an operator is spawning jobs, and there is no available place in the CRD to add a NodeSelector, I've used the PodNodeSelector admission controller with an annotation on the namespace to limit my postgres-operator namespace, and that of the actual db to an arch of amd64. -- see https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/ |
|
Did you have to manually rebuild stuff? Is what you have open source/present somewhere? |
|
Any update on this one? A couple of years on with arm64 becoming more prominent I just tried spinning up the operator using the HELM chart on ARM64 cluster and the result the dreaded symptoms (see below) of an unsupported architecture. $kubectl logs pgo-6986ddf484-d2pn8
exec /usr/local/bin/postgres-operator: exec format error |
|
In the mean time, would you consider adding a nodeSelector for those architectures that are supported? That would at least help those of us who have mixed architecture clusters. I've been doing this manually for my clusters, but it would be great if it worked out of the box. |
Same here. I don't have a multi-arch cluster, only ARM64 nodes. Gutted to find this operator isn't compatible with my cluster. |
|
We're setting up a multi-arch cluster and are also a bit troubled by the lack of ARM support and lack of built in multi-arch support via Could anyone recommend alternative Postgres Operators which support ARM or multi-arch out of the box that could be considered until this gets resolved? |
|
I'm also still anxiously awaiting ARM64 support. In the meantime though @dbackeus , if you have a multi arch cluster but want to run crunchy only on the amd64 nodes, I've successfully used node affinity selectors in the cluster yaml definition. This works for me because I am not using any of the scheduled jobs/backups , so this at least gets the core cluster up and running on amd64 nodes. I looked at and tested many alternatives, and none were as robust as crunchy was to node failures, etc, so this works for me for now, but I actually have some ARM64 nodes arriving soon that would be faster than my amd64 nodes, so would like this resolved soon as well. |
|
Is there anyone from Crunchy (or just a contributor who knows a bit more than me?) who can advise if there are open work items for this? We are happy to dedicate time for the OSS components that may need work/support to make this work on ARM64. Frankly, the savings we will get in the first month would pay for our labor investment. Happy to direct some resources this way if we can get even a basic point in the right direction! |
|
+1 for this issue. Basically it blocked everyone will ARM node to try postgres-operator, is there a branch we work on for getting an ARM image? |
|
I've built arm images of crunchy before; it's fairly straightforward with the instructions. Perhaps it's something that can be added as a GitHub action ? |
|
+1 Would really appreciate! |
|
Please add upstream support for multiarch Docker images... |
|
Decided to try crunchy products for the first time, was disappointed by the lack of ARM support, which, it seems, has been basically mandatory for all serious software for several years now. Until this support appears, I cannot use this operator, because my cluster is fully arm64 and this won't change |
|
It's too bad that this doesnt work on arm64 |
Can you please tell us procedure you followed |
|
@jelmer , can you guide us how can we build crunchy arm images in local, can we modify the makefile and achieve this. |
It's been a while since I've used the Crunchy operator, but I believe it was just a matter of replacing amd64 with arm64 in a few places in the makefile / overriding those variables from the command line, and then following the regular process for building images as described on https://access.crunchydata.com/documentation/crunchy-postgres-containers/4.5.0/contributing/building/ |
|
+1 |
|
ARM support was introduced in the latest release (5.4.0). See announcement here: https://www.crunchydata.com/blog/announcing-crunchy-postgres-for-kubernetes-5-4 I guess this issue should be closed? |
I don't know what kind of ARM support they added, but at the moment the latest version 5.4 doesn't work on arm64 processor. The good old “executable format error”. It is a pity that such an interesting and convenient project still does not support ARM. Looks like I'll have to rebuild it manually if I want to try it with my arm64 cluster |
|
"Currently ARM images are available for crunchy software subscription customers only and not available as part of our developer program. We may change this in the future, but at this time we're working closely with customers to ensure a quality experience as this architecture before we explore making more broadly available in the developer portal." https://groups.google.com/a/crunchydata.com/g/postgres-operator/c/roplBujSiDU 😳 |
|
This should be a warning to anyone considering using their products in production, paid or otherwise. They explicitly refused "free" (community) labor to accomplish this, and instead did it internally, possibly just by copying the community answers, and are now charging for it.. |
|
As pointed out here in places, Crunchy Postgres for Kubernetes is our commercial product. It is composed not only of the Postgres Operator but also corresponding components and container builds that go through various certification and testing. For many of our customers we have specific commitments in how that software is built and tested. Part of that is being built on specific hardware. Because of our commitment to our customers we could not simply grab a cloud instance and build on it. As a result it took us over a year to procure the needed hardware to build our ARM containers, the demand for ARM hardware is quite high and we worked through multiple vendors before finally having an amazing experience with Ampere. At this time those builds are only available to customers of Crunchy Postgres for Kubernetes (in version 5.4) because we want to work closely with those using them to ensure high quality and a good experience. We are evaluating the prospect of making these ARM builds available within the Developer Portal as part of the Crunchy Data Developer Program, but don't have a set timeline for that yet. PGO community members have commented elsewhere in this thread regarding how it may be possible to build PGO for ARM infrastructure. We have not reviewed / validated those comments, but perhaps they are useful to individuals looking to build PGO ARM infrastructure. Our only request in doing so is that you abide by the Apache license applicable to the source code and project policies. Finally, we have recently updated our documentation - which includes FAQs to address the intersection between PGO (the open source project) and Crunchy Postgres for Kubernetes (the Crunchy Data commercial product): https://access.crunchydata.com/documentation/postgres-operator/latest/faq |
|
This is where community involvement would have assisted Crunchy. I have
spare ARM hardware, I'm sure others do also. There are numerous ways to
build ARM containers without procuring hardware (github actions + qemu).
That being said, if I can't evaluate open source versions of licensable
products that my customers may want, I won't recommend those products to my
customers. At this point, I consider Crunchy's operator a couple years
late, and a few dollars too much.
…--
Curtis Ruck
On Tue, Aug 8, 2023 at 2:04 PM Craig Kerstiens ***@***.***> wrote:
As pointed out here in places, Crunchy Postgres for Kubernetes is our
commercial product. It is composed not only of the Postgres Operator but
also corresponding components and container builds
<https://access.prodtest.crunchyds.com/documentation/postgres-operator/latest/overview>
that go through various certification and testing. For many of our
customers we have specific commitments in how that software is built and
tested. Part of that is being built on specific hardware.
Because of our commitment to our customers we could not simply grab a
cloud instance and build on it. As a result it took us over a year to
procure the needed hardware to build our ARM containers, the demand for ARM
hardware is quite high and we worked through multiple vendors before
finally having an amazing experience with Ampere.
At this time those builds are only available to customers of Crunchy
Postgres for Kubernetes (in version 5.4) because we want to work closely
with those using them to ensure high quality and a good experience. We are
evaluating the prospect of making these ARM builds available within the
Developer Portal as part of the Crunchy Data Developer Program, but don't
have a set timeline for that yet.
PGO community members have commented elsewhere in this thread regarding
how it may be possible to build PGO for ARM infrastructure. We have not
reviewed / validated those comments, but perhaps they are useful to
individuals looking to build PGO ARM infrastructure. Our only request in
doing so is that you abide by the Apache license applicable to the source
code and project policies.
Finally, we have recently updated our documentation - which includes FAQs
to address the intersection between PGO (the open source project) and
Crunchy Postgres for Kubernetes (the Crunchy Data commercial product):
https://access.crunchydata.com/documentation/postgres-operator/latest/faq
—
Reply to this email directly, view it on GitHub
<#2035 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAK4HYO4IDYNPA2IL3ERR23XUJ5SXANCNFSM4TPYV2EQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Not only that, but Docker also supports multiarch builds out-of-the-box, anyone can easily build arm64 image on x86 host That's said, both alternative products (Zalando's Operator and Cloudnative-pg) are supporting arm64 without any kind of paywall Thus, the path is clear, and it does not lead to CrunchyData. I don't consider companies that pretend to be open source, only to close even basic things behind a paywall. |
|
The excuse of "our internal build processes are too complicated to change or explain" is a huge red flag in any environment. Worse is the comment that "red tape" auditing and so forth is too much work to release for free. That strongly hints that their process overhead is so bad it is EXPECTED to regularly prevent things from getting done, and so it makes a convenient scapegoat. |
|
I switched to https://cloudnative-pg.io/ and I am not looking back. ARM64 support is there and in many other respect it is saner way to run Postgres on Kube |
|
Thank you for taking the time to share your thoughts and insights with us. We genuinely value feedback from the community, as it helps us refine our products and services to better meet the needs of users. First, we genuinely appreciate the offer from community members like yourself, willing to assist with spare ARM hardware. We also recognize the vast potential of tools like GitHub actions, as well as Docker's multi-arch builds. As we pointed out before, there are reasons that these options simply do not work for our customers. We understand the concerns about open-source ethos and we believe in its power to drive innovation and collaboration. Our commitment to Crunchy Postgres for Kubernetes, our commercial offering, does not diminish our belief in the open-source community. In line with our commitment to transparency, we will continue to update our documentation, providing clarity on the differentiation between PGO (the open-source project) and Crunchy Postgres for Kubernetes (the commercial product). We apologize if our approach may not resonate with everyone, but we truly value the feedback and the discussion. Our goal is to ensure that our users and customers have the best possible experience, and discussions like these help drive us towards that goal. Thank you for your understanding, and we appreciate your continued feedback. |
|
This whole saga just reminds me weaveworks/weave#3868 (comment) |
|
I mean seriously, why would I want to pay for anything this company makes when they act like this for something so simple to implement. I'll make a note of avoiding any of their products now. |
|
There are those who rollover and create tons of "policy" and "red-tape" excuses to earn the big contracts from the backwards enterprises and those who push back, teach them the error in their way, and refuse to do things the wrong way. Luckily we have choices and can choose to support the open source projects that support modern best practices and dump the ones who are poorly managed and don't care as long as the enterprise contracts still roll in. |
+1 To that @laurivosandi and @disconn3ct. Look, @andrewlecuyer and @craigkerstiens I know you guys have no obligation to give anything away other than the licenses to which you've already committed yourselves. You're under no obligation to give ARM builds away. I don't even expect you to. Hell, there's probably a lot of valid financial reasons not to, but when was the last time that letting the CFO's office run a technology company actually worked for more than a few quarters? Especially one that relies on an OSS base (and the contributors and users of it as their shadow development, test, and advocacy team)? I can't think of many. I am a capitalist. I am the CTO of a for-profit SaaS company who runs a lot of Postgres in product, both self-hosted and managed services, depending on our end offerings (I am also an OSS contributor on a variety of projects). I have nothing wrong with charging for goods & services. But I believe where my "I am a capitalist" ends is that I believe you need to be bringing true value to the products & services you sell if you want to be taken seriously. There is no law or standard that you have to do that, but I hope you can see how the very sterile corporate-speak proffered here is actually just making people like me close our wallets and go elsewhere. I fully understand there is no free lunch when you're running a commercial business @andrewlecuyer and @craigkerstiens. We don't expect a free lunch (nor do we want one: something worth using is worth paying for, especially when we need support and committed reliability). In fact, we looked at Crunchy's commercial offering (superset of PGO) and we looked at EDB (superset of CloudNativePG). We chose EDB/CloudNativePG. I understand your business (and customers?) maybe different, and that's "subjective", but objectively, our assessment showed that CloudNativePG and EDB's product was vastly better and their business processes & build processes much more agile, and we selected (a paid, multi-year) engagement with them. How Crunchy has handled this issue specifically a lot of influence in that decision for me. You guys were cheaper, too. I think the OSS community has gotten wise to this notion of companies who want to create artificial (and relatively low-value) differentiation between paid and OSS offerings, and whether that's what you're doing here, I think you need to know that there are people like me who are more than willing to pay good money for (or that really, somehow you can't reasonably figure out how to safely, securely, and compliantly offer multi-arch builds, let's be honest: this is not a technical problem, it is technically very feasible and easy unless your technology team has made some atrocious decisions). Be genuine and assume your users (and your community) is smarter than you think they are, you can't go wrong with that. Because, if you think that by protecting ARM64 behind a paywall will protect your revenue, you're being shortsighted. We were ready to move our workloads into production running PGO and Crunchy was the shoo-in: we ripped it out and put in CloudNativePG because I just won't do business with vendors who rely on artificial paywalls instead of value-delivered to justify their invoices. I know I am not alone. In fact, I hope more people join me in refusing to support vendors who use the OSS community as the core of their product & brand/community and build a revenue stream off of setting up artificial paywalls around that. |
|
@24601 Wanted to follow up and say thanks for the thoughtful and thorough feedback. I assure you we are listening. As such pulling some details from a response on another similar ticket that I think dives deeper into some of this. After some (considerable) internal discussion on the topic, we have decided to make a few changes to how we approach these things and hopefully clarify our position. Specifically:
The updated and clarified guidance to the Developer Program terms is summarized below, with the full updated terms available here.
The longer answer is we (Crunchy Data) care a lot about the integrity of open source. We develop a considerable amount of open source code, we contribute to and maintain a number of open source projects, our team includes a number of individuals who have spent their careers building open source projects and communities. We care about and value the meaning of 'open source' and the benefits it provides in both the use of open source and the predictability of open source licensing. It is important to us to continue to provide the source code we develop as open source projects on an open source basis and under open source licenses as we have always done. That of course affords users with considerable rights to use the software source - in many instances in ways that are at odds with our commercial interests. That said, we value the integrity of open source, on balance benefit from it greatly and will continue to ensure we do our small part to maintain that integrity. At the same time we are a commercial entity and thus need to balance commercial interests. We have commercial products that we sell to our customers. These commercial products are not 'free'. These products are generally commercial distributions of open source that is either community or Crunchy Data maintained. Our customers value the validation, testing and certification of the software we distribute, the software supply chain security value associated with those practices, and the high quality support from our team of experts. These practices may seem overly 'enterprise' to some and indeed our customers of Crunchy Postgres for Kubernetes tend to skew 'large enterprises', but we have customers across the full spectrum of users who value these commercial offerings. The software we deliver as part of the Developer Program is an important part of these products. These products have a real cost to deliver and we have a number of customers who see the value in paying for them. While open source licensing of source code is a topic we care about, software users - including developers - of course generally are looking for software and not simply source code. Our Crunchy Data Developer Program was initially created to support developers use of our software. Based on feedback from the community and our team it now seems appropriate to update that program to better serve various interests. In balancing the interests of supporting developers use our software and our commercial interests, we do not want to make it prohibitive for developers (or small organizations) to use our software. That said, we do want to make clear we intend for larger enterprise users to pay for use of this software in production, where the software creates real value. While we make these updates to the Developer Program terms, we are also working to improve or communication and support for community users. More to come on that hopefully soon. Creating simple rules to balance important competing interests almost always produces some bad outcomes in corner cases. We are committed to continuing to listen to feedback on these issues and to do our best to learn from that feedback. In the meantime, we hope that this new clarified guidance will provide more transparency and provide a clearer, better outcome for a large spectrum of our software users and communities of interest. |
The community maintenance isn't free either, but it doesn't seem that the corp understands that yet. Please take this as a wake-up call to think much more carefully before treating the community as freeloaders. |
and others
Hello!
AWS Graviton2 instances are becoming more tempting and after testing Postgres on them, we'd like to deploy ARM nodes.
I did found an article on your site regarding ARM testing https://info.crunchydata.com/blog/postgresql-benchmarks-apple-intel-macbook-pro-2011-2019 but no mentions of ARM here.
So, do the currently released images support ARM? I mean the whole suite, not just the operator.
They seem arch-agnostic to me, but confirmation would be better.
The text was updated successfully, but these errors were encountered: