Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Dev Services for PostgreSQL database with images from Red Hat registry #44196

Open
michalvavrik opened this issue Oct 30, 2024 · 18 comments
Labels
area/devservices area/jdbc Issues related to the JDBC extensions area/kubernetes kind/enhancement New feature or request

Comments

@michalvavrik
Copy link
Member

michalvavrik commented Oct 30, 2024

Description

I would like to develop my Quarkus application with database image I actually use later. I am using registry.redhat.io/rhel9/postgresql-16:latest, but when I set quarkus.datasource.devservices.image-name=registry.redhat.io/rhel9/postgresql-16:latest then database startup is never detected. This image is available after login and safe, I'd like to use it because it aligns with my OpenShift deployment where I use it as well. It seems that this image is also available in GH CI, therefore Quarkus should be able to test it with community setup only.

quarkus create app --extensions=jdbc-postgresql,hibernate-orm
cd code-with-quarkus/

# PASS
quarkus dev

# FAIL
quarkus dev -Dquarkus.datasource.devservices.image-name=registry.redhat.io/rhel9/postgresql-16:latest

Implementation ideas

No response

@michalvavrik michalvavrik added area/hibernate-orm Hibernate ORM area/jdbc Issues related to the JDBC extensions kind/enhancement New feature or request labels Oct 30, 2024
Copy link

quarkus-bot bot commented Oct 30, 2024

/cc @geoand (devservices,openshift), @iocanel (openshift), @stuartwdouglas (devservices)

@geoand
Copy link
Contributor

geoand commented Oct 30, 2024

cc @yrodiere

@gsmet
Copy link
Member

gsmet commented Oct 30, 2024

This image is publicly available and safe, I'd like to use it because it aligns with my OpenShift deployment where I use it as well.

I don't know, I get:

com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Head "https://registry.redhat.io/v2/rhel9/postgresql-16/manifests/latest\": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication"}

Could you paste the log you obtain when starting this container?

@michalvavrik
Copy link
Member Author

This image is publicly available and safe, I'd like to use it because it aligns with my OpenShift deployment where I use it as well.

I don't know, I get:

com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Head "[https://registry.redhat.io/v2/rhel9/postgresql-16/manifests/latest](https://registry.redhat.io/v2/rhel9/postgresql-16/manifests/latest%5C)": unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication"}

Could you paste the log you obtain when starting this container?

docker image rm registry.redhat.io/rhel9/postgresql-16:latest
docker image pull registry.redhat.io/rhel9/postgresql-16:latest

outside of VPN. Also you can check this GH CI log (search for redhat), because I assure you it has no credentials etc.: https://github.com/quarkus-qe/quarkus-test-suite/actions/runs/11498059256/job/32003343241?pr=2129

@michalvavrik
Copy link
Member Author

Could you paste the log you obtain when starting this container?

well I gave you steps to reproduce it, but I'll run it again report back, moment

@michalvavrik
Copy link
Member Author

here you go:

2024-10-30 19:09:07,135 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor#launchDatabases threw an exception: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image registry.redhat.io/rhel9/postgresql-16:latest
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:367)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:121)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image registry.redhat.io/rhel9/postgresql-16:latest
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:359)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330)
	at io.quarkus.devservices.postgresql.deployment.PostgresqlDevServicesProcessor$1.startDatabase(PostgresqlDevServicesProcessor.java:80)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:293)
	... 10 more
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344)
	... 13 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:563)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 14 more
Caused by: java.lang.IllegalStateException: Wait strategy failed. Container exited with code 2
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:533)
	... 16 more
Caused by: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32790] should be listening)
	at org.rnorth.ducttape.timeouts.Timeouts.callFuture(Timeouts.java:68)
	at org.rnorth.ducttape.timeouts.Timeouts.doWithTimeout(Timeouts.java:60)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilReady(WaitAllStrategy.java:54)
	at org.testcontainers.containers.PostgreSQLContainer.waitUntilContainerStarted(PostgreSQLContainer.java:149)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500)
	... 16 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32790] should be listening)
	at org.testcontainers.containers.wait.strategy.HostPortWaitStrategy.waitUntilReady(HostPortWaitStrategy.java:112)
	at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilNestedStrategiesAreReady(WaitAllStrategy.java:66)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.lambda$waitUntilReady$0(WaitAllStrategy.java:58)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:354)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:272)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:62)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:91)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:430)
	at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:57)
	at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:138)
	at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:93)
	at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:131)
	at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:62)
Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor#launchDatabases threw an exception: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image registry.redhat.io/rhel9/postgresql-16:latest
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:367)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:121)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image registry.redhat.io/rhel9/postgresql-16:latest
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:359)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330)
	at io.quarkus.devservices.postgresql.deployment.PostgresqlDevServicesProcessor$1.startDatabase(PostgresqlDevServicesProcessor.java:80)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:293)
	... 10 more
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344)
	... 13 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:563)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354)

	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 14 more
Caused by: java.lang.IllegalStateException: Wait strategy failed. Container exited with code 2
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:533)
	... 16 more
Caused by: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32790] should be listening)
	at org.rnorth.ducttape.timeouts.Timeouts.callFuture(Timeouts.java:68)
	at org.rnorth.ducttape.timeouts.Timeouts.doWithTimeout(Timeouts.java:60)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilReady(WaitAllStrategy.java:54)
	at org.testcontainers.containers.PostgreSQLContainer.waitUntilContainerStarted(PostgreSQLContainer.java:149)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500)
	... 16 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32790] should be listening)
	at org.testcontainers.containers.wait.strategy.HostPortWaitStrategy.waitUntilReady(HostPortWaitStrategy.java:112)
	at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilNestedStrategiesAreReady(WaitAllStrategy.java:66)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.lambda$waitUntilReady$0(WaitAllStrategy.java:58)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

	at io.quarkus.builder.Execution.run(Execution.java:123)
	at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79)
	at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:161)
	at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:350)
	... 9 more
Caused by: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image registry.redhat.io/rhel9/postgresql-16:latest
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:367)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.launchDatabases(DevServicesDatasourceProcessor.java:121)
	at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
	at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
	at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
	at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	at org.jboss.threads.JBossThread.run(JBossThread.java:483)
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed for image registry.redhat.io/rhel9/postgresql-16:latest
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:359)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:330)
	at io.quarkus.devservices.postgresql.deployment.PostgresqlDevServicesProcessor$1.startDatabase(PostgresqlDevServicesProcessor.java:80)
	at io.quarkus.datasource.deployment.devservices.DevServicesDatasourceProcessor.startDevDb(DevServicesDatasourceProcessor.java:293)
	... 10 more
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:344)
	... 13 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:563)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:354)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 14 more
Caused by: java.lang.IllegalStateException: Wait strategy failed. Container exited with code 2
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:533)
	... 16 more
Caused by: java.lang.RuntimeException: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32790] should be listening)
	at org.rnorth.ducttape.timeouts.Timeouts.callFuture(Timeouts.java:68)
	at org.rnorth.ducttape.timeouts.Timeouts.doWithTimeout(Timeouts.java:60)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilReady(WaitAllStrategy.java:54)
	at org.testcontainers.containers.PostgreSQLContainer.waitUntilContainerStarted(PostgreSQLContainer.java:149)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500)
	... 16 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32790] should be listening)
	at org.testcontainers.containers.wait.strategy.HostPortWaitStrategy.waitUntilReady(HostPortWaitStrategy.java:112)
	at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.waitUntilNestedStrategiesAreReady(WaitAllStrategy.java:66)
	at org.testcontainers.containers.wait.strategy.WaitAllStrategy.lambda$waitUntilReady$0(WaitAllStrategy.java:58)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)

@michalvavrik
Copy link
Member Author

michalvavrik commented Oct 30, 2024

2024-10-30 19:09:07,082 ERROR [tc.reg.red.io/rhel9/postgresql-16:latest] (build-32) Log output from the failed container:
postgres does not know where to find the server configuration file.
You must specify the --config-file or -D invocation option or set the PGDATA environment variable.

I forgot to mention @gsmet , I had to deal with it for our OpenShfit tests few weeks back, fix (or workaround?) was this: quarkus-qe/quarkus-test-framework#1305. And for PGDATA we do other PGDATA=/tmp/psql but I'll leave it to experts.

@maxandersen
Copy link
Member

podman run registry.redhat.io/rhel9/postgresql-16:latest
Trying to pull registry.redhat.io/rhel9/postgresql-16:latest...
Error: initializing source docker://registry.redhat.io/rhel9/postgresql-16:latest: unable to retrieve auth token: invalid username/password: unauthorized: Please login to the Red Hat Registry using your Customer Portal credentials. Further instructions can be found here: https://access.redhat.com/RegistryAuthentication

doesn't look like these are publically available?

@michalvavrik
Copy link
Member Author

Ah, sorry about that. I probably have some local config I forgot about. Interesting that GH CI can handle pulling images from RH registry though.

@michalvavrik
Copy link
Member Author

michalvavrik commented Oct 31, 2024

@gsmet @maxandersen I am sorry I said that this image was publicly available. I was wrong. I reviewed few days ago PR of my colleague that insisted it was public and all checks I did seemed like he was right (like me and @mjurc could pull it as well, GH CI succeeded, ...). I am leaving this issue open because I think proposal makes sense and is testable considering GH CI can pull it without additional setup (for whatever reason).

@gsmet
Copy link
Member

gsmet commented Oct 31, 2024

Well, I think you are trying to replace an official PostgreSQL image by something that doesn't respect the official PostgreSQL image contract. At least, that's what it looks like given it seems to ask for additional parameter.

If you can determine what this image requires to boot properly and if you can paste the log output of the container start, we might be able to do something about it.

But it's not certain.

@michalvavrik
Copy link
Member Author

michalvavrik commented Oct 31, 2024

Well, I think you are trying to replace an official PostgreSQL image by something that doesn't respect the official PostgreSQL image contract. At least, that's what it looks like given it seems to ask for additional parameter.

If my RHBQ application runs in OpenShift, I could use https://catalog.redhat.com/software/containers/rhel9/postgresql-16/657b03866783e1b1fb87e142 for development so that there are no hidden surprises when I use that image in OpenShift.

If you can determine what this image requires to boot properly and if you can paste the log output of the container start, we might be able to do something about it.

Yes, I can do that, I can probably open PR that shows how I can get it work and check that GH CI passes, but in the end I think it will be you who has to handle this. I'll try it later this week depending how my work schedule allows (which means we can check next week if I remember your plans :-) ). Let's wait till I check whether I am right or wrong.

@gsmet
Copy link
Member

gsmet commented Oct 31, 2024

If my RHBQ application runs in OpenShift, I could use https://catalog.redhat.com/software/containers/rhel9/postgresql-16/657b03866783e1b1fb87e142 for development so that there are no hidden surprises when I use that image in OpenShift.

I understood your requirement. What I'm saying is that this image is not a 1-1 replacement for the official PostgreSQL image. It has different requirements.

So we need to figure out what they are to see if we can make them work or not - while not breaking the support for the official image. Thus why I'm asking what are the additional parameters that are necessary to make it start and the log output to make sure the startup tests will work with it.

Typically, we are testing:

        this.waitStrategy =
            new LogMessageWaitStrategy()
                .withRegEx(".*database system is ready to accept connections.*\\s")
                .withTimes(2)
                .withStartupTimeout(Duration.of(60, ChronoUnit.SECONDS));

@michalvavrik
Copy link
Member Author

michalvavrik commented Oct 31, 2024

Sounds reasonable @gsmet , thanks. Yes, I know it is doable because I already have it working in QE TS, like you can see TC starting it when you do:

git clone [email protected]:quarkus-qe/quarkus-test-suite.git
cd quarkus-test-suite/sql-db/hibernate-reactive
mvn clean verify -Dit.test=PostgresqlDatabaseHibernateReactiveIT -Daarch64

But there is something in PostgresqlDevServicesProcessor that makes different to generic TC container QE uses. I can't debug this difference right now. So I can come back few days later and check.

I am really glad you would be willing to fix that when I provide configuration requirements.

@yrodiere yrodiere added the triage/needs-feedback We are waiting for feedback. label Nov 4, 2024
@michalvavrik
Copy link
Member Author

My JIRA ticket just got closed and this is probably of little interest to community if downstream doesn't want it, so I'll close this.

@maxandersen
Copy link
Member

I think this is showing relevant issue and possible improvements we should look into.

It is perfectly valid to explore if there is at least a way to have the user configure its way out of differences that various postgressql images has.

@agreedSkiing
Copy link
Contributor

agreedSkiing commented Jan 3, 2025

Currently experimenting with the image redhat postgres 13:1-212 image which kind of works after I add the config below just to get the image to start correctly

quarkus.datasource.devservices.image-name = rhel8/postgresql-13:1-212
quarkus.datasource.devservices.container-env."POSTGRESQL_USER" = ${quarkus.datasource.username:user}
quarkus.datasource.devservices.container-env."POSTGRESQL_PASSWORD" = ${quarkus.datasource.password:pass}
quarkus.datasource.devservices.container-env."POSTGRESQL_DATABASE" = ${quarkus.datasource.devservices.db-name:quarkus}
quarkus.datasource.devservices.command = /usr/bin/run-postgresql
quarkus.datasource.devservices.db-name = quarkus

and but since the regex can’t find the correct log statement it fails after printing the after containers log outputs which are

2025-01-03 15:50:36,342+01 ERROR [tc.reg.dig.se:5050/digg/external/postgresql-13:1-212] () () (build-11) Log output from the failed container:
Warning: Can't detect cpu quota from cgroups
Warning: Can't detect cpuset size from cgroups, will use nproc
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/pgsql/data/userdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/pgsql/data/userdata -l logfile start

waiting for server to start....2025-01-03 14:49:38.702 UTC [30] LOG:  redirecting log output to logging collector process
2025-01-03 14:49:38.702 UTC [30] HINT:  Future log output will appear in directory "log".
 done
server started
/var/run/postgresql:5432 - accepting connections
=> sourcing /usr/share/container-scripts/postgresql/start/set_passwords.sh ...
ALTER ROLE
waiting for server to shut down.... done
server stopped
Starting server...
2025-01-03 14:49:40.117 UTC [1] LOG:  redirecting log output to logging collector process
2025-01-03 14:49:40.117 UTC [1] HINT:  Future log output will appear in directory "log".

Doing a docker exec into the container with psql -U user quarkus connects to the database

P.S. This also works for the latest rhel9 postgres 9.5-1734582033 which logs

2025-01-03 16:07:37,849+01 ERROR [tc.reg.dig.se:.] () () (build-54) Log output from the failed container:
Warning: Can't detect cpu quota from cgroups
Warning: Can't detect cpuset size from cgroups, will use nproc
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/pgsql/data/userdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok


Success. You can now start the database server using:

    pg_ctl -D /var/lib/pgsql/data/userdata -l logfile start

initdb: warning: enabling "trust" authentication for local connections
initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.
waiting for server to start....2025-01-03 15:06:42.800 UTC [30] LOG:  redirecting log output to logging collector process
2025-01-03 15:06:42.800 UTC [30] HINT:  Future log output will appear in directory "log".
 done
server started
/var/run/postgresql:5432 - accepting connections
=> sourcing /usr/share/container-scripts/postgresql/start/set_passwords.sh ...
ALTER ROLE
waiting for server to shut down..... done
server stopped
Starting server...
2025-01-03 15:06:44.388 UTC [1] LOG:  redirecting log output to logging collector process
2025-01-03 15:06:44.388 UTC [1] HINT:  Future log output will appear in directory "log".

@agreedSkiing
Copy link
Contributor

agreedSkiing commented Jan 3, 2025

If my RHBQ application runs in OpenShift, I could use https://catalog.redhat.com/software/containers/rhel9/postgresql-16/657b03866783e1b1fb87e142 for development so that there are no hidden surprises when I use that image in OpenShift.

I understood your requirement. What I'm saying is that this image is not a 1-1 replacement for the official PostgreSQL image. It has different requirements.

So we need to figure out what they are to see if we can make them work or not - while not breaking the support for the official image. Thus why I'm asking what are the additional parameters that are necessary to make it start and the log output to make sure the startup tests will work with it.

Typically, we are testing:

        this.waitStrategy =
            new LogMessageWaitStrategy()
                .withRegEx(".*database system is ready to accept connections.*\\s")
                .withTimes(2)
                .withStartupTimeout(Duration.of(60, ChronoUnit.SECONDS));

Don't know if this a quick solution, but if the generic database devservice would allow a user to define the start up regex themself with something like quarkus.datasource.devservices.check.at.start-up.for or quarkus.datasource.devservices.start-up.check-for...

This might not be such a good idea in the long run though, since given my previous post I needed to set specific container-env for the red hat image since they are not the same as the dockerhub postgres image

@yrodiere yrodiere removed the area/hibernate-orm Hibernate ORM label Jan 3, 2025
@yrodiere yrodiere removed the triage/needs-feedback We are waiting for feedback. label Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/devservices area/jdbc Issues related to the JDBC extensions area/kubernetes kind/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

6 participants