Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hibernate reactive + Flyway extension causes UnsatisfiedResolutionException #10716

Closed
burmanm opened this issue Jul 14, 2020 · 66 comments · Fixed by #36012
Closed

Hibernate reactive + Flyway extension causes UnsatisfiedResolutionException #10716

burmanm opened this issue Jul 14, 2020 · 66 comments · Fixed by #36012
Assignees
Labels
area/hibernate-orm Hibernate ORM area/hibernate-reactive Hibernate Reactive area/persistence OBSOLETE, DO NOT USE kind/bug Something isn't working
Milestone

Comments

@burmanm
Copy link

burmanm commented Jul 14, 2020

Describe the bug
Simply adding the quarkus-flyway extension when using Hibernate Reactive causes:

Caused by: javax.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type javax.persistence.EntityManagerFactory and qualifiers [@Default]
	- java member: io.quarkus.hibernate.reactive.runtime.ReactiveSessionFactoryProducer#emf
	- declared on CLASS bean [types=[io.quarkus.hibernate.reactive.runtime.ReactiveSessionFactoryProducer, java.lang.Object], qualifiers=[@Default, @Any], target=io.quarkus.hibernate.reactive.runtime.ReactiveSessionFactoryProducer]
	at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:487)
	at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:362)
	at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:226)

Expected behavior
Flyway should be usable in an application that uses the hibernate-reactive extension.

Actual behavior
Will not start.

To Reproduce
Steps to reproduce the behavior:

  1. Take hibernate-reactive-quickstart
  2. Add the following dependencies:
        <!-- Flyway specific dependencies -->
        <dependency>
        <groupId>io.quarkus</groupId>
        <artifactId>quarkus-flyway</artifactId>
        </dependency>

        <!-- JDBC driver dependencies -->
        <dependency>
        <groupId>io.quarkus</groupId>
        <artifactId>quarkus-jdbc-postgresql</artifactId>
        </dependency>
  1. mvn quarkus:dev

Environment (please complete the following information):

  • Output of java -version: openjdk version "11.0.7" 2020-04-14
  • Quarkus version or git rev: 1.6.0.Final

Additional context
(Add any other context about the problem here.)

@burmanm burmanm added the kind/bug Something isn't working label Jul 14, 2020
@quarkusbot quarkusbot added area/hibernate-orm Hibernate ORM area/hibernate-reactive Hibernate Reactive area/persistence OBSOLETE, DO NOT USE labels Jul 14, 2020
@quarkusbot
Copy link

/cc @aguibert, @gavinking, @Sanne
/cc @gsmet, @Sanne

@geoand
Copy link
Contributor

geoand commented Jul 14, 2020

Interesting. Do you have a small reproducer showing this problem?

I assume if it's just adding as simple as adding quarkus-flyway to the hibernate-reactive quickstart we don't need one.

@geoand
Copy link
Contributor

geoand commented Jul 14, 2020

Actually, this doesn't seem like a bug to me... Flyway requires that a blocking driver be present and the Hibernate Reactive Quickstart doesn't use one.

@burmanm
Copy link
Author

burmanm commented Jul 14, 2020

The reproducer is the quickstart steps. So its a design choice to limit Hibernate Reactive from using Flyway? Even if Hibernate Reactive doesn't use it, what prevents it from just ignoring that blocking one (and let just Flyway use it) ?

Kinda defeats the point of quarkus-flyway if it doesn't work with any async db extensions.

@geoand
Copy link
Contributor

geoand commented Jul 14, 2020

Kinda defeats the point of quarkus-flyway if it doesn't work with any async db extensions.

quarkus-flyway needs a blocking driver because Flyway needs a blocking driver - it's not a Quarkus limitation.

As for using quarkus-flyway along with hibernate-reactive, did you try adding a blocking driver and configuring the jdbc URL? The Hibernate-Reactive quickstart doesn't do that obviously since the blocking driver isn't needed.

@burmanm
Copy link
Author

burmanm commented Jul 14, 2020

Yes, I had a working flyway installation in my own project. I tried adding hibernate-reactive to it but that started the issues, the quickstart modification was just a reproducer.

@geoand
Copy link
Contributor

geoand commented Jul 14, 2020

OK, but the way you mention to reproduce the issue doesn't surface that concern.

Let me try something else and get back to you

@geoand
Copy link
Contributor

geoand commented Jul 14, 2020

Okay, my bad. I see that just adding quarkus-flyway and quarkus-postgresql and configuring a JDBC URL does indeed exhibit the behavior you said.

So yeah, it's definitely an issue, quarkus-hibernate-reactive shouldn't be blocked from starting just by the presence of quarkus-flyway (quarkus-postgresql on it's own doesn't cause a problem).

@geoand
Copy link
Contributor

geoand commented Jul 14, 2020

@Sanne, @aguibert @gsmet this ultimately comes down to the fact that HibernateOrmProcessor.defineTypeOfImpliedPU defines a PU when quarkus-flyway is thrown into the mix (because it results in AgroalProcessor being run which is normally excluded by the hibernate-reactive extension).
That leads to 2 PersistenceUnitDescriptorBuildItem being created and HibernateOrmProcessor not registering DefaultEntityManagerFactoryProducer as a bean as a result (in HibernateOrmProcessor.registerBeans).

So it seems to me that in order to fix this, we need some more clever way of interacting between the datasource configuration and hibernate extensions code.

@gsmet
Copy link
Member

gsmet commented Jul 15, 2020

@Sanne @aguibert I think we should enable Hibernate Reactive if the Hibernate Reactive extension is around and a default reactive datasource is configured. I wouldn't take the presence of Agroal into consideration.

@Sanne
Copy link
Member

Sanne commented Jul 15, 2020

@Sanne @aguibert I think we should enable Hibernate Reactive if the Hibernate Reactive extension is around and a default reactive datasource is configured. I wouldn't take the presence of Agroal into consideration.

I agree, that's what I had in mind to do. I suppose I got confused with things while working on the previous fix.

@Sanne Sanne self-assigned this Jul 15, 2020
@vhdirk
Copy link
Contributor

vhdirk commented Jul 30, 2020

I can confirm this issue also exists for quarkus-liquibase. Is there a way to work around this for the time being?

@gwenneg
Copy link
Member

gwenneg commented Mar 5, 2021

@Sanne Is there a way to work around this issue until a fix is included into a Quarkus release?

@gwenneg
Copy link
Member

gwenneg commented Mar 5, 2021

By "this issue", I don't mean the ReactiveSessionFactoryProducer injection issue which can be easily worked around by duplicating the Quarkus producer. I'm more interested in the exception that comes after that:

2021-03-05 14:33:43,921 INFO  [org.fly.cor.int.lic.VersionPrinter] (Quarkus Main Thread) Flyway Community Edition 7.2.0 by Redgate
2021-03-05 14:33:43,973 INFO  [org.fly.cor.int.dat.bas.DatabaseType] (Quarkus Main Thread) Database: jdbc:postgresql://127.0.0.1:5432/notifications (PostgreSQL 13.2)
2021-03-05 14:33:43,993 INFO  [org.fly.cor.int.com.DbMigrate] (Quarkus Main Thread) Current version of schema "public": 1.9.0
2021-03-05 14:33:43,994 INFO  [org.fly.cor.int.com.DbMigrate] (Quarkus Main Thread) Schema "public" is up to date. No migration necessary.
2021-03-05 14:33:43,998 DEBUG [io.qua.hib.rea.run.FastBootHibernateReactivePersistenceProvider] (Quarkus Main Thread) Located 2 persistence units; checking each
2021-03-05 14:33:43,998 DEBUG [io.qua.hib.rea.run.FastBootHibernateReactivePersistenceProvider] (Quarkus Main Thread) Checking persistence-unit [name=default-reactive, explicit-provider=null] against incoming persistence unit name [default-reactive]
2021-03-05 14:33:44,002 INFO  [org.hib.rea.pro.imp.ReactiveIntegrator] (Quarkus Main Thread) HRX000001: Hibernate Reactive Preview
2021-03-05 14:33:44,014 DEBUG [io.qua.hib.rea.run.FastBootHibernateReactivePersistenceProvider] (Quarkus Main Thread) Located 2 persistence units; checking each
2021-03-05 14:33:44,014 DEBUG [io.qua.hib.rea.run.FastBootHibernateReactivePersistenceProvider] (Quarkus Main Thread) Checking persistence-unit [name=default-reactive, explicit-provider=null] against incoming persistence unit name [<default>]
2021-03-05 14:33:44,014 DEBUG [io.qua.hib.rea.run.FastBootHibernateReactivePersistenceProvider] (Quarkus Main Thread) Excluding from consideration 'default-reactive' due to name mis-match
2021-03-05 14:33:44,015 DEBUG [io.qua.hib.rea.run.FastBootHibernateReactivePersistenceProvider] (Quarkus Main Thread) Checking persistence-unit [name=<default>, explicit-provider=null] against incoming persistence unit name [<default>]
2021-03-05 14:33:44,017 INFO  [io.sma.rea.mes.provider] (Quarkus Main Thread) SRMSG00207: Cancel subscriptions
2021-03-05 14:33:44,017 ERROR [io.qua.run.Application] (Quarkus Main Thread) Failed to start application (with profile dev): java.lang.IllegalStateException: Booting an Hibernate Reactive serviceregistry on a non-reactive RecordedState!
        at io.quarkus.hibernate.reactive.runtime.boot.registry.PreconfiguredReactiveServiceRegistryBuilder.checkIsReactive(PreconfiguredReactiveServiceRegistryBuilder.java:76)
        at io.quarkus.hibernate.reactive.runtime.boot.registry.PreconfiguredReactiveServiceRegistryBuilder.<init>(PreconfiguredReactiveServiceRegistryBuilder.java:66)
        at io.quarkus.hibernate.reactive.runtime.FastBootHibernateReactivePersistenceProvider.rewireMetadataAndExtractServiceRegistry(FastBootHibernateReactivePersistenceProvider.java:164)
        at io.quarkus.hibernate.reactive.runtime.FastBootHibernateReactivePersistenceProvider.getEntityManagerFactoryBuilderOrNull(FastBootHibernateReactivePersistenceProvider.java:143)
        at io.quarkus.hibernate.reactive.runtime.FastBootHibernateReactivePersistenceProvider.createEntityManagerFactory(FastBootHibernateReactivePersistenceProvider.java:75)
        at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:80)
        at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:55)
        at io.quarkus.hibernate.orm.runtime.JPAConfig$LazyPersistenceUnit.get(JPAConfig.java:117)
        at io.quarkus.hibernate.orm.runtime.JPAConfig.startAll(JPAConfig.java:41)
        at io.quarkus.hibernate.orm.runtime.HibernateOrmRecorder.startAllPersistenceUnits(HibernateOrmRecorder.java:86)
        at io.quarkus.deployment.steps.HibernateOrmProcessor$startPersistenceUnits-2026547058.deploy_0(HibernateOrmProcessor$startPersistenceUnits-2026547058.zig:74)
        at io.quarkus.deployment.steps.HibernateOrmProcessor$startPersistenceUnits-2026547058.deploy(HibernateOrmProcessor$startPersistenceUnits-2026547058.zig:40)
        at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:971)
        at io.quarkus.runtime.Application.start(Application.java:90)
        at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:97)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:62)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:38)
        at io.quarkus.runtime.Quarkus.run(Quarkus.java:104)
        at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at io.quarkus.runner.bootstrap.StartupActionImpl$3.run(StartupActionImpl.java:134)
        at java.base/java.lang.Thread.run(Thread.java:834)

@Sanne
Copy link
Member

Sanne commented Mar 5, 2021

Hi @gwenneg , I'm not sure yet about a clever workaround - I need to have a better look, I'll start with this on Monday.

The core of the issue is that both ORM blocking and non-blocking are being triggered for initialization and there's a confusing mix of services being shared among them.

In terms of quick workarounds - would you be able to keep Flyway in a separate application? I've always wondered how far this belongs in the same deployment unit; for example it should be easy to have an app module which does take care of the migration, and a different app module which just boots on the migrated DB. It certainly would be more "minimal".

@gwenneg
Copy link
Member

gwenneg commented Mar 5, 2021

Thanks for your help on this one!

Keeping Flyway in a separate application could work indeed. I have a different kind of solution which seems to work fine in JVM mode (not so sure about native mode but I don't need it for now):

@ApplicationScoped
public class FlywayWorkaround {

    @ConfigProperty(name = "quarkus.datasource.reactive.url")
    String datasourceUrl;

    @ConfigProperty(name = "quarkus.datasource.username")
    String datasourceUsername;

    @ConfigProperty(name = "quarkus.datasource.password")
    String datasourcePassword;

    public void runFlywayMigration(@Observes StartupEvent event) {
        Flyway flyway = Flyway.configure().dataSource("jdbc:" + datasourceUrl, datasourceUsername, datasourcePassword).load();
        flyway.migrate();
    }
}

I wish I could keep quarkus-flyway in the project, but I need to focus on quarkus-hibernate-reactive for now so I'll probably stick with that workaround.

@andreas-eberle
Copy link
Contributor

@gwenneg: How does your application.properties look like? I tried your code but I still get an error that the jdbc.url is not defined.

@andreas-eberle
Copy link
Contributor

It looks to me that hibernate-reactive/reactive-postgres checks if the jdbc.url property is not configured on the datasource. At least it crashes as soon as that is the case. Why is this the case? Why shouldn't I be allowed to use the same datasource in a reactive and non-reactive way? The configuration could be the same for reactive and non-reactive and I could create different instances for each kind of driver.

@gwenneg
Copy link
Member

gwenneg commented Mar 5, 2021

@andreas-eberle My applications.properties file looks like this:

quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=removed
quarkus.datasource.password=removed
quarkus.datasource.jdbc=false
quarkus.datasource.reactive.url=postgresql://127.0.0.1:5432/mydb

The quarkus.datasource.jdbc=false line is very important. If it's missing, Quarkus won't boot.

@andreas-eberle
Copy link
Contributor

@gwenneg. Thanks a lot!! We would have never thought of that!

@Sanne
Copy link
Member

Sanne commented Mar 5, 2021

The reason is that in early days a datasource couldn't be both jdbc and reactive, and the Hibernate would look at what kind of datasource was configured to decide to boot as Hibernate ORM vs as Hibernate Reactive.

Until recently the integration of Hibernate Reactive was a POC taking advantage of some simplied assumptions; among these: if you're using Hibernate Reactive there won't be any blocking JDBC-based stuff. Clearly that's naive, but was considered acceptable as we focused on maturing the core of Hibernate Reactive.

Now that a single datasource can be both reactive and blocking, and we can have multiple datasource with different configurations, we need to correct the assumptions that the extension is making; we'll probably need some explicit way for users to choose what kind of Hibernate they want - however I'd love it if we could infer this automatically, provided it can be done in a robust and non-suprising way.

I'm open to suggestions :)

Same basic draft:

  • first, filter on options which are available (e.g. if jdbc=false or the matching JDBC driver is missing) then JDBC won't be available, and vice-versa for the reactive aspect.
  • second, we could look at defined injection points. If, for example, there are no injection points defined for @EntityManager or the @Session, @EntityManagerFactory etc... then we don't create the blocking flavour of Hibernate ORM.

Currently the Hibernate Reactive extension depends on the blocking one, so if the Hibernate Reactive extension is missing it's easy: no hibernate reactive will start - but the opposite isn't true. I propose this is fine initially, we can refine the automatic guess in the future, as we could evolve the Hibernate Reactive extension to no longer depend on the blocking ORM extension.

More importantly, some other extensions use the JPA service while not using annotations to define injection points; for example if such other extension should perform programmatic lookup of EntityManager, this would fail. So we'll also need to introduce an SPI to allow other extensions to flag their actual need for a certain ORM instance to boot.

Interestingly it seems secutity-jpa uses the annotation explicitly, so that wouldn't need such an SPI - and I'm not aware of other modules needing this, so we might be able to implemet this in small steps.

@andreas-eberle
Copy link
Contributor

I'd like to suggest to only have a single url property for datasources. As far as I understand, at the moment there is jdbc.url as well as reactive.url with the jdbc.url always starting with the addtional jdbc: prefix. I think it would be much cleaner to just have a single url property without the jdbc: prefix and add that prefix if JDBC is used and do not add it when reactive is used. This would reduce two properties which are mostly the same into one single one containing the required information.

@famod
Copy link
Member

famod commented Mar 16, 2021

I can confirm this issue also exists for quarkus-liquibase.

FTR: #14682

@cdmikechen
Copy link

@seeseemelk Yes, it is a custom config. I've written notes in codes.

I dont' get it. How to you reference the migration files? 🤔

I have explained in the code, you need to specify manually with a list.

@seeseemelk
Copy link

@seeseemelk Yes, it is a custom config. I've written notes in codes.

I dont' get it. How to you reference the migration files? 🤔

This is what I use in my application: https://gist.github.com/seeseemelk/27c601ad8ce5a6a89d423a476d6e9237

@cdmikechen
Copy link

@seeseemelk Yes, it is a custom config. I've written notes in codes.

I dont' get it. How to you reference the migration files? 🤔

This is what I use in my application: https://gist.github.com/seeseemelk/27c601ad8ce5a6a89d423a476d6e9237

Could you be sure that it will work under native as well?

@TheParad0X
Copy link

@seeseemelk Yes, it is a custom config. I've written notes in codes.

I dont' get it. How to you reference the migration files? 🤔

This is what I use in my application: https://gist.github.com/seeseemelk/27c601ad8ce5a6a89d423a476d6e9237

Can I see your application.properties? Do you have to list all migration files individually?

@seeseemelk
Copy link

Good point, I completely forgot about the application.properties.
I believe it does work under native, but yes, one does need to list all migration files individually.

quarkus.datasource.db-kind=postgresql
quarkus.datasource.jdbc=false
quarkus.hibernate-orm.database.generation=none
quarkus.native.resources.includes=db/migration/*.sql
todo.migration.files=V1__initial.sql

@y-luis-rojo
Copy link

Hi. What's the state of this issue? We're evaluating the adoption of Quarkus in my company but this issue seems a key question: we have no way to avoid DB migrations and reactive seems one of the flagships of Quarkus. How is it suppose we could move on without giving-up on one? I am skeptic to go on with the workaround for a production ready product. At least, is there any timeline for this to be fixed?

@SidMorad
Copy link

Could you be sure that it will work under native as well?

@cdmikechen Yes, I did make native image and this workaround works there too.
Thanks to you and @seeseemelk .

@arvind-das
Copy link

@andreas-eberle My applications.properties file looks like this:

quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=removed
quarkus.datasource.password=removed
quarkus.datasource.jdbc=false
quarkus.datasource.reactive.url=postgresql://127.0.0.1:5432/mydb

The quarkus.datasource.jdbc=false line is very important. If it's missing, Quarkus won't boot.

Looks like with new versions coming up, it does not work. My quarkus version is 2.13.2.Final.

With above configurations mentioned , it throws following error

org.flywaydb.core.api.FlywayException: No database found to handle vertx-reactive:mysql://localhost:3306/myapp

is there a specific version I should stick to ?

Thanks

@humcqc
Copy link
Contributor

humcqc commented May 3, 2023

Hello,
In 3.0 seems I have a slightly different error for the same use case:
2023-05-03 11:51:18,447 INFO [io.qua.dep.dev.IsolatedDevModeMain] (main) Attempting to start live reload endpoint to recover from previous Quarkus startup failure 2023-05-03 11:51:18,954 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors [error]: Build step io.quarkus.arc.deployment.SyntheticBeansProcessor#initRuntime threw an exception: java.lang.IllegalStateException: A synthetic bean with identifier 9c40b3d5cbb6ee6bb4cf2cb7bab0a1fda6694fe5 is already registered: SYNTHETIC bean [types=[org.hibernate.SessionFactory, jakarta.persistence.EntityManagerFactory, java.lang.Object], qualifiers=[@jakarta.enterprise.inject.Default, @Any], target=n/a] at io.quarkus.arc.processor.BeanDeployment.addSyntheticBean(BeanDeployment.java:1370) at io.quarkus.arc.processor.BeanDeployment$BeanRegistrationContextImpl.accept(BeanDeployment.java:1610) at io.quarkus.arc.processor.BeanDeployment$BeanRegistrationContextImpl.accept(BeanDeployment.java:1593) at io.quarkus.arc.processor.BeanConfigurator.done(BeanConfigurator.java:95) at io.quarkus.arc.deployment.SyntheticBeansProcessor.configureSyntheticBean(SyntheticBeansProcessor.java:92) at io.quarkus.arc.deployment.SyntheticBeansProcessor.initRuntime(SyntheticBeansProcessor.java:56) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:909) at io.quarkus.builder.BuildContext.run(BuildContext.java:282)

@Sanne , the exception seems not the same, do you think it should be handled in the same issue ?

Seems it involve different components and some assumptions should be reviewed for the hibernate reactive part.
Do you plan to work on this soon ? or do you recommend to workaround this for the moment ?

Thanks

@wjglerum
Copy link
Contributor

wjglerum commented Jun 7, 2023

We also hit this issue recently and came up with relative clean solution. We follow the approach from above, but also make sure everything works nicely in dev mode and production.

  • Configure Flyway with the datasource properties provided by Quarkus and replace the reactive url with the jdbc url for Flyway
  • Optionally inject the Quarkus Scheduler so it can be paused during migration, this way you avoid startup issues with scheduled jobs
  • Optionally clean the schema with Flyway, useful for dev mode
  • Finally validate the applied schema with Hibernate, by retrieving the SchemaManager from the SessionFactory, useful for prod mode where you want to make sure the schema is actually usable by Hibernate on startup
@Startup
public class FlywayMigration {

    FlywayMigration(Scheduler scheduler,
                    FlywayConfig flywayConfig,
                    SessionFactory sessionFactory,
                    @ConfigProperty(name = "quarkus.datasource.reactive.url") String datasourceUrl,
                    @ConfigProperty(name = "quarkus.datasource.username") String datasourceUsername,
                    @ConfigProperty(name = "quarkus.datasource.password") String datasourcePassword) {

        Flyway flyway = Flyway
                .configure()
                .dataSource(datasourceUrl.replace("vertx-reactive:", "jdbc:"), datasourceUsername, datasourcePassword)
                .cleanDisabled(!flywayConfig.clean())
                .load();

        if (flywayConfig.migrateAtStart()) {
            scheduler.pause();
            if (flywayConfig.cleanAtStart()) {
                flyway.clean();
            }
            flyway.migrate();
            sessionFactory.getSchemaManager().validateMappedObjects();
            scheduler.resume();
        }
    }

    @ConfigMapping(prefix = "application.database.flyway")
    public interface FlywayConfig {

        boolean migrateAtStart();

        @WithDefault("false")
        boolean cleanAtStart();
    }
}

For this you need the following extensions:

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-flyway</artifactId>
</dependency>
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-hibernate-reactive</artifactId>
</dependency>
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-jdbc-postgresql</artifactId>
</dependency>

<!-- Optional -->
<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-scheduler</artifactId>
</dependency>

We have the following config values set:

application.database.flyway.migrate-at-start=true
quarkus.hibernate-orm.database.generation=none

## Explicitly turns off JDBC and Flyway
quarkus.datasource.jdbc=false
quarkus.flyway.enabled=false

And in dev mode we set application.database.flyway.clean-at-start=true to get back the drop-and-create experience from Hibernate in dev mode.

@miguelborges99
Copy link

miguelborges99 commented Oct 22, 2023

The fix was released in 3.5, but I still need to do the workaround described above .
Actually, to have flyway working in a native image, I need to implement the following code:

@Startup
public class FlywayMigration {

    FlywayMigration(Scheduler scheduler,
                    SessionFactory sessionFactory,
                    FlywayConfig config) {

        final FluentConfiguration configuration = Flyway
                .configure()
                .dataSource("jdbc:".concat(config.getDatasourceUrl()), config.getDatasourceUsername(), config.getDatasourcePassword())
                .locations(config.getLocations())
                .validateMigrationNaming(config.getValidateMigrationNaming())
                .baselineOnMigrate(config.getBaselineOnMigrate())
                .baselineVersion(config.getBaselineVersion())
                .baselineDescription(config.getBaselineDescription())
                .connectRetries(config.getConnectRetries())
                .schemas(config.getSchemas())
                .table(config.getTable());

        if (System.getProperty("org.graalvm.nativeimage.imagecode") != null) {
            configuration.resourceProvider(new GraalVMResourceProvider(configuration.getLocations()));
            configuration.javaMigrationClassProvider(new GraalVMClassProvider());
        }

        final Flyway flyway = configuration.load();

        if (Boolean.TRUE.equals(config.getMigrateAtStart())) {
            scheduler.pause();
            if (config.getCleanAtStart()) {
                flyway.clean();
            }
            flyway.migrate();
            sessionFactory.getSchemaManager().validateMappedObjects();
            scheduler.resume();
        }
    }
}


public class GraalVMResourceProvider implements ResourceProvider {
    private final Location[] locations;

    public GraalVMResourceProvider(Location[] locations) {
        this.locations = Arrays.copyOf(locations, locations.length);
    }

    @Override
    public LoadableResource getResource(String name) {
        if (getClassLoader().getResource(name) == null) {
            return null;
        }
        return new ClassPathResource(null, name, getClassLoader(), StandardCharsets.UTF_8);
    }

    @Override
    public Collection<LoadableResource> getResources(String prefix, String[] suffixes) {
        try (FileSystem fileSystem = FileSystems.newFileSystem(URI.create("resource:/"), Map.of())) {
            final List<LoadableResource> result = new ArrayList<>();
            for (Location location : locations) {
                final Path path = fileSystem.getPath(location.getPath());
                try (Stream<Path> files = Files.walk(path)) {
                    files.filter(Files::isRegularFile)
                            .filter(file -> file.getFileName().toString().startsWith(prefix))
                            .filter(file -> hasSuffix(file.getFileName().toString(), suffixes))
                            .map(file -> (LoadableResource) new ClassPathResource(null,
                                    file.toString(), getClassLoader(), StandardCharsets.UTF_8))
                            .forEach(result::add);
                }
            }

            // Sort DB migration files
            final List<LoadableResource> resources = result.stream()
                    .sorted().collect(Collectors.toList());

            return resources;
        } catch (IOException ex) {
            throw new UncheckedIOException(ex);
        }
    }

    private boolean hasSuffix(String input, String[] suffixes) {
        for (String suffix : suffixes) {
            if (input.endsWith(suffix) || input.toUpperCase().endsWith(suffix.toUpperCase())) {
                return true;
            }
        }
        return false;
    }

    private static ClassLoader getClassLoader() {
        return GraalVMResourceProvider.class.getClassLoader();
    }
}

public class GraalVMClassProvider implements ClassProvider<JavaMigration> {

    @Override
    public Collection<Class<? extends JavaMigration>> getClasses() {
        return Collections.emptySet();
    }
}

Do I need to have the workaround described? What properties should I have to have Hibernate reactive and flyway working at the same time?

@yrodiere
Copy link
Member

@miguelborges99 We have a (simple) integration test that works with this configuration.

Several things to take care about:

  • Hibernate Reactive can only use the default datasource, and Flyway can't use a reactive datasource. So you need to configure the default datasource as reactive for Hibernate Reactive, and a separate, named, blocking datasource for Flyway.
  • I don't think you're supposed to instantiate Flyway objects yourself. See the guide for the recommended way of using Flyway.

If you still encounter problems, can you please open another issue with a reproducer? Thank you.

@humcqc
Copy link
Contributor

humcqc commented Oct 23, 2023

Hi @yrodiere ,
"Hibernate Reactive can only use the default datasource", does it mean that for multi-tenancy in reactive we cannot use multiple datasource, one per tenant ?

Thanks

@yrodiere
Copy link
Member

"Hibernate Reactive can only use the default datasource"

Yes, and that's documented here: https://quarkus.io/guides/hibernate-reactive#hr-limitations

does it mean that for multi-tenancy in reactive we cannot use multiple datasource, one per tenant ?

It seems the Hibernate Reactive extension for Quarkus doesn't support multi-tenancy at the moment: #15959

"static" configuration with named datasources certainly won't work due to this limitation.

A more dynamic approach where you create your own connections could work in theory, but I'm pretty sure it won't in practice because of the missing configuration mentioned in #15959.

Schema-based multitenancy is more likely to work to with Hibernate Reactive, FWIW. Though I'm not entirely sure either, since this I couldn't find relevant tests.

@humcqc
Copy link
Contributor

humcqc commented Oct 23, 2023

Thanks for the quick reply, i will continue the discussion on : #33342

@arvind-das
Copy link

arvind-das commented Nov 19, 2023

The solution provided in the thread to set datasource.jdbc: false works but then in the dev ui, there is no option to generate the initial migration script, is it happening with me only or others are also facing the same issue.

@erickloss
Copy link

Hello all,

I ran into the same Issue not being able to use flyway migration in combination with a reactive data source. The workaround above works fine for me (running flyway manually on application startup).

Just FYI: I constantly got exceptions when I open the Quarkus DEV UI:

(executor-thread-1) Error in JsonRPC Call: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
	at io.smallrye.mutiny.unchecked.UncheckedSupplier.lambda$toSupplier$0(UncheckedSupplier.java:45)
	at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
	at io.smallrye.mutiny.operators.uni.builders.UniCreateFromItemSupplier.subscribe(UniCreateFromItemSupplier.java:28)
	at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
	at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
	at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:582)
	at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538)
	at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
	at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.devui.runtime.comms.JsonRpcRouter.lambda$invoke$0(JsonRpcRouter.java:98)
	at io.smallrye.mutiny.unchecked.UncheckedSupplier.lambda$toSupplier$0(UncheckedSupplier.java:41)
	... 11 more
Caused by: java.lang.NullPointerException: Cannot invoke "io.quarkus.agroal.runtime.DataSources.getActiveDataSourceNames()" because the return value of "io.quarkus.arc.InstanceHandle.get()" is null
	at io.quarkus.flyway.runtime.FlywayContainerUtil.getActiveFlywayContainers(FlywayContainerUtil.java:26)
	at io.quarkus.flyway.runtime.FlywayContainersSupplier.get(FlywayContainersSupplier.java:16)
	at io.quarkus.flyway.runtime.devui.FlywayJsonRpcService.getNumberOfDatasources(FlywayJsonRpcService.java:165)
	at io.quarkus.flyway.runtime.devui.FlywayJsonRpcService_ClientProxy.getNumberOfDatasources(Unknown Source)
	... 17 more

I fixed that by removing the quarkus-flyway extension from the project. That also means there is no flyway tile in the Dev UI, but that's fine for me.

Cheers

@yrodiere
Copy link
Member

I ran into the same Issue not being able to use flyway migration in combination with a reactive data source.

This was supposed to be fixed in #36012, in Quarkus 3.5.0.CR1.

Can you please open a new issue with a reproducer?

Just FYI: I constantly got exceptions when I open the Quarkus DEV UI:

This looks like a bug too. Let's see what your reproducer looks like though, there must be something else at play.

@duartevinicius91
Copy link

Hi guys
I'm using the following config in my environment (and it works ;D)
I just tried to use a different datasource configuration for the migrations

`quarkus.datasource.db-kind = postgresql
quarkus.datasource.username = postgres
quarkus.datasource.password = postgres
quarkus.datasource.reactive.url = vertx-reactive:postgresql://localhost:5432/mydb
quarkus.datasource.jdbc = false

quarkus.flyway."flyway-migration".active = true
quarkus.flyway."flyway-migration".migrate-at-start = true
quarkus.datasource."flyway-migration".db-kind = postgresql
quarkus.datasource."flyway-migration".jdbc.url = jdbc:postgresql://localhost:5432/mydb
quarkus.datasource."flyway-migration".username = postgres
quarkus.datasource."flyway-migration".password = postgres`

@yrodiere
Copy link
Member

Hey, FWIW you don't need two datasources, so the right configuration would be something like this:

quarkus.datasource.db-kind = postgresql
quarkus.datasource.username = postgres
quarkus.datasource.password = postgres
quarkus.datasource.reactive.url = vertx-reactive:postgresql://localhost:5432/mydb
quarkus.datasource.jdbc.url = jdbc:postgresql://localhost:5432/mydb
quarkus.datasource.jdbc = false

quarkus.flyway.migrate-at-start = true

Or this if you use dev services:

quarkus.datasource.db-kind = postgresql
quarkus.datasource.username = postgres
quarkus.datasource.password = postgres
%prod.quarkus.datasource.reactive.url = vertx-reactive:postgresql://localhost:5432/mydb
%prod.quarkus.datasource.jdbc.url = jdbc:postgresql://localhost:5432/mydb
quarkus.datasource.jdbc = false

quarkus.flyway.migrate-at-start = true

I updated the documentation to make this clearer: https://quarkus.io/version/main/guides/flyway#reactive-datasources

@rovio-deepak
Copy link

Hi @yrodiere - I'm a recent victim of this scenario as well. I need Flyway working for local development and for other environments as well. Setting

quarkus.datasource.jdbc = false

Prevented Flyway from working locally for me. Is that expected? My final configuration looks like

quarkus.hibernate-orm.enabled=true
quarkus.hibernate-orm.database.generation=none
quarkus.hibernate-orm.log.sql=true
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=<username>
quarkus.flyway.migrate-at-start=true

%stage.quarkus.datasource.credentials-provider=my-ssm-credentials-provider
%stage.quarkus.datasource.db-version=13.12
%stage.quarkus.datasource.reactive.url=vertx-reactive:postgresql://<host>
%stage.quarkus.datasource.jdbc.url=jdbc:postgresql://<host>

But now I fear that Hibernate Reactive does not use the correct reactive driver anymore (since JDBC is not disabled). Does this look correct to you? Thanks!

@yrodiere
Copy link
Member

Hello

Setting

quarkus.datasource.jdbc = false

Prevented Flyway from working locally for me. Is that expected?

Yes. Flyway uses JDBC. You disable JDBC, Flyway doesn't work.

My final configuration looks like

quarkus.hibernate-orm.enabled=true
quarkus.hibernate-orm.database.generation=none
quarkus.hibernate-orm.log.sql=true
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=<username>
quarkus.flyway.migrate-at-start=true

%stage.quarkus.datasource.credentials-provider=my-ssm-credentials-provider
%stage.quarkus.datasource.db-version=13.12
%stage.quarkus.datasource.reactive.url=vertx-reactive:postgresql://<host>
%stage.quarkus.datasource.jdbc.url=jdbc:postgresql://<host>

But now I fear that Hibernate Reactive does not use the correct reactive driver anymore (since JDBC is not disabled).

Then fear not: Hibernate Reactive is absolutely incapable of using JDBC.

Does this look correct to you?

Yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/hibernate-orm Hibernate ORM area/hibernate-reactive Hibernate Reactive area/persistence OBSOLETE, DO NOT USE kind/bug Something isn't working
Projects
None yet