Unable to run native tests in non prod mode

[Legion2]

Describe the bug

If I execute ./gradlew testNative to run the native tests. Quarkus always builds using the prod profile instead of the test profile. Even if quarkus.profile=test and quarkus.test.native-image-profile=test the native executable is created with the prod profile. At test execution time the test profiles then only overrides some of the configuration set by the prod profile, which results in an invalid configuration. This is potenially dangerous, because the production configuration is configured to access production databases which can result in dataloss during test execution (e.g. if local developer aws account has write access to database and the native test clear the database after each test case, this could delete all data in the production database)

Expected behavior

native tests do not use the prod profile at all (not at compile time and runtime). It should be opt-in that the native tests use an existing runner artifact which was build with some (unkown) profile before. The ./gradlew testNative command should use the test profile by default for compilation (quarkus.profile=test) and at runtime (quarkus.test.native-image-profile=test).

Actual behavior

Native tests are executed using the prod profile by default, which is against the principle of least surprise, because there is a test profile which is normally used for tests. ./gradlew testNative uses existing runner artifacts from any previous build, which may has access to production data and should not be executed on the developer machine and used for testing.

How to Reproduce?

1. create a quarkus project
2. run the native tests
3. see that quarkus is with prod profile activated

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.4.2.Final

Build tool (ie. output of mvnw --version or gradlew --version)

------------------------------------------------------------
Gradle 7.2
------------------------------------------------------------

Build time:   2021-08-17 09:59:03 UTC
Revision:     a773786b58bb28710e3dc96c4d1a7063628952ad

Kotlin:       1.5.21
Groovy:       3.0.8
Ant:          Apache Ant(TM) version 1.10.9 compiled on September 27 2020
JVM:          11.0.12 (Ubuntu 11.0.12+7-Ubuntu-0ubuntu3)
OS:           Linux 5.13.0-21-generic amd64

Additional information

No response

[gsmet]

Yes, that's a known limitation. The reason is that otherwise we would have to build the native executable twice.

[Legion2]

It should atleast be an option to run (build) the native executable for integration tests.

IMHO using the prod executable for (fast) tests executions without new compilation should be an opt-in feature. Because normally the prod executable should no be used for tests, it could have credentials to access production data.

[yrodiere]

[the prod executable] could have credentials to access production data

That's questionable; ideally you wouldn't put secrets in your executable. You would use environment variables or a .env file or an application.properties in your deployment directory, or any other independent source of secret. At least I believe that's the best practice these days.

Regardless, I agree (runtime) prod settings shouldn't be applied by default during integration tests, be it only because it's inconvenient. #24581 discusses a possible solution, so I'll close this issue in favor of #24581. Feel free to ping me if you feel we should reopen.

[Legion2]

Of cause we don't put credentials into the binary, but the URLs and configuration in the prod executable are pointing to production systems and credentials are discovered from the environment. For example, the application is configured to use the prod S3 bucket and uses the default aws credentials provider, which discovers credentials. Now if a developer runs this application locally the credentials of the developer are used, and if he has sufficient permissions, the application writes into the prod S3 bucket during integration tests. This is only an example and we have some other systems where this is a problem. Often it is not possible to configure all settings via environment variables, because they must be known at build time.