Cannot find features on classpath inside Spring Boot 3.2.0 uberjar #2828

ljpengelen · 2023-12-10T14:45:30Z

👓 What did you see?

When trying to execute features inside a Spring Boot 3.2.0 application packaged as an uberjar, the following error is shown.

org.junit.platform.commons.JUnitException: TestEngine with ID 'cucumber' failed to discover tests
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:132) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:107) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:78) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:99) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        at nl.cofx.cucumber.boot.CucumberTestRunner.run(CucumberTestRunner.java:18) ~[!/:0.0.1-SNAPSHOT]
        at nl.cofx.cucumber.boot.CucumberSpringBootApplication.run(CucumberSpringBootApplication.java:21) ~[!/:0.0.1-SNAPSHOT]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
        at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:445) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) ~[spring-context-6.1.1.jar!/:6.1.1]
        at org.springframework.boot.context.event.EventPublishingRunListener.started(EventPublishingRunListener.java:103) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at org.springframework.boot.SpringApplicationRunListeners.lambda$started$5(SpringApplicationRunListeners.java:76) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at java.base/java.lang.Iterable.forEach(Iterable.java:75) ~[na:na]
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at org.springframework.boot.SpringApplicationRunListeners.started(SpringApplicationRunListeners.java:76) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:329) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1342) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1331) ~[spring-boot-3.2.0.jar!/:3.2.0]
        at nl.cofx.cucumber.boot.CucumberSpringBootApplication.main(CucumberSpringBootApplication.java:16) ~[!/:0.0.1-SNAPSHOT]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[na:na]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
        at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[na:na]
        at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
Caused by: java.lang.IllegalArgumentException: 'path' must contain '/!'
        at org.springframework.boot.loader.net.protocol.nested.NestedLocation.parse(NestedLocation.java:98) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.net.protocol.nested.NestedLocation.fromUri(NestedLocation.java:89) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.nio.file.NestedFileSystemProvider.getPath(NestedFileSystemProvider.java:88) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at java.base/java.nio.file.Path.of(Path.java:208) ~[na:na]
        at java.base/java.nio.file.Paths.get(Paths.java:98) ~[na:na]
        at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.uriToPath(ZipFileSystemProvider.java:76) ~[jdk.zipfs:na]
        at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.newFileSystem(ZipFileSystemProvider.java:98) ~[jdk.zipfs:na]
        at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:339) ~[na:na]
        at java.base/java.nio.file.FileSystems.newFileSystem(FileSystems.java:288) ~[na:na]
        at io.cucumber.core.resource.JarUriFileSystemService.openFileSystem(JarUriFileSystemService.java:51) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.JarUriFileSystemService.open(JarUriFileSystemService.java:32) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.JarUriFileSystemService.handleJarUriScheme(JarUriFileSystemService.java:100) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.JarUriFileSystemService.open(JarUriFileSystemService.java:71) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.PathScanner.open(PathScanner.java:41) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.PathScanner.findResourcesForUri(PathScanner.java:29) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.ResourceScanner.findResourcesForUri(ResourceScanner.java:61) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.ResourceScanner.lambda$findResourcesForUris$3(ResourceScanner.java:104) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[na:na]
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[na:na]
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[na:na]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[na:na]
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[na:na]
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[na:na]
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[na:na]
        at io.cucumber.core.resource.ResourceScanner.findResourcesForUris(ResourceScanner.java:107) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.core.resource.ResourceScanner.scanForClasspathResource(ResourceScanner.java:115) ~[cucumber-core-7.14.1.jar!/:7.14.1]
        at io.cucumber.junit.platform.engine.FeatureResolver.resolveClasspathResource(FeatureResolver.java:188) ~[cucumber-junit-platform-engine-7.14.1.jar!/:na]
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ~[na:na]
        at io.cucumber.junit.platform.engine.DiscoverySelectorResolver.resolve(DiscoverySelectorResolver.java:76) ~[cucumber-junit-platform-engine-7.14.1.jar!/:na]
        at io.cucumber.junit.platform.engine.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:48) ~[cucumber-junit-platform-engine-7.14.1.jar!/:na]
        at io.cucumber.junit.platform.engine.CucumberTestEngine.discover(CucumberTestEngine.java:43) ~[cucumber-junit-platform-engine-7.14.1.jar!/:na]
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152) ~[junit-platform-launcher-1.10.1.jar!/:1.10.1]
        ... 38 common frames omitted

✅ What did you expect to see?

I expect the features to be executed and to see the output of the pretty plugin.

📦 Which tool/library version are you using?

I'm using Cucumber 7.14.1 in combination with Spring Boot 3.2.0.

🔬 How could we reproduce it?

I've created a minimal reproducible example: https://github.com/ljpengelen/cucumber-spring-boot-3.2.0

The README.md documents the steps required to reproduce the issue.

📚 Any additional context?

The problem is caused by recent changes to the code that supports Spring Boot's uberjar loading: https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.2-Release-Notes.

This text was originally generated from a template, then edited by hand. You can modify the template here.

The text was updated successfully, but these errors were encountered:

mpkorstanje · 2023-12-10T18:47:01Z

Interesting. The changelog states that:

The previous URL format of jar:file:/dir/myjar.jar:BOOT-INF/lib/nested.jar!/com/example/MyClass.class has been replaced with jar:nested:/dir/myjar.jar/!BOOT-INF/lib/nested.jar!/com/example/MyClass.class.

But looking at the tests I wrote for this, I would expect the original to be jar:file:/dir/myjar.jar!/BOOT-INF/lib/nested.jar!/com/example/MyClass.class. A typo?

https://github.com/cucumber/cucumber-jvm/blob/3ae7af56c92d970b5b240b2bbd81cea9c5125058/cucumber-core/src/test/java/io/cucumber/core/resource/ResourceScannerTest.java#L162C1-L186

mpkorstanje · 2023-12-10T19:04:52Z

Looking at this part of the stack trace

Caused by: java.lang.IllegalArgumentException: 'path' must contain '/!'
        at org.springframework.boot.loader.net.protocol.nested.NestedLocation.parse(NestedLocation.java:98) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.net.protocol.nested.NestedLocation.fromUri(NestedLocation.java:89) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]
        at org.springframework.boot.loader.nio.file.NestedFileSystemProvider.getPath(NestedFileSystemProvider.java:88) ~[cucumber-spring-boot-3.2.0-0.0.1-SNAPSHOT.jar:0.0.1-SNAPSHOT]

It looks like Spring Boot 3.2 provides its own file system for handling jar:nested uris. So ideally we treated it as a "regular jar scheme" and come up with a better discriminator for Spring Boot < 3.2 and regular uris + Spring Boot >= 3.2

Because I don't think the current check is very robust:

        String[] parts = uri.toString().split(JAR_URI_SEPARATOR);
        // Regular jar schemes
        if (parts.length <= 2) {

ljpengelen · 2023-12-10T19:30:41Z

I'm using the following in #2829

String uriAsString = uri.toString();

// Spring Boot jar scheme since 3.2.0
if (uriAsString.startsWith("jar:nested")) {
    int indexOfLastSeparator = uriAsString.lastIndexOf(JAR_URI_SEPARATOR);
    String jarUri = uriAsString.substring(0, indexOfLastSeparator);
    String jarPath = uriAsString.substring(indexOfLastSeparator + 1);
    return open(new URI(jarUri), fileSystem -> fileSystem.getPath(jarPath));
}

It works, but I don't know if this is the robustness you're looking for.

mpkorstanje · 2023-12-10T20:12:47Z

I was thinking about something more robust indeed.

Your pull request made me realize that the current URI parsing is incorrect. The jar uri format is:

jar:<url>!/[<entry>]

And being a recursive format, the current implementation will never parse it correctly. So while checking for "jar:nested" would solve the case for Spring, it doesn't make the existing code correct. I would like to imagine that had I correctly implemented the the current url parsing and not implemented #1821, this change from Spring could have gone unnoticed.

That said, I found this "specification" on https://www.iana.org/assignments/uri-schemes/prov/jar, I'd really like a more authoritative specification. 😄

Spring Boot 3.2 changed the URL format of their nested jars[1] to be more compliant with JDK expectations. They now represented nested jars as their own `nested` scheme rather than the `file` scheme. This allows these URLs to be used seamlessly with `FileSystems.newFileSystem`. Unfortunately the workarounds for Spring Boot 3.1 did not account for this. Additionally, our jar uri parsing assumed naively that there would only be a single `!/` in a regular jar uri. However, jar uris are recursively defined as[2]: ``` jar:<url>!/[<entry>] ``` And while this should allow Cucumber to discover resources in nested jars as well it does seem that Spring Boot 3.2 still has some issues[3]. Closes: #2828 1. https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.2-Release-Notes 2. https://www.iana.org/assignments/uri-schemes/prov/jar 3. spring-projects/spring-boot#38595

mpkorstanje · 2023-12-10T23:07:21Z

Okay. I think I've got something I'm reasonably happy with. I appreciate the reproducer, having that saved a lot of time.

Though it is nothing though that actually scanning nested jars in Spring still doesn't work. But that seems to be on Spring Boots end. Scanning regular files works with this fix.

Spring Boot 3.2 changed the URL format of their nested jars[1] to be more compliant with JDK expectations. They now represented nested jars as their own `nested` scheme rather than the `file` scheme. This allows these URLs to be used seamlessly with `FileSystems.newFileSystem`. Unfortunately the workarounds for Spring Boot 3.1 did not account for this. Additionally, our jar uri parsing assumed naively that there would only be a single `!/` in a regular jar uri. However, jar uris are recursively defined as[2]: ``` jar:<url>!/[<entry>] ``` And while this should allow Cucumber to discover resources in nested jars as well it does seem that Spring Boot 3.2 still has some issues[3]. Closes: #2828 1. https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.2-Release-Notes 2. https://www.iana.org/assignments/uri-schemes/prov/jar 3. spring-projects/spring-boot#38595

ljpengelen mentioned this issue Dec 10, 2023

Support new Spring Boot loader implementation #2829

Closed

7 tasks

mpkorstanje mentioned this issue Dec 10, 2023

[Core] Support nested jar file systems #2830

Merged

7 tasks

mpkorstanje closed this as completed in #2830 Dec 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot find features on classpath inside Spring Boot 3.2.0 uberjar #2828

Cannot find features on classpath inside Spring Boot 3.2.0 uberjar #2828

ljpengelen commented Dec 10, 2023

mpkorstanje commented Dec 10, 2023

mpkorstanje commented Dec 10, 2023 •

edited

Loading

ljpengelen commented Dec 10, 2023

mpkorstanje commented Dec 10, 2023 •

edited

Loading

mpkorstanje commented Dec 10, 2023 •

edited

Loading

Cannot find features on classpath inside Spring Boot 3.2.0 uberjar #2828

Cannot find features on classpath inside Spring Boot 3.2.0 uberjar #2828

Comments

ljpengelen commented Dec 10, 2023

👓 What did you see?

✅ What did you expect to see?

📦 Which tool/library version are you using?

🔬 How could we reproduce it?

📚 Any additional context?

mpkorstanje commented Dec 10, 2023

mpkorstanje commented Dec 10, 2023 • edited Loading

ljpengelen commented Dec 10, 2023

mpkorstanje commented Dec 10, 2023 • edited Loading

mpkorstanje commented Dec 10, 2023 • edited Loading

mpkorstanje commented Dec 10, 2023 •

edited

Loading

mpkorstanje commented Dec 10, 2023 •

edited

Loading

mpkorstanje commented Dec 10, 2023 •

edited

Loading