Testing reusable composables without activity not possible

[jjkester]

👓 What did you see?

When using a ComposeContentTestRule (created with createComposeRule()) in order to test a reusable composable without an activity, the test activity finishes before the test has completed (or even started). The test fails with an exception indicating the activity has already been destroyed.

A detailed description and example can be found in the README of the project referenced below.

✅ What did you expect to see?

The activity is only destroyed at the end of the test. The test passes.

📦 Which tool/library version are you using?

• Cucumber Android 4.9.0
• Android Gradle plugin 7.2.0
• Jetpack Compose 1.1.1
• Kotlin 1.6.10

Detailed versions can be found in the build.gradle file of the project referenced below.

🔬 How could we reproduce it?

1. Creating a minimal scenario consisting of two steps
2. Creating a steps class annotated with @WithJUnitRule containing a compose rule created with createComposeRule()
3. Implementing the first step to call setContent on the compose rule, setting a composable (for example some text)
4. Implementing the second step to assert the set content is displayed (for example some test)
5. Run the Cucumber tests

Project demonstrating the issue: https://github.com/jjkester/cucumber-compose

📚 Any additional context?

In the README of the project referenced above I have detailed my observations, including the debugging effort I went through and what I could observe/conclude from that.

---

This text was originally generated from a template, then edited by hand. You can modify the template here.

[lsuski]

I've debugged your project and it looks like ActivityFinisherRunListener is finishing ComponentActivity started in ComposeRule befiore test started.

 @Override
  public void testStarted(Description description) throws Exception {
    instrumentation.runOnMainSync(activityFinisher);
    waitForActivitiesToFinishRunnable.run();
  }

It looks like in cucumber this event is sent after rule before method.

[lsuski]

For now I see that it can be disabled:

 protected boolean shouldWaitForActivitiesToComplete() {
    // TODO(b/72831103): default this argument to false in a future release
    return Boolean.parseBoolean(
        InstrumentationRegistry.getArguments().getString("waitForActivitiesToComplete", "true"));
  }

[jjkester]

Thanks for the quick reply. May I ask how you found out this was the problem? Was it just experience or did I overlook something?

I have put breakpoints on the constructor and methods of ActivityFinisherRunListener. It looks to me like there is a difference in order of the test started event and the rules firing between regular JUnit4 and Cucumber in this case. Does that make sense?

In general, you'd want to wait for activities from previous tests to complete before starting a new one, so I'm not sure disabling it is the way to go. However, apparently it should default to false at some point in the future. I'll try that. I'll also have a quick try, provided I have some time, to see if some hacking to switch the event order and rule execution resolves the issue. Also I'll try to check the order JUnit runs it.

[lsuski]

What I've done was:

1. put breakpoint in setContent and then in ActivityScenario.launch to see if activity is opened and what activity because you don't specify any
2. put breakpoint in ActivityScenario.activityLifecycleObserver to see what is happening with activity
3. I realised that PAUSED, STOPPED, DESTROYED is called which was strange
4. I've put breakpoint in Activity.finish to see show is finishing this activity
5. I found that ActivityFinisher so I tracked its usages and found ActivityFinisherRunListener
6. Then I found the difference in order

As you can see in TODO they will eventually change this behaviour. After each test there is close method called on scenario so it will close activity anyway

[lsuski]

Your test fails now because it's expects wrong texts

[lsuski]

Use this:

class CucumberRunner : CucumberAndroidJUnitRunner() {

    override fun onCreate(bundle: Bundle?) {
        bundle?.putString("waitForActivitiesToComplete", "false")
        super.onCreate(bundle)
    }
}

and this

 Scenario Outline: Greet different names
    Given a greeting screen for name "<text>"
    Then "Hello <text>!" is displayed
    ```
    and only last test fails because no content is set

[lsuski]

Regarding implementation change it is probably possible to change the order and to run rules before after testStarted event. To do that I would have to move rules managment to test listener. However I can't promise any timeline for that as I'm extremely busy in my commercial project and this scenario is not relevant in my case.

[jjkester]

I never got to debugging the actual test cases, but thanks for the heads-up.

I have debugged the rule and activity finisher under "normal" JUnit conditions, and there the test started event fires before the rule is executed. A more future-proof solution would be, in my opinion, to see if Cucumber Android can use JUnit rules in a similar way to JUnit itself. Then we're not making things work by addressing the symptom (which is fine in the short-term, don't get me wrong) but actually improving the situation for all possible rules out there. Rule developers do make assumptions, as clearly illustrated by this example.

As mentioned I'll see if I can make a proof-of-concept and submit it as a PR. No promises though, this scenario is relevant for me but I'm also quite busy, so I do understand.

[lsuski]

Cucumber has different engine. There is no support for junit rules. Junit rule is some kind of wrapper, in Cucumber you have before/after hooks. Without significant changes in cucumber-core/junit it won't be possible to do that. I don't want also to copy lots of thing from cucumber-jvm repo