Intermittent test failure: metrics with async requests

[tjquinno]

Environment Details

• Helidon Version: 2.x
• Helidon SE or Helidon MP
• JDK version:
• OS:
• Docker version (if applicable):

---

Problem Description

Intermittent test failure in pipeline:

java.lang.AssertionError: 
Change in inflight after slow request completes
Expected: is <0L>
     but: was <1L>
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at io.helidon.microprofile.metrics.HelloWorldAsyncResponseTest.testInflightRequests(HelloWorldAsyncResponseTest.java:189)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.weld.bean.proxy.AbstractBeanInstance.invoke(AbstractBeanInstance.java:38)
	at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106)
	at io.helidon.microprofile.metrics.HelloWorldAsyncResponseTest$Proxy$_$$_WeldClientProxy.testInflightRequests(Unknown Source)

[tjquinno]

The problem could be that the test uses an Optional<ConcurrentGauge> local variable to save a reference to a metric which the server will update while the test is running. The test might need to retrieve the metric fresh each time from the registry; the JVM (unaware of the async update) might be optimizing away the fetch.

[tjquinno]

The intermittent test failures resulted from a problem in the test itself. I had written it originally with a hidden assumption that the server-side response.whenComplete action would have run in its thread and updated the inflight metric before the test client obtained the completed response and fetched the same (presumably, by then, updated) inflight metric.

In fact, likely due to thread scheduling, the client would occasionally get the response and retrieve the metric before the thread on the server had run to update the metric. In those rare cases, the test would report an error.