[BUG] test_hypot[Double] with DATAGEN_SEED=1700118086 failes

[revans2]

Describe the bug

[2023-11-16T08:36:34.333Z] ----------------------------- Captured stdout call -----------------------------
[2023-11-16T08:36:34.333Z] ### CPU RUN ###
[2023-11-16T08:36:34.333Z] ### GPU RUN ###
[2023-11-16T08:36:34.333Z] ### COLLECT: GPU TOOK 0.15743589401245117 CPU TOOK 0.15597891807556152 ###
[2023-11-16T08:36:34.333Z] --- CPU OUTPUT
[2023-11-16T08:36:34.333Z] +++ GPU OUTPUT
[2023-11-16T08:36:34.333Z] @@ -162,7 +162,7 @@
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=4.7649288220271566e+38)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=5.716482950764417e+60)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=9.16493876188188e+119)
[2023-11-16T08:36:34.333Z] -Row(HYPOT(a, b)=1.012833120432452e-227)
[2023-11-16T08:36:34.333Z] +Row(HYPOT(a, b)=1.0128331204324521e-227)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=1.0125195708346668e-07)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=0.011335000267386385)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=inf)
[2023-11-16T08:36:34.333Z] @@ -458,7 +458,7 @@
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=inf)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=4.189982105230443e+197)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=None)
[2023-11-16T08:36:34.333Z] -Row(HYPOT(a, b)=3.422031462089623e-123)
[2023-11-16T08:36:34.333Z] +Row(HYPOT(a, b)=3.422031462089622e-123)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=3.4211840517918124e+282)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=1.253731930287053e+29)
[2023-11-16T08:36:34.333Z]  Row(HYPOT(a, b)=6.442446824252015e+255)
[2023-11-16T08:36:34.333Z] @@ -513,7 +513,7 @@
(more to show)
[2023-11-16T08:36:34.335Z] =========================== short test summary info ============================
[2023-11-16T08:36:34.335Z] FAILED ../../src/main/python/arithmetic_ops_test.py::test_hypot[Double][DATAGEN_SEED=1700118086, INJECT_OOM, APPROXIMATE_FLOAT] - AssertionError: GPU and CPU float values are different [1968, 'HYPOT(a, b)']
[2023-11-16T08:36:34.335Z] = 1 failed, 19231 passed, 2174 skipped, 620 xfailed, 278 xpassed, 78 warnings in 5706.96s (1:35:06) =

It looks like the result is close, but not quire close enough to make APPROX_FLOAT happy.

[revans2]

Actually I reproduced this and it looks like it is not a small diff. It looks like Spark overflowed and produced an inf, when we produced a very large value instead.

[revans2]

-Row(a=-1.7976931348623157e+308, b=3.049517727610612e+300, HYPOT(a, b)=inf)
+Row(a=-1.7976931348623157e+308, b=3.049517727610612e+300, HYPOT(a, b)=1.7976931348623157e+308)

[ttnghia]

I've checked out the hypot code from openjdk (https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/lang/FdLibm.java#L1968) and realized that our implementation (GpuHypot) is completely different so it is almost impossible to guarantee a match, unless we reimplement our code. We can only try to make our output close to it as much as we can. I'll continue to study this to see what can we do for it.

[revans2]

@ttnghia be very careful here. That code is GPL licensed and we cannot copy it in any way.

[ttnghia]

Yes, and that is also one of the biggest problems now.

[firestarman]

moved to 24.02

[sameerz]

Removing the target release, as we do not have a near term fix, nor do we know of anyone using this feature.