mulh implementation when compiled with clang 16.0.0+/icc2023.2.1+ and -O2 no longer works

[vmoya-smd]

The code in riscv/arith.h that implements mulh doesn't work when optimized starting in clang 16.0.0. It also seems to fail in the latest versions of icc.

inline int64_t mulh(int64_t a, int64_t b) {
int negate = (a < 0) != (b < 0);
uint64_t res = mulhu(a < 0 ? -a : a, b < 0 ? -b : b);
return negate ? ~res + (a * b == 0) : res;
}

The reason is that (a * b) is optimized out and the generated x86-64 assembly justs tests for a or b being 0. The implication seems to be that inputs that generate an overflow don't produce a valid result so can be ignored. The fail case is when the lower 64b of the result are 0 but the higher are not. For example: 0xffffc69888000000 * 0x0053564000000000.

Casting a and b to uint64_t seems to work but I'm not sure if it's a safe workaround.

inline int64_t mulh(int64_t a, int64_t b) {
int negate = (a < 0) != (b < 0);
uint64_t res = mulhu(a < 0 ? -a : a, b < 0 ? -b : b);
return negate ? ~res + ((uint64_t) a * (uint64_t) b == 0) : res;
}

Test case in Compiler Explorer

[aswaterman]

I've convinced myself that your proposed workaround is the right fix, since unsigned overflow is well defined (and since signed multiplication and unsigned multiplication produce the same result modulo 2^size).