jdk22 OpenJDK com/sun/crypto/provider/Cipher/AEAD/AEADBufferTest - javax.crypto.ShortBufferException: Need at least 130 bytes of space in output buffer

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk22_j9_sanity.openjdk_aarch64_mac_Personal_testList_2/1
jdk_security2
com/sun/crypto/provider/Cipher/AEAD/AEADBufferTest.java

16:06:12  Algo: ChaCha20-Poly1305 	Ops: [HEAP, HEAP]
16:06:12  Encrypt:  Data Index: 1
16:06:12  	input len: 114  inOfs 0  outOfs 0  in/out buffer: different
16:06:12  	input len: 114  inOfs 0  outOfs 0  in/out buffer: in-place
16:06:12  Decrypt:  Data Index: 1
16:06:12  	input len: 130  inOfs 0  outOfs 0  in/out buffer: different
16:06:12  	input len: 130  inOfs 0  outOfs 0  in-place: same
16:06:12  javax.crypto.ShortBufferException: Need at least 130 bytes of space in output buffer
16:06:12  	at java.base/javax.crypto.CipherSpi.bufferCrypt(CipherSpi.java:753)
16:06:12  	at java.base/javax.crypto.CipherSpi.engineDoFinal(CipherSpi.java:723)
16:06:12  	at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2555)
16:06:12  	at AEADBufferTest.cryptoSameBuffer(AEADBufferTest.java:635)
16:06:12  	at AEADBufferTest.decrypt(AEADBufferTest.java:364)
16:06:12  	at AEADBufferTest.test(AEADBufferTest.java:317)
16:06:12  	at AEADBufferTest.main(AEADBufferTest.java:783)

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk22_j9_sanity.openjdk_aarch64_linux_Nightly_testList_2/1
jdk_security2_0
com/sun/crypto/provider/Cipher/AEAD/AEADBufferTest.java

22:25:49  java.security.ProviderException: Error in Native CipherBlockChaining
22:25:49  	at java.base/com.sun.crypto.provider.NativeChaCha20Cipher$EngineAEADEnc.doUpdate(NativeChaCha20Cipher.java:1026)
22:25:49  	at java.base/com.sun.crypto.provider.NativeChaCha20Cipher$EngineAEADEnc.doFinal(NativeChaCha20Cipher.java:1045)
22:25:49  	at java.base/com.sun.crypto.provider.NativeChaCha20Cipher.engineDoFinal(NativeChaCha20Cipher.java:782)
22:25:49  	at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2470)
22:25:49  	at AEADBufferTest.cryptoSameBuffer(AEADBufferTest.java:630)
22:25:49  	at AEADBufferTest.encrypt(AEADBufferTest.java:342)
22:25:49  	at AEADBufferTest.test(AEADBufferTest.java:314)
22:25:49  	at AEADBufferTest.offsetTests(AEADBufferTest.java:661)
22:25:49  	at AEADBufferTest.main(AEADBufferTest.java:769)
22:25:49  	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
22:25:49  	at java.base/java.lang.reflect.Method.invoke(Method.java:586)
22:25:49  	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
22:25:49  	at java.base/java.lang.Thread.run(Thread.java:1582)
22:25:49  
22:25:49  JavaTest Message: Test threw exception: java.security.ProviderException
22:25:49  JavaTest Message: shutting down test
22:25:49  
22:25:49  stderr:
22:25:49  An OpenSSL error occurred
22:25:49  error:060DB0A2:digital envelope routines:evp_EncryptDecryptUpdate:partially overlapping buffers

[pshipton]

https://openj9-jenkins.osuosl.org/job/Test_openjdk22_j9_sanity.openjdk_aarch64_linux_Nightly_testList_1/3
jdk_security2_1
com/sun/crypto/provider/Cipher/AEAD/OverlapByteBuffer.java

22:22:56  STDOUT:
22:22:56  Using array-backed ByteBuffer
22:22:56  inOfsInBuf  = 1
22:22:56  outOfsInBuf = 0
22:22:56  stderr:
22:22:56  An OpenSSL error occurred
22:22:56  error:060DB0A2:digital envelope routines:evp_EncryptDecryptUpdate:partially overlapping buffers
22:22:56  STDERR:
22:22:56  java.lang.Exception: Error with base offset -1
22:22:56  	at OverlapByteBuffer.test(OverlapByteBuffer.java:170)
22:22:56  	at OverlapByteBuffer.main(OverlapByteBuffer.java:60)
22:22:56  	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
22:22:56  	at java.base/java.lang.reflect.Method.invoke(Method.java:586)
22:22:56  	at com.sun.javatest.regtest.agent.MainActionHelper$AgentVMRunnable.run(MainActionHelper.java:333)
22:22:56  	at java.base/java.lang.Thread.run(Thread.java:1582)
22:22:56  Caused by: java.security.ProviderException: Error in Native CipherBlockChaining
22:22:56  	at java.base/com.sun.crypto.provider.NativeChaCha20Cipher$EngineAEADEnc.doUpdate(NativeChaCha20Cipher.java:1026)
22:22:56  	at java.base/com.sun.crypto.provider.NativeChaCha20Cipher$EngineAEADEnc.doFinal(NativeChaCha20Cipher.java:1045)
22:22:56  	at java.base/com.sun.crypto.provider.NativeChaCha20Cipher.engineDoFinal(NativeChaCha20Cipher.java:782)
22:22:56  	at java.base/javax.crypto.CipherSpi.bufferCrypt(CipherSpi.java:788)
22:22:56  	at java.base/javax.crypto.CipherSpi.engineDoFinal(CipherSpi.java:723)
22:22:56  	at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2555)
22:22:56  	at OverlapByteBuffer.test(OverlapByteBuffer.java:141)
22:22:56  	... 5 more

Update exclude list adoptium/aqa-tests#5038

[pshipton]

@jasonkatonica this will be blocking jdk22 GA unless we determine otherwise.

[jasonkatonica]

@pshipton we will look closer into this one sometime soon.

[jasonkatonica]

This test is specifically testing overlapping input and output buffers that are being used for consumption of clear text and cipher text. The test fails on this line.

The Cipher documentation has a line mentioned as so:

Note: this method should be copy-safe, which means the input and output buffers can reference the same byte array and no unprocessed input data is overwritten when the result is copied into the output buffer.

In our case however for this cipher we call the openssl library. There are a few OpenSSL checks at spot1 and spot2 that could throw an error when calling this method in those two spots.

This test appears to have been modified about 3 weeks ago using this commit perhaps the additional coverage added for ChaCha20-Poly1305 ends up causing a failure to occur in OpenSSL.

[jasonkatonica]

Given that there is javadoc surrounding expected behavior here for Cipher in the case of overlapping input and output buffers we have a few options to consider so far. We believe that this behavior has been present for a long period of time and users have not run into any issues so far although we believe this to be a real issue that needs to be addressed.

1. Update OpenSSL to allow for overlapping buffers.
2. Make and update to our OpenSSL cipher classes to copy the input or output buffer to a new location, perform the cryptographic operation, and then copy the result back to where it is expected. This will certainly affect performance although we would only do this when overlapping buffers are present. Given that this is currently not working this shouldn't be too big of a problem.
3. Allow this to fail and we ignore the test.

We are currently considering option 2 above as our best option. Option 1 is outside of the teams scope without dedicating large amounts of time and Option 3 seems to violate what the public javadoc and expected behavior really should be from a functional perspective.

[jasonkatonica]

Some further investigation needs to be done why the code located in this method is not already creating a copy of the buffer before it reaches the NativeChaCha20Cipher class. This method seems to have the logic we need in this case.

[jasonkatonica]

This issue should now be fixed within the native chacha20 implementation and merged into JDK 11, 17, 21, 22, and the head stream. See associated PRs above for description on what was changed.

An additional PR was submitted to remove the previously failing tests from the ignore list for JDK 22 and 23.