test: reduce test size to fit incremental GC (#531)

Currently, the CI builds for the incremental GC
(dfinity/motoko#3756) fail because they run out
of memory for this test case (where memory is not collected). The
incremental GC has a somewhat higher memory footprint as it uses larger
object headers, with an additional forwarding pointer.

Therefore, reducing the test size a little bit to also fit for the
future incremental GC.

test/RBTreeTestMore.mo

@@ -133,7 +133,7 @@ func shuffle(array : [Nat]) : [Nat] {
   )
 };
 
-let testSize = 6000;
+let testSize = 5000;
 
 let testKeys = shuffle(Array.tabulate<Nat>(testSize, func(index) { index }));