Remove unnecessary indexing in Chunk combinators

[mpilquist]

Fixes #2603 and #2627. Also addresses the issue described here: circe/circe-fs2#273 (cc @samspills)

The Chunk.Queue type provides the ability to efficiently create a single chunk out of any number of individual chunks. In 2.x, Chunk.Queue was not itself a subtype of Chunk. After various individual chunks had been accumulated in to a Chunk.Queue, the queue had to be converted to a single chunk via toChunk. This operation returned a single array backed chunk. Unfortunately, this relied on reflection and various tricks to build the array backed chunk without access to a ClassTag at the call site and ultimately was unsound. We fixed this in #2181.

One of the main changes in #2181 was making Chunk.Queue a subtype of Chunk. Doing so meant that Chunk could no longer guarantee O(1) lookup by index -- rather, it was now O(number of chunks in chunk queue). When a chunk queue consists only of singleton chunks, that becomes O(n). We added a Chunk#compact operation which built a chunk backed by an array, potentially primitive array.

As a result of #2181, the various combinators that used to call Chunk.Queue#toChunk no longer had to, resulting in a significant performance improvement as they no longer had to copy all the accumulated elements to a new array backed chunk. This tradeoff normally pays off, though not in the case where the O(n) lookup ends up dominating. The performance regressions reported in #2603 and #2627 are examples of such cases.

So why not add a call to .compact in the implementation of operations like chunkN? In short, we can't. We don't have a ClassTag[O] in those cases so we can't create primitive arrays. This gets us right back to the initial issue #2181 was addressing -- unsoundness as a result of too clever reflection.

This PR takes a different approach. We maintain the worst case O(n) lookup and we leave chunkN/unconsN as-is. Instead, we reimplement all the combinators on Chunk to avoid indexed lookup. If a user wants O(1) indexed lookup, they must call compact.

This PR also adds a new compactUntagged operation which stores the elements in an Array[Any] (boxed).

[mpilquist]

Identical to IndexedSeqChunk

[SystemFw]

So if I understand correctly, by using foreach in the combinators instead of apply directly, we can override foreach in Chunk.Queue to just traverse linearly once and keep the combinators O(N) rather than O(N^2) right?

[mpilquist]

So if I understand correctly, by using foreach in the combinators instead of apply directly, we can override foreach in Chunk.Queue to just traverse linearly once and keep the combinators O(N) rather than O(N^2) right?

Yep, that's right -- likewise for iterator

[SystemFw]

I think this is a good idea for our internal code. We might still want to explore a better apply for client code, but that's an orthogonal concern, so I'd go ahead with this