Different Slice sizes on "put" operation and comparison

[PFedorovMsk]

Hello!
I put into my rocksdb::DB a key and value, each of which is an 8 byte rocksdb::Slice (the original data type is long long).
I also use my own comparator class, in which I check the sizes of the compared Slices. And from time to time I see that some slices are 16 bytes in size instead expected 8 bytes
Is this normal behavior or a bug?

I am using rocksdb 5.14.2 on Centos 6/7 and Debian 9+.
Almost identical code using leveldb did not have this behavior

Thanks

[jay-zhuang]

I don't think it's expected, are you able to share the code to reproduce the issue?

[PFedorovMsk]

In my custom comparaotor:

template<typename NumT, typename BaseComparatorT, typename SliceT>
void NumericComparator<NumT, BaseComparatorT, SliceT>::checkSize(const SliceT &slice) const
{
	if (slice.size() != sizeof(NumT))
	{
		...
		exit(2);
	}
}

template<typename NumT, typename BaseComparatorT, typename SliceT>
int NumericComparator<NumT, BaseComparatorT, SliceT>::Compare(const SliceT &a, const SliceT &b) const
{
	// ! >>>>>>
	// sometimes, size of a or/and b is 16 bytes instead expected 8 bytes (for long long)
	checkSize(a);
	checkSize(b);
	// ! <<<<<<

	NumT v1 = *(NumT *) a.data();
	NumT v2 = *(NumT *) b.data();

	if (v1 == v2) { return 0; }

	if (ascOrder) { return v1 < v2 ? -1 : 1; }
	else { return v1 < v2 ? 1 : -1; }
}

Usage:

using NumericComparatorRocksDb = NumericComparator<long long, rocksdb::Comparator, rocksdb::Slice>;

rocksdb::Options opts;
opts.create_if_missing = true;
opts.comparator = new NumericComparatorRocksDb(true);


if (!rdbWriter.open(cmdParams.getPathToRocksDb(), opts))
{
	std::cerr << logStr << "Failed to open DB " << cmdParams.getPathToRocksDb() << std::endl;
	return 1;
}

while (ldbReader.next())
{
	rocksdb::Slice key(ldbReader.key().data(), ldbReader.key().size());
	rocksdb::Slice value(ldbReader.value().data(), ldbReader.value().size());

        // Here I am checking the key and value sizes. They are ALWAYS 8 bytes

	if (!rdbWriter.put(key, value))
	{
		std::cerr << logStr << "Failed to write: " <<  ...;
	}
}

rdbWriter.close();

[mrambacher]

My guess is that sometimes you are being (erroneously) passed an InternalKey rather than a UserKey.

Do you know what under which circumstances you get the larger key? Perhaps a stack trace/abort when you got a larger key might be helpful in debugging this.

[ajkr]

There was at least one bug where we passed internal keys to the user comparator. The one I remember was #4575. I'd suggest upgrading (preferably) or going through bug fixes in HISTORY.md to see if there's any other fixes that are needed to run 5.14 safely.