1.1.4 doesn't write content size to frame header when using multi-threaded APIs

[indygreg]

Upon upgrading to 1.1.4, python-zstandard's test suite raised a few failures related to the content size not being written into the frame header when using the multi-threaded compression APIs. You can see the test failures by following links to Travis or AppVeyor in indygreg/python-zstandard#17.

I haven't tested the C code explicitly, but what python-zstandard is doing is roughly:

ZSTD_parameters zparams;
memset(&zparams, 0, sizeof(zparams));
destSize = ZSTD_compressBound(sourceSize);

zparams.cParams = ZSTD_getCParams(3, sourceSize, 0);
zparams.fParams.contentSizeFlag = 1;

cctx = ZSTDMT_createCCtx(threads);
ZSTDMT_compressCCtx(cctx, dest, destSize, source, sourceSize, 3);

In 1.1.3, the content size was written into the frame header. In 1.1.4, it isn't. This feels like a regression.

The same behavior occurs with ZSTDMT_initCStream_advanced() + ZSTDMT_compressStream(). So this feels like something inside ZSTDMT functions not looking at fParams. That, or python-zstandard was doing something wrong before and zstd changed behavior to expose a bug. Either way, it feels like an issue worth reporting.

[Cyan4973]

Thanks for notification @indygreg . This looks like a regression.
ZSTDMT_compressCCtx() should always include the frame content size in header.
I'll look into it.

Edit :
Test reproduced, and added to CI test suite

[Cyan4973]

Just created PR #637 with a short patch to fix this issue.
The patch also improves the automated test suite to check this condition from now on.