Missing last two Bytes when decompressing

[dns2utf8]

Hi all

I have a weird decompression bug with deflate. I isolated it to this buffer:

let hex_buffer = [0xaa, 0x56, 0x72, 0x4b, 0x2d, 0x49, 0xce, 0xf0, 0xc8, 0xcf, 0xcf, 0xf6, 0x4d, 0x2d, 0x49, 0x4c, 0x49, 0x2c, 0x49, 0x54, 0xb2, 0xaa, 0x56, 0xca, 0xc8, 0xcf, 0xcf, 0x56, 0xb2, 0x02, 0x53, 0xc5, 0xfa, 0x19, 0xa9, 0x39, 0x39, 0xf9, 0x4a, 0x3a, 0x4a, 0xd9, 0xa9, 0x95, 0x4a, 0x56, 0x4a, 0x89, 0x79, 0x95, 0x19, 0xf9, 0xe5, 0x4a, 0x00, 0x00, 0xff, 0xff];

  let mut decoder = read::DeflateDecoder::new(&*hex_buffer);

            let mut buf = Vec::new();
            let decoded_len = decoder
                .read_to_end(&mut buf)
                .context("unable to read compressed payload")?;

let s = String::from_utf8(buf).context("utf-8 error")?;
let expected =  "{\"FetchHookMetadata\":{\"hook\":\"hooks/hello\",\"key\":\"anyhow\"}}";

This should return this String, but it is missing the last two }}

Since this buffer will be coming over WebSocket I extended it with [ 0x00, 0x00, 0xff, 0xff ] but not doing that made no difference.

Hope anyone has an idea on why it just does not decompress the last two Bytes.
Thank you in advance,
Stefan

[fintelia]

How are you producing the compressed data? It doesn't actually include those final two bytes. Which either means the compressor wasn't flushed when writing the output, or if you're using this library, the recent flush changes might have broken it.

[Byron]

Does this mean that if you run cargo update -p miniz_oxide, that it will work?
Or do we need a new release of miniz_oxide as well? It seems that we could also undo the flush changes to have a fix out more quickly.

[fintelia]

No, there hasn't been a miniz_oxide release that includes Frommi/miniz_oxide#179. Might be best to undo the flush changes, get a new miniz_oxide release, and then re-add the flush changes and #498 (which adds unit tests to make sure flushes work...) at the same time.

[Byron]

I don't know what I was thinking, given that the PR is still in draft and failing CI - it's not even merged in yet.
It should have been clear to me that that the PR here depends on the one in miniz_oxide.

Oh well, @fintelia can you come up with a PR that reverts the flush changes and bumps the patch level of this crate? Then I can merge it and create a new release. I'd do it myself, but can't self-merge here.

Thanks a million.

[Byron]

Thanks everyone for the help, particularly @fintelia for the rapid emergency response :).

Please let me close this issue in anticipation that the new release fixes it.

If it doesn't, please do let me know.