Cannot parse file header with uncompressed size, compressed size, or local file header offset of 0xffffffff, if not in Zip64 format #109
Description
This issue is a companion to #108, but affects a different part of the parser. yauzl 2.10.0 cannot parse zip files whose uncompressedSize, compressedSize, or relativeOffsetOfLocalHeader is 0xffffffff, unless the file is in Zip64 format. The relevant code is here:
Lines 333 to 337 in 02a5ca6
When yauzl sees a 0xffffffff value in one of these fields, it assumes that the zip file must be in Zip64 format. But APPNOTE.TXT 4.4.8, 4.4.9, and 4.4.16 say (emphasis mine):
If an archive is in ZIP64 format and the value in this field is 0xFFFFFFFF, the size will be in the corresponding 8 byte ZIP64 extended information extra field.
The way I interpret this statement, the logic should not be what yauzl does now:
if a value is 0xffffffff:
parse the Zip64 extended information extra field
but should instead be:
if a Zip64 extended information extra field is found:
replace only the values that are 0xffffffff
This issue is basically the same as golang/go#31692. The difference is that Go archive/zip has a special-case workaround for the compressed size, but not the other two fields.
The test cases which follow are able to be parsed by Info-ZIP UnZip (unzip) and Python zipfile (python3 -m zipfile -e). Go archive/zip can parse the uncompressed size test but not the other two.
Uncompressed size test case
ffffffff-uncompressedSize.zip.gz (remove 1 layer of gzip before testing)
# 65535 * 65537 = 0xffffffff
dd if=/dev/zero bs=65535 count=65537 of=pad
touch -d '2019-05-01 00:00:00 UTC' pad
rm -f ffffffff-uncompressedSize.zip
TZ=UTC zip -9 -X ffffffff-uncompressedSize.zip pad
gzip -9 -k ffffffff-uncompressedSize.zip
zipinfo -v says:
uncompressed size: 4294967295 bytes
yauzl does not parse it:
events.js:183
throw er; // Unhandled 'error' event
^
Error: expected zip64 extended information extra field
at node_modules/yauzl/index.js:347:46
at node_modules/yauzl/index.js:631:5
at node_modules/fd-slicer/index.js:32:7
at FSReqWrap.wrapper [as oncomplete] (fs.js:658:17)
Compressed size test case
It's difficult to construct a test case that has compressedSize = 0xffffffff and uncompressedSize < 0xffffffff. It could plausibly happen if the compressor implements "store" mode not as compressionMethod 0, but as compressionMethod 8 (DEFLATE) and non-compressed blocks, which increase the compressed size slightly. The example here instead has compressedSize = uncompressedSize = 0xffffffff.
ffffffff-compressedSize.zip.gz.gz (remove 2 layers of gzip before testing)
# 65535 * 65537 = 0xffffffff
dd if=/dev/zero bs=65535 count=65537 of=pad
touch -d '2019-05-01 00:00:00 UTC' pad
rm -f ffffffff-compressedSize.zip
TZ=UTC zip -0 -X ffffffff-compressedSize.zip pad
zipinfo -v says:
compressed size: 4294967295 bytes
uncompressed size: 4294967295 bytes
yauzl does not parse it:
events.js:183
throw er; // Unhandled 'error' event
^
Error: expected zip64 extended information extra field
at node_modules/yauzl/index.js:347:46
at node_modules/yauzl/index.js:631:5
at node_modules/fd-slicer/index.js:32:7
at FSReqWrap.wrapper [as oncomplete] (fs.js:658:17)
Local file header offset test case
ffffffff-relativeOffsetOfLocalHeader.zip.gz.gz (remove 2 layers of gzip before testing)
# 216186 * 19867 = 0xffffffff - len("pad") - 30
dd if=/dev/zero bs=216186 count=19867 of=pad
echo test > test.txt
touch -d '2019-05-01 00:00:00 UTC' pad test.txt
rm -f ffffffff-relativeOffsetOfLocalHeader.zip
TZ=UTC zip -0 -X ffffffff-relativeOffsetOfLocalHeader.zip pad test.txt
zipinfo -v says:
offset of local header from start of archive: 4294967295
(00000000FFFFFFFFh) bytes
yauzl does not parse it:
events.js:183
throw er; // Unhandled 'error' event
^
Error: expected zip64 extended information extra field
at node_modules/yauzl/index.js:347:46
at node_modules/yauzl/index.js:631:5
at node_modules/fd-slicer/index.js:32:7
at FSReqWrap.wrapper [as oncomplete] (fs.js:658:17)