retdec-fileinfo allocates 4 GB to read the debug directory of a 2.7 MB file

[s3rvac]

retdec-fileinfo allocates 4 GB to read the debug directory of a 2.7 MB file.

Steps to reproduce

$ /usr/bin/time -v retdec-fileinfo FILE

where FILE is 022AFB79C539B23AECFAD315C7D9A712D747B521952184446026F7F2AA9005B8.

Expected output

retdec-fileinfo allocates a reasonable amount of memory to analyze the above 2.7 MB file.

Actual output

[..]
Maximum resident set size (kbytes): 4211072
[..]

which is around 4 GB of used RAM.

Analysis

Here is an output from valgrind --tool=massif (a heap profiler). Below, you can also find a screenshot from massif-visualizer:

As can be seen from the output, in DebugDirectory::read(), a std::vector is resized to 4 GB. This happens in src/pelib/DebugDirectory.cpp on line 49:

47             // Load the debug info data from the file, not from the image.
48             // Some samples may have debug info part of the overlay
49             debugEntry.data.resize(debugEntry.idd.SizeOfData);

For the above file, debugEntry.idd.SizeOfData is 4294967295 (UINT_MAX).

Notes

• I suggest verifying whether there is an overflow (the value would suggest that).
• When fixing the issue, please add the file into our regression-tests suite. We can just check that the analysis finishes successfully. We will at least have this file in the test suite.

My configuration

• Current master (commit 956b1a3)
• 64b Debian, GCC 9, Debug build

[ladislav-zezula]

#825
avast/retdec-regression-tests#67