triage sok function recall #98
Description
pick a testcase:
(env) user@hostname ~/c/l/r/e/SoK> python benchmark.py tee
lancelot vs SoK test suite
functions:
precision: 0.971
recall: 0.529
basic blocks:
precision: 0.990
recall: 0.813
instructions:
precision: 0.998
recall: 0.812
worst performing test cases:
-------- -----------------------------------
0.319658 SoK-windows-testsuite/cl_O2/tee
0.319658 SoK-windows-testsuite/cl_Ox/tee
0.320613 SoK-windows-testsuite/cl_O1/tee
0.32097 SoK-windows-testsuite/cl_Od/tee
0.737123 SoK-windows-testsuite/cl_m32_O2/tee
0.737123 SoK-windows-testsuite/cl_m32_Ox/tee
0.738281 SoK-windows-testsuite/cl_m32_O1/tee
0.738636 SoK-windows-testsuite/cl_m32_Od/tee
-------- -----------------------------------dump the functions:
python dump_ground_truth_report.py SoK-windows-testsuite/cl_O2/tee/tee.gt.json.gz | grep function | sort > /tmp/gt-functions.txt
python dump_lancelot_report.py SoK-windows-testsuite/cl_O2/tee/tee.exe | grep function | sort > /tmp/lan-functions.txt
diff:
diff /tmp/gt-functions.txt /tmp/lan-functions.txt | head -n 30 master
2d1
< function: 0x140001010
5,7d3
< function: 0x140001400
< function: 0x140001460
< function: 0x140001484
13,18d8
< function: 0x14000161c
< function: 0x140001630
< function: 0x1400017a8
< function: 0x1400017bc
< function: 0x1400017c4
< function: 0x1400017cc
21d10
< function: 0x14000184c
24d12
< function: 0x1400019d0
27,32d14
< function: 0x140001bc4
< function: 0x140001bdc
< function: 0x140001bfc
< function: 0x140001c08
< function: 0x140001c54
< function: 0x140001c84
34,40d15
< function: 0x140001ccc
< function: 0x140001d00
< function: 0x140001d18
< function: 0x140001d40
< function: 0x140001d58