lfi-bench/TODO at master · lfi-project/lfi-bench · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
Cleanup/Finish
--------------

-need aac decode benchmark.
-add support for new targets to consolidate and plot.

-add x86-64 support for lfi
-skia and libvpx are using different build systems so arch is hard coded and they don't play
nice with everythng else. maybe vcpkg can help move everything to cmake.

-explore moving from git clones to vcpkg for grabbing code and installing.
-get get libvpx and dav1d onto cmake.
-cleanup how skia builds benchmarks
-fix skia, maybe compiler update broke?
-fix libvpx, benchmark code changed.

-move to vendoring benchmarks so they don't break when benchmarks change, can periodically refresh.
-do a full run to check that runtime updates work.
-expat benchmark runs too long and is not representative, replace it.
-libpng benchmark is dumb and runs too long.
-add ioctl support in lfi runtime for brotli, finish incorporating brotli.
-libwepb runs too long.
-check that inputs are large enough that user space costs are and not being amoritized.

-zstd decompress benchmark has too much system time, not enough user, needs to be fixed. compress runs to long, need to fix.
-should check zlib also.


Microbenchmarks
---------------
-bulk memory operations.

Move to LFI_ROOT
---------------
make all tool paths relative to LFI_ROOT
if enviroment variable set, use that, otherwise default to LFI_ROOT in lfi-bench
add script to pull and uncompress latest lfi-release.

Migrate to New runtime
----------------------
-include runtime as git submodule
-change lfi-run calls to refer to locally installed

Integration Test Support:
------------------------
-test if build fails
-input validation (test that output is correct) - make validate
-run verifier (test that binaries verify without errors) - make verify (for compiler development)
-lfi-bind tests (maybe in lfi-bind instead?)
-add sanitizer tests
-add PAC/MTE tests

Setup Github workflow to run once a day/on demand
-----------------------------------


Next
-----

ffmpeg (lots of formats)

other
-----
libsqlite -- ships with android
freetype -- part of AOSP

image processing
----------------
opencv -- doesn't ship with android, popular with apps
neven  (face detector)- part of AOSP

parsing
-------
libxstl, tinyxml -- AOSP

compression:
-------------

liblzma, bzip2, liblz4, minizip,snappy,libarchive

codecs
--------
libvips -- does dynamic code generation (liborc) and multithreading, will require some extra work
sonivox