-
Notifications
You must be signed in to change notification settings - Fork 330
Switch to sampling profiler for driver profiling #1399
Description
Profiling the load driver is important to make sure there's no accidental client bottleneck. Rally currently uses yappi for this: https://esrally.readthedocs.io/en/latest/command_line_reference.html#clr-enable-driver-profiling.
My understanding is that yappi is basically cProfile with asyncio support. It's a deterministic tracing profiler, which means that every single function call is recorded. In practice, just like cProfile, it's going to tell us functions are slow because the profiler is making them slow. This is why cProfile is basically a bad practice in Python.
We should use a sampling profiler instead, like vmprof or py-spy. vmprof appears to be useful for async/await code: python-trio/trio#943. It would be nice if profiling stayed as easy as it is now, but producing useful results is even more important!