Skip to content

Switch to sampling profiler for driver profiling #1399

@pquentin

Description

@pquentin

Profiling the load driver is important to make sure there's no accidental client bottleneck. Rally currently uses yappi for this: https://esrally.readthedocs.io/en/latest/command_line_reference.html#clr-enable-driver-profiling.

My understanding is that yappi is basically cProfile with asyncio support. It's a deterministic tracing profiler, which means that every single function call is recorded. In practice, just like cProfile, it's going to tell us functions are slow because the profiler is making them slow. This is why cProfile is basically a bad practice in Python.

We should use a sampling profiler instead, like vmprof or py-spy. vmprof appears to be useful for async/await code: python-trio/trio#943. It would be nice if profiling stayed as easy as it is now, but producing useful results is even more important!

Metadata

Metadata

Assignees

No one assigned

    Labels

    :Load DriverChanges that affect the core of the load driver such as scheduling, the measurement approach etc.enhancementImproves the status quohelp wantedWe'd be happy about a community contribution

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions