Skip to content

Delegate CRC calculation to fastcrc when it is installed#1066

Merged
tridge merged 10 commits intoArduPilot:masterfrom
collmot:feat/crc-calculation-in-c
Jul 11, 2025
Merged

Delegate CRC calculation to fastcrc when it is installed#1066
tridge merged 10 commits intoArduPilot:masterfrom
collmot:feat/crc-calculation-in-c

Conversation

@ntamas
Copy link
Contributor

@ntamas ntamas commented Jun 28, 2025

This PR delegates the calculation of the CRC checksum in MAVLink messages to the crcmod-plus module when it is installed, falling back to the previous pure Python implementation if needed.

This PR delegates the calculation of the CRC checksum in MAVLink messages to the fastcrc module when it is installed, falling back to the previous pure Python implementation if needed.

In my benchmarks, with real MAVLink traffic from ~1000 drones (recorded by Wireshark), this boost the speed of the MAVLink Python parser from 54K packets per second to 100K packets per second.

Note that I have not declared crcmod-plus as a dependency (not even as an optional extra) yet because I don't know what the preferred solution would be. I would probably declare it in setup.py as an optional dependency and leave it up to the user to decide whether they are okay with an extra C module as a dependency or not. crcmod-plus provides wheels for all the major Python distributions and versions, with an ABI3 interface for future-proofing, so it is a low-risk dependency in my opinion, unlikely to cause any problems for end users of pymavlink.

fastcrc is now declared as a dependency in setup.py in the install_requires section.

@ntamas ntamas changed the title Delegate CRC calculation to crcmod-plus when it is installed Delegate CRC calculation to fastcrc when it is installed Jul 7, 2025
@ntamas
Copy link
Contributor Author

ntamas commented Jul 7, 2025

CI jobs are failing because mypy is explicitly prevented from using typing information from site-packages (where fastcrc is installed) so it cannot typecheck anything that involves fastcrc. Either we need to change the mypy config or provide type stubs for fastcrc (which have a slight chance to go out of sync with fastcrc itself in the future).

Copy link
Contributor

@tridge tridge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work!

@tridge
Copy link
Contributor

tridge commented Jul 9, 2025

Either we need to change the mypy config or provide type stubs for fastcrc

I'd be happy with either solution. Would crcmod-plus be any better for the type info?

@ntamas
Copy link
Contributor Author

ntamas commented Jul 9, 2025

No, both libraries are typed. The advantages of crcmod-plus would be 1) that it accepts any Sequence[int] out of the box, not only bytes (so we don't need to convert) and 2) that it provides ABI3 wheels so I don't have to rebuild the wheels if a newer minor version of Python is released (but this could be changed easily if one can convince the fastcrc authors to provide ABI3 wheels).

I'll modify the mypy config to look into site-packages for typings.

@ntamas
Copy link
Contributor Author

ntamas commented Jul 9, 2025

All green now.

@tridge tridge merged commit dc69e8b into ArduPilot:master Jul 11, 2025
20 checks passed
@ntamas ntamas deleted the feat/crc-calculation-in-c branch July 11, 2025 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants