optimize-for-streams-detection

[ByiProX]

1. Optimize repeated images detection when loads streams.
2. Add option arg skip_step for stream detection instead of reading every 4th frame.

🛠️ PR Summary

_{Made with ❤️ by Ultralytics Actions}

🌟 Summary

Enhancement of the webcam stream processing in YOLOv5 by allowing customizable frame skipping.

📊 Key Changes

• Introduced a new skip_step option to specify how many frames to skip in a video stream before processing.
• In detect.py, LoadStreams now receives skip_step to control frame skipping.
• LoadStreams class in utils/datasets.py is modified to skip frames according to skip_step.
• Added a mechanism to prevent processing the exact same image frame repeatedly in webcam streams.

🎯 Purpose & Impact

• Allows Efficient Processing: Users can now fine-tune how often frames are processed from a stream, enabling a balance between performance and real-time detection needs. 🚀
• Prevents Redundant Calculations: By avoiding repeated processing of identical frames, it saves computational resources and time, benefiting environments where image scenes change infrequently. 💡
• Customizable for Different Use Cases: Applicable in various scenarios, from surveillance to live streaming, where users might prefer different frame rates for object detection. ⚙️

[glenn-jocher]

@ByiProX thanks for the PR!

The intended use-case for LoadStreams() is to connect to multiple cameras for batched inference, i.e. you have 8 RTSP cameras, you pass their addresses in a streams.txt file (one per line), and then detect.py will load them all asynchronously and run batched inference on them:

python detect.py --source streams.txt

This results in an incredibly fast multi-camera inference system, but a few issues are:

• Some cameras may suffer connectivity issues, so one image in the batch will be static, or fail to load, while the other cameras operate correctly.
• Queue buildup if the system is not realtime capable. The cv2 loader will load the previous frame of the RTSP stream, not the current frame. So for example if your connection is slow and you can only download 1 image per second, the inference stream will never be realtime, it will always be running inference on past frames.

Ideally we'd want a streamloader that checks for and pulls the current frame at a proposed framerate, does not build up a queue, and skips inference if the entire batch is identical as you propose here.

[glenn-jocher]

@ByiProX see also #2222 for a recent PR to patch a frame loss bug on multi-RTSP streaming.

[glenn-jocher]

/rebase

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.