iOS iDetection Speed Table

[glenn-jocher]

🚀 Feature

We'd like to create an iDetection Speed Table for each year's iPhone release, going back as far as iDetection will run, which may be around iPhone 5 or 6. Crowd-sourcing the table will help show real-world performance of the models on various hardware.

If you have an iOS device, you can contribute by downloading iDetection, testing your speeds, and then uploading your results here. We try to test near 100% battery at room temperature (about 25°C), and record the speed after about 10 seconds of inference for each model.

Please help by uploading your results! Thank you!!

iDetection v7.8 Inference Speeds

	Year	ASIC -process	ANE (TOPS)		YOLOv5s (ms)	YOLOv5m (ms)	YOLOv5l (ms)	YOLOv5x (ms)
iPhone 6	2014	A8-20nm	-		90	180	350	500
iPhone 6s	2015	A9-16nm	-		148	216	304	475
iPhone 7	2016	A10-16nm	-		94.7	140.7	216.7	289.8
iPhone 8/X	2017	A11-10nm	0.6		-	-	-	-
iPhone XR/XS	2018	A12-7nm	5.0		22.3	25.8	43.2	57.7
iPhone 11	2019	A13-7nm	6.0		17.4	21.3	27.8	41.2
iPhone 12	2020	A14-5nm	11.0		14.3	16.5	21.0	28.8
iPhone 13	2021	A15-5nm	15.8

*CoreML models exported as FP8 320x192 with release v3.1
*Measured with iDetection v7.8 at 100% battery at 25°C. Average speed after 10 seconds recorded.

[NanoCode012]

Hi Glenn, I got an old 7Plus. I think the results will definitely be lower than a new one.

5s - 10.4 FPS - 94.7 ms
5m - 7.1 FPS - 140.7 ms
5l - 4.6 FPS - 216.7 ms
5x - 3.7 FPS - 289.8 ms

Started from 100%. Default Params.

[glenn-jocher]

@NanoCode012 great, thanks bud! I've added your data points to the table.

[glenn-jocher]

Here are the results of a recent run for iDetection v7.8 YOLOv5s iPhone 11 for 20 minutes, starting from 94% battery at room temperature. The x axis is frame count. About 36000 frames were processed over 1200s (20 minutes), equal to about 30 FPS. You can see initial speed of 17 ms matches table numbers for this device, with thermal throttling starting at about 10 minutes, with iOS automatically reducing processing capacity to about 60% original, resulting in inference times of 27 ms. I don't track memory or thermal state, I'll try to add them in the future. But according to the below charts, everything seems to be working well. It's also possible that longer runs will result in more throttling, so I'll try a longer run to 60 minutes in the future also.

[glenn-jocher]

Additional profiling information for iDetection v7.8 YOLOv5s iPhone 11 run. Most importantly, RAM usage is steady over time and very low at about 100 MB out of 4 GB.

RAM Usage

CPU Usage

Energy Impact

[glenn-jocher]

Updated results for iDetection v7.8 YOLOv5s and YOLOv5m on iPhone 11 for 60 minutes of continuous usage (about 120,000 images processed each). Plots include system RAM usage over time, indicating very small app memory usage (only 70 MB for YOLOv5s and 80 MB for YOLOv5m! :). Conclusion: iPhone 11 is able to run YOLOv5s/m steady state at or near realtime 30 FPS, even after thermal throttling begins around 15 minutes into usage. Results are excellent. Plan to reproduce same test on iPhone 12 soon.

EDIT: Raw data uploaded below for above plots.
frames_iPhone11_YOLOv5s.txt
frames_iPhone11_YOLOv5m.txt