Edge AI · 2026-08-25
The full pipeline from ONNX to RKNN, quantization accuracy loss, and real throughput at 1080p.
The client needed appearance-defect detection on a production line: 30 fps per camera, no network allowed. We chose the RV1126B — this note records the deployment.
PyTorch training → ONNX export → RKNN Toolkit2 quantization → on-device inference. Quantization is INT8 with a calibration set of 500 field images. The key point: the calibration set must include the site's actual lighting conditions — calibrating on public datasets costs 8-12 accuracy points.
Accuracy Loss
With field-image calibration, INT8 accuracy loss stays within 1 point — acceptable.
Measured Throughput
At 1080p input with YOLOv8n:
- Pure inference: 42 fps
- Including pre-processing (resize, normalize): 31 fps
- Including post-processing (NMS): 28 fps
We wrote the pre-processing on RGA hardware acceleration ourselves — 3× faster than OpenCV's CPU path. Sticking with OpenCV would have dropped throughput to ~15 fps.
RKNN input must be a fixed size. Dynamic shapes are unsupported — every frame is resized to the model input and detection boxes mapped back proportionally.
Cap the inference queue length. We started with an unbounded queue and watched memory climb to 1.2 GB in 20 minutes; a fixed queue of depth 4 holds steady at 300 MB.