RKNN Model Zoo Example Deployment

1. rknn_model_zoo Example Workflow

rknn_model_zoo is an official repository provided by Rockchip, containing deployment examples of various mainstream AI algorithms supported by the RKNPU. The latest examples include deployments for models such as MobileNet and YOLO. In this chapter, YOLOv10 is used as an example to demonstrate how to use the rknn_model_zoo. This workflow demonstrates the complete process from ONNX model conversion to inference on the development board, covering key steps such as model conversion, deployment, and execution.

On the device side, two inference approaches are available:

• High-performance inference using a C program
• Rapid development and validation using Python (RKNN-Toolkit-Lite2)

2. Convert ONNX Model to RKNN in RKNN-Toolkit2 Environment

1. Download rknn_model_zoo.

   git clone https://github.com/airockchip/rknn_model_zoo.git

2. Download the YOLOv10 ONNX model.

   cd <rknn_model_zoo Path>/rknn_model_zoo/examples/yolov10/model
   chmod a+x download_model.sh
   ./download_model.sh

3. Convert the model to RKNN format. Run the conversion script convert.py located in the rknn_model_zoo/examples/yolov10/python directory.

   conda activate RKNN-Toolkit2
   cd <rknn_model_zoo Path>/rknn_model_zoo/examples/yolov10/python
    python3 convert.py ../model/yolov10n.onnx rv1126b
   # output model will be saved as ../model/yolov10.rknn

   python3 convert.py <onnx_model> <TARGET_PLATFORM> <dtype(optional)> <output_rknn_path(optional)>

   Parameter Description:

   • <onnx_model>: Path to the ONNX model
   • <TARGET_PLATFORM>: Target NPU platform (e.g., rv1126b)
   • <quant_dtype> : Optional. Can be set to i8 or fp
     • i8: Enable quantization (default)
     • fp: Disable quantization
   • <output_rknn_path>: Optional. Specifies the output path of the RKNN model. By default, the model is saved in the same directory as the ONNX model with the name yolov10n.rknn.

3. Model Inference (C)

3.1 Cross-compiling the C Inference Demo on x86_64 Linux

1. After successfully converting the ONNX model to an RKNN model, the next step is to cross-compile the demo located in the rknn_model_zoo/examples/yolov10 directory. Before building, set the following environment variable:

   export GCC_COMPILER=/home/luckfox/Luckfox/rv1126b/sdk-251224/tools/linux/toolchain/aarch64-rockchip1240-linux-gnu/bin/aarch64-rockchip1240-linux-gnu

2. Run the build-linux.sh script in the rknn_model_zoo directory. This script will compile the demo:

   chmod +x ./build-linux.sh
   ./build-linux.sh -t rv1126b -a aarch64 -d yolov10

   • Build process:

        (RKNN-Toolkit2) luckfox@xt:~/conda/rknn_model_zoo$ ./build-linux.sh -t rv1126b -a aarch64 -d yolov10
        ./build-linux.sh -t rv1126b -a aarch64 -d yolov10
        /home/luckfox/Luckfox/rv1126b/sdk-251224/tools/linux/toolchain/aarch64-rockchip1240-linux-gnu/bin/aarch64-rockchip1240-linux-gnu
        ===================================
        BUILD_DEMO_NAME=yolov10
        BUILD_DEMO_PATH=examples/yolov10/cpp
        TARGET_SOC=rv1126b
        TARGET_ARCH=aarch64
        BUILD_TYPE=Release
        ENABLE_ASAN=OFF
        DISABLE_RGA=OFF
        DISABLE_LIBJPEG=OFF
        INSTALL_DIR=/home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo
        BUILD_DIR=/home/luckfox/conda/rknn_model_zoo/build/build_rknn_yolov10_demo_rv1126b_linux_aarch64_Release
        CC=/home/luckfox/Luckfox/rv1126b/sdk-251224/tools/linux/toolchain/aarch64-rockchip1240-linux-gnu/bin/aarch64-rockchip1240-linux-gnu-gcc
        CXX=/home/luckfox/Luckfox/rv1126b/sdk-251224/tools/linux/toolchain/aarch64-rockchip1240-linux-gnu/bin/aarch64-rockchip1240-linux-gnu-g++
        ===================================
        -- The C compiler identification is GNU 12.4.0
        -- The CXX compiler identification is GNU 12.4.0
        -- Detecting C compiler ABI info
        -- Detecting C compiler ABI info - done
        -- Check for working C compiler: /home/luckfox/Luckfox/rv1126b/sdk-251224/tools/linux/toolchain/aarch64-rockchip1240-linux-gnu/bin/aarch64-rockchip1240-linux-gnu-gcc - skipped
        -- Detecting C compile features
        -- Detecting C compile features - done
        -- Detecting CXX compiler ABI info
        -- Detecting CXX compiler ABI info - done
        -- Check for working CXX compiler: /home/luckfox/Luckfox/rv1126b/sdk-251224/tools/linux/toolchain/aarch64-rockchip1240-linux-gnu/bin/aarch64-rockchip1240-linux-gnu-g++ - skipped
        -- Detecting CXX compile features
        -- Detecting CXX compile features - done
        -- !!!!!!!!!!!CMAKE_SYSTEM_NAME: Linux
        -- Looking for pthread.h
        -- Looking for pthread.h - found
        -- Performing Test CMAKE_HAVE_LIBC_PTHREAD
        -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
        -- Found Threads: TRUE
        -- Configuring done
        -- Generating done
        -- Build files have been written to: /home/luckfox/conda/rknn_model_zoo/build/build_rknn_yolov10_demo_rv1126b_linux_aarch64_Release
        [ 16%] Building C object utils.out/CMakeFiles/imageutils.dir/image_utils.c.o
        [ 16%] Building C object utils.out/CMakeFiles/imagedrawing.dir/image_drawing.c.o
        [ 25%] Building C object utils.out/CMakeFiles/audioutils.dir/audio_utils.c.o
        [ 33%] Building C object utils.out/CMakeFiles/fileutils.dir/file_utils.c.o
        [ 41%] Linking C static library libfileutils.a
        ...
        ...
        ...
        [100%] Linking CXX executable rknn_yolov10_demo
        [100%] Built target rknn_yolov10_demo
        Consolidate compiler generated dependencies of target imagedrawing
        [ 16%] Built target imagedrawing
        Consolidate compiler generated dependencies of target imageutils
        [ 33%] Built target imageutils
        Consolidate compiler generated dependencies of target fileutils
        [ 50%] Built target fileutils
        Consolidate compiler generated dependencies of target rknn_yolov10_demo
        [ 83%] Built target rknn_yolov10_demo
        Consolidate compiler generated dependencies of target audioutils
        [100%] Built target audioutils
        Install the project...
        -- Install configuration: "Release"
        -- Installing: /home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/./rknn_yolov10_demo
        -- Set runtime path of "/home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/./rknn_yolov10_demo" to "$ORIGIN/../lib"
        -- Installing: /home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/model/bus.jpg
        -- Installing: /home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/model/coco_80_labels_list.txt
        -- Installing: /home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/model/yolov10.rknn
        -- Installing: /home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/lib/librknnrt.so
        -- Installing: /home/luckfox/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo/lib/librga.so

3. After cross-compilation is completed, an install directory will be generated under the rknn_model_zoo path. This directory contains the compiled binaries and required library files.

   (RKNN-Toolkit2) ubuntu@ubuntu:~/conda/rknn_model_zoo/install/rv1126b_linux_aarch64/rknn_yolov10_demo$ ls
   lib  model  rknn_yolov10_demo

3.2 Deploy the install directory to the Luckfox Aura board for inference

1. Transfer the entire rknn_yolov10_demo directory to the development board, then run the following command:

   scp -r rv1126b_linux_aarch64/rknn_yolov10_demo/ luckfox@192.168.253.105:/home/aura
   ./rknn_yolov10_demo ./model/yolov10.rknn ./model/bus.jpg

2. After inference is completed, an output image out.png will be generated.

   (base) root@aura:/home/aura/rv1126b_linux_aarch64/rknn_yolov10_demo# ./rknn_yolov10_demo ./model/yolov10.rknn ./model/bus.jpg
    load lable ./model/coco_80_labels_list.txt
    model input num: 1, output num: 6
    input tensors:
    index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
    output tensors:
    index=0, name=470, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-38, scale=0.114574
    index=1, name=484, n_dims=4, dims=[1, 80, 80, 80], n_elems=512000, size=512000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.002001
    index=2, name=491, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-57, scale=0.095044
    index=3, name=505, n_dims=4, dims=[1, 80, 40, 40], n_elems=128000, size=128000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003505
    index=4, name=512, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-58, scale=0.061253
    index=5, name=526, n_dims=4, dims=[1, 80, 20, 20], n_elems=32000, size=32000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003792
    model is NHWC input fmt
    model input height=640, width=640, channel=3
    origin size=640x640 crop size=640x640
    input image: 640 x 640, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
    scale=1.000000 dst_box=(0 0 639 639) allow_slight_change=1 _left_offset=0 _top_offset=0 padding_w=0 padding_h=0
    rga_api version 1.10.1_[0]
    184 im2d_rga_impl rga_get_info(718): Can not get the correct RGA version, please check the driver, version=4.1.34669
   
    184 im2d_rga_impl rga_check(1280): rga im2d: rga2 get info failed!
   
    Error on improcess STATUS=0
    RGA error message: Fatal error: rga im2d: rga2 get info failed!
   
    try convert image use cpu
    finish
    rknn_run
    bus @ (88 137 556 437) 0.937
    person @ (109 234 226 536) 0.906
    person @ (210 240 284 511) 0.899
    person @ (478 233 560 519) 0.800
    person @ (79 331 114 518) 0.351
    write_image path: out.png width=640 height=640 channel=3 data=0x7f864a7010
    (base) root@aura:/home/aura/rv1126b_linux_aarch64/rknn_yolov10_demo# ls
    lib  model  out.png  rknn_yolov10_demo

   • Inference Result:

     

     

4. Model Inference (Python)

1. Copy the rknn_model_zoo/examples/yolov10 directory to the development board. Then navigate to the model directory and implement a Python inference script based on RKNN-Toolkit-Lite2.

   import numpy as np
   import cv2
   from rknnlite.api import RKNNLite
   
   MODEL_PATH = 'yolov10.rknn'
   IMG_PATH = 'bus.jpg'
   IMG_SIZE = 640
   OBJ_THRESH = 0.65
   NMS_THRESH = 0.35
   
   CLASSES = [
       'person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
       'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow',
       'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
       'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard',
       'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple',
       'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
       'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone',
       'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear',
       'hair drier', 'toothbrush'
   ]
   
   def sigmoid(x):
       return 1 / (1 + np.exp(-np.clip(x, -20, 20)))
   
   def softmax(x, axis=1):
       x_exp = np.exp(x - np.max(x, axis=axis, keepdims=True))
       return x_exp / np.sum(x_exp, axis=axis, keepdims=True)
   
   def main():
       rknn = RKNNLite()
   
       print(f"load rknn model: {MODEL_PATH}")
       if rknn.load_rknn(MODEL_PATH) != 0:
           print("load rknn failed!"); return
       if rknn.init_runtime() != 0:
           print("init runtime failed!"); return
   
       try:
           input_attrs = rknn.get_sdk_get_inputs_attrs()
           output_attrs = rknn.get_sdk_get_outputs_attrs()
           print(f"model input num: {len(input_attrs)}, output num: {len(output_attrs)}")
           print("input tensors:")
           for i, a in enumerate(input_attrs):
               print(f"  index={i}, name={a['name']}, dims={a['dims']}, zp={a['zp']}, scale={a['scale']:.6f}")
           print("output tensors:")
           for i, a in enumerate(output_attrs):
               print(f"  index={i}, name={a['name']}, dims={a['dims']}, zp={a['zp']}, scale={a['scale']:.6f}")
       except:
           pass
   
       img_src = cv2.imread(IMG_PATH)
       h_ori, w_ori = img_src.shape[:2]
       print(f"input image: {w_ori} x {h_ori}")
   
       img = cv2.resize(img_src, (IMG_SIZE, IMG_SIZE))
       img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
   
       print("rknn_run")
       outputs = rknn.inference(inputs=[np.expand_dims(img, 0)])
   
       reg_layers = [o for o in outputs if o.shape[1] == 64]
       cls_layers = [o for o in outputs if o.shape[1] == 80]
       reg_layers.sort(key=lambda x: x.shape[2], reverse=True)
       cls_layers.sort(key=lambda x: x.shape[2], reverse=True)
   
       strides = [8, 16, 32]
       boxes_all, scores_all, ids_all = [], [], []
   
       for i in range(len(reg_layers)):
           reg = reg_layers[i].astype(np.float32)
           cls = cls_layers[i].astype(np.float32)
           stride = strides[i]
   
           scores = sigmoid(cls)
           b, c, h, w = reg.shape
           reg_dist = reg.reshape(1, 4, 16, h, w)
           reg_dist = softmax(reg_dist, axis=2)
           dist = (reg_dist * np.arange(16).reshape(1, 1, 16, 1, 1)).sum(2)
   
           grid_y, grid_x = np.indices((h, w))
           grid = np.stack([grid_x, grid_y], axis=0).astype(np.float32)
   
           x1 = (grid[0:1] + 0.5 - dist[:, 0:1]) * stride
           y1 = (grid[1:2] + 0.5 - dist[:, 1:2]) * stride
           x2 = (grid[0:1] + 0.5 + dist[:, 2:3]) * stride
           y2 = (grid[1:2] + 0.5 + dist[:, 3:4]) * stride
   
           box = np.concatenate([x1, y1, x2, y2], axis=1).transpose(0, 2, 3, 1).reshape(-1, 4)
           prob = scores.transpose(0, 2, 3, 1).reshape(-1, 80)
   
           max_s = np.max(prob, axis=1)
           max_id = np.argmax(prob, axis=1)
   
           mask = max_s > OBJ_THRESH
           for idx in np.where(max_id == 10)[0]:
               if max_s[idx] < 0.8:
                   mask[idx] = False
   
           keep = np.where(mask)
           if len(keep[0]) > 0:
               boxes_all.append(box[keep])
               scores_all.append(max_s[keep])
               ids_all.append(max_id[keep])
   
       if boxes_all:
           boxes_all = np.concatenate(boxes_all)
           scores_all = np.concatenate(scores_all)
           ids_all = np.concatenate(ids_all)
   
           indices = cv2.dnn.NMSBoxes(boxes_all.tolist(), scores_all.tolist(), OBJ_THRESH, NMS_THRESH)
   
           sw, sh = w_ori / IMG_SIZE, h_ori / IMG_SIZE
           if len(indices) > 0:
               for i in indices.flatten():
                   b = boxes_all[i]
                   x1, y1 = int(b[0]*sw), int(b[1]*sh)
                   x2, y2 = int(b[2]*sw), int(b[3]*sh)
                   cls_id = int(ids_all[i])
                   label = CLASSES[cls_id] if cls_id < len(CLASSES) else f"ID:{cls_id}"
                   conf = scores_all[i]
   
   
                   print(f"{label} @ ({x1} {y1} {x2} {y2}) {conf:.3f}")
   
                   cv2.rectangle(img_src, (x1, y1), (x2, y2), (0, 255, 0), 2)
                   text = f"{label} {conf:.2f}"
                   t_size = cv2.getTextSize(text, cv2.FONT_HERSHEY_SIMPLEX, 0.5, 2)[0]
                   cv2.rectangle(img_src, (x1, y1 - t_size[1] - 5), (x1 + t_size[0], y1), (0, 255, 0), -1)
                   cv2.putText(img_src, text, (x1, y1 - 5),
                               cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 0), 1, lineType=cv2.LINE_AA)
   
   
       cv2.imwrite("out.png", img_src)
       print(f"write_image path: out.png width={w_ori} height={h_ori}")
       rknn.release()
   
   if __name__ == '__main__':
       main()
   

2. Ensure that the RKNN-Toolkit-Lite2 runtime environment is properly configured on the board, and execute the following command to run the program:

   conda activate rknn-toolkit-lite2
   python test.py

   (rknn-toolkit-lite2) root@aura:/home/aura/test/yolov10/model# python test.py
    load rknn model: yolov10.rknn
    I RKNN: [10:33:58.012] RKNN Runtime Information, librknnrt version: 2.3.2 (429f97ae6b@2025-04-09T09:09:27)
    I RKNN: [10:33:58.012] RKNN Driver Information, version: 0.9.8
    I RKNN: [10:33:58.014] RKNN Model Information, version: 6, toolkit version: 2.3.2(compiler version: 2.3.2 (e045de294f@2025-04-07T19:48:25)), target: RKNPU f3, target platform: rv1126b, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
    W RKNN: [10:33:58.040] query RKNN_QUERY_INPUT_DYNAMIC_RANGE error, rknn model is static shape type, please export rknn with dynamic_shapes
    W Query dynamic range failed. Ret code: RKNN_ERR_MODEL_INVALID. (If it is a static shape RKNN model, please ignore the above warning message.)
    input image: 640 x 640
    rknn_run
    bus @ (88 137 556 437) 0.718
    person @ (109 234 226 536) 0.712
    person @ (210 240 284 511) 0.711
    person @ (478 233 560 519) 0.690
    write_image path: out.png width=640 height=640

   • Inference Result:

     

5. Summary

Python (RKNN-Toolkit-Lite2) Solution: Suitable for rapid validation and functional debugging. It offers high development efficiency and can be easily integrated with tools such as OpenCV to build the inference pipeline. However, it depends on the Python runtime environment, and preprocessing is mainly performed on the CPU, resulting in relatively limited overall performance and real-time capability.

C/C++ (Native API) Solution: More suitable for production deployment and high-performance scenarios. By directly invoking low-level RKNN APIs and leveraging hardware acceleration (such as RGA), it enables efficient inference and preprocessing with lower resource usage and better real-time performance. However, development and debugging are more complex, making it better suited for final product deployment.