how ncnn support yolov8n?

[superbayes]

detail | 详细描述 | 詳細な説明

1,下载官方yolov8n.pt

2,执行命令:

yolo task=detect mode=export model=yolov8n.pt format=onnx simplify=True opset=13

3,get yolov8n.onnx, convert to yolov8n.bin,yolov8.param

4,deploy failed,base on NCNN

希望nihui大佬能够在知乎写一篇教程,讲解如何使用yolov8n

@nihui
thanks a lot!

[bmharper]

I'm also looking into this.

So far, I'm seeing the following problem, when using the small yolov8s model, via onnx2ncnn:

Inside build/src/layer/x86/convolution_x86_fma.cpp, which seems to be generated at build time from https://github.com/Tencent/ncnn/blob/master/src/layer/x86/convolution_x86.cpp#L505:

This line is a problem:

Mat weight_data_r2 = weight_data.reshape(maxk, num_input, num_output);

weight_data_r2 ends up being empty, because reshape detects that maxk * num_input * num_output != the original number of elements.
This eventually crashes with a divide by zero.

I can't figure out yet what the root cause is.

[apanand14]

detail | 详细描述 | 詳細な説明

1,下载官方yolov8n.pt

2,执行命令:

yolo task=detect mode=export model=yolov8n.pt format=onnx simplify=True opset=13

3,get yolov8n.onnx, convert to yolov8n.bin,yolov8.param

4,deploy failed,base on NCNN

希望nihui大佬能够在知乎写一篇教程,讲解如何使用yolov8n

@nihui thanks a lot!

can you share your param file?

[superbayes]

detail | 详细描述 | 詳細な説明

1,下载官方yolov8n.pt

2,执行命令:

yolo task=detect mode=export model=yolov8n.pt format=onnx simplify=True opset=13

3,get yolov8n.onnx, convert to yolov8n.bin,yolov8.param

4,deploy failed,base on NCNN

希望nihui大佬能够在知乎写一篇教程,讲解如何使用yolov8n

@nihui thanks a lot!

can you share your param file?
yolov8n_my.zip

[bmharper]

Here is my YOLOv8 small param file:
y8s.zip

[glenn-jocher]

@superbayes @apanand14 @bmharper I have great news 😃! I've recently added official support for Ultralytics YOLOv8 NCNN export ✅ in PR ultralytics/ultralytics#3529 with the help of @nihui which is part of ultralytics==8.0.129. NCNN works for all tasks including Detect, Segment, Pose and Classify.

You can now export with CLI:

yolo export model=yolov8n.pt format=ncnn

or Python:

from ultralytics import YOLO

# Create a model
model = YOLO('yolov8n.pt')

# Export the model to NCNN with arguments
model.export(format='ncnn', half=True, imgsz=640)

Output is a yolov8n_ncnn_model/ directory containing model.bin, model.param and metadata.yaml, along with extra PNNX files. For details see https://github.com/pnnx/pnnx README.

To get this update:

• Git – Run git pull from within your ultralytics/ directory or run git clone https://github.com/ultralytics/ultralytics again
• Pip – Update with pip install -U ultralytics
• Notebooks – Check out the updated notebooks
• Docker – Run sudo docker pull ultralytics/ultralytics:latest to update your image

Please let us know if NCNN export is working correctly for you, and don't hesitate to report any other issues you find or feature requests you may have. Happy training with YOLOv8 🚀!

[bmharper]

Thanks for taking the time to work on this @glenn-jocher.
I'm finally getting around to trying it out, but I can't figure out where this is going wrong.

Here is a sample of the raw float32 output I'm seeing from layer "out0":

shape: 2100 84 1 1
box 0:  30.4 -30.0  79.4  91.6 0.0 0.0 0.0 0.0 ... 0.4 0.5 0.7 0.4 0.4 0.5 0.5 0.4
box 1:  37.9 -24.3  87.8  80.6 0.0 0.0 0.0 0.0 ... 0.5 0.5 1.0 0.4 0.4 0.5 0.5 0.5
box 2:  37.6 -24.6 105.2  80.7 0.0 0.0 0.0 0.0 ... 0.7 0.3 0.4 0.4 0.4 0.5 0.4 0.5
box 3:  34.4 -32.0 126.8  91.5 0.0 0.0 0.0 0.0 ... 0.3 0.5 0.3 0.4 0.5 0.3 0.5 0.6
box 4:  30.8 -33.8 149.4  95.7 0.0 0.0 0.0 0.0 ... 0.4 0.9 0.5 0.7 0.3 0.5 0.6 0.6
...

The steps I'm taking are:

1. Use yolo export model=yolov8n.pt format=ncnn imgsz=320 to get .param and .bin files for ncnn.
2. Run the model inside ncnn, on an image with one person inside it.
3. Dump the first 5 (out of 2100) boxes.

The reason I say this is not working, is this:

1. It looks like the format is something like XYWH+Classes , but all the sample code I've seen assume that XYWH are in the range 0..1. My numbers look more like integer pixel coordinates - definitely not unit coordinates.
2. At least one of the 2100 boxes should have a high probability for "person" (class 0). However the max 'person' probability I get is 0.002.

I can confirm that running yolo detect .... imgsz=320 on this image does indeed detect the person correctly.

Do the numbers that I'm getting make any sense to you? Could it be that the model is tuned for int8 output, and hence the integer pixel coordinates? Even if that was the case, it still doesn't explain why I'm not getting any "person" detections.

Thanks!

[bmharper]

I'm going to respond to my own question here, in case anybody else stumbles into this problem:

The outputs from the YOLOv8 model are shaped like this:

0     1     ...   2099
----------------------
x     x     ...
y     y     ...
w     w     ...
h     h     ...
p0    p0    ...
...   ...   ...
p79   p79   ...

The first candidate box is column 0. The second candidate box is column 1, etc. In my above example there are 2100 candidate boxes, but the exact number will depend on the imgsz parameter that you specified when running yolo export ....

The xywh coordinates are in pixels. When you export your model with yolo export ..., the imgsz parameter determines the input size of the NN model, and you can specify a non-square model with imgsz=[H,W]. X and Y are the box centers, and W and H are width and height.

The p0..p79 are the probabilities of each class (assuming the regular 80 class YOLO pretrained models). If you have a custom model, then there could obviously be more or less classes. I don't know whether it makes sense to apply a sigmoid function to the probabilities, but so far I am not applying a sigmoid, but just using them out of the box.

The example ncnn code for yolov7 is quite close to a working solution for YOLOv8. With YOLOv8 it's actually quite a bit simpler because there are no anchors. You just walk through all of the candidate boxes, picking out any box where max(p0..p79) is greater than some threshold probability (eg 0.25). Then you run NMS on those boxes, and finally accept any with some (probably higher) threshold probability (eg 0.5). The yolov7 example code that I linked here does all of this.

One more note: Before consuming the raw output of the YOLOv8 model, you'll want to transpose it. Each candidate box is 84 elements wide, so if your output is float32, that's a 336 byte stride between each element of the vector.

[nihui]

针对onnx模型转换的各种问题，推荐使用最新的pnnx工具转换到ncnn
In view of various problems in onnx model conversion, it is recommended to use the latest pnnx tool to convert your model to ncnn

pip install pnnx
pnnx model.onnx inputshape=[1,3,224,224]

详细参考文档
Detailed reference documentation
https://github.com/pnnx/pnnx
https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx#how-to-use-pnnx

[nihui]

yolov8 example #5506

[nihui]

hi, yolov8 examples are updated with full support for detection, segmentation, classification, pose estimation and obb
https://github.com/Tencent/ncnn/tree/master/examples

android demo
https://github.com/nihui/ncnn-android-yolov8

detailed instruction (zh)
https://zhuanlan.zhihu.com/p/16030630352