yolov8s-pose转ncnn

[Rachel-liuqr]

error log | 日志或报错信息 | ログ

model | 模型 | モデル

1. original model：yolov8s-pose.pt

how to reproduce | 复现步骤 | 再現方法

1.yolov8s-pose.pt转onnx后，再使用ncnn2onnx时报错：
Unsupported slice step!
ScatterND not supported yet！

2.然后尝试先转为torchscript
转torchscript代码如下：
from ultralytics import YOLO
model = YOLO("yolov8s-pose.pt") # load a pretrained model (recommended for training)
success = model.export(format="torchscript", simplify=True) # export the model to onnx format
assert success

再使用pnnx工具转模型，输入命令如下
./pnnx yolov8s-pose.torchscript inputshape=[1,3,640,640]
报错如下
pnnxparam = yolov8s-pose.pnnx.param
pnnxbin = yolov8s-pose.pnnx.bin
pnnxpy = yolov8s-pose_pnnx.py
pnnxonnx = yolov8s-pose.pnnx.onnx
ncnnparam = yolov8s-pose.ncnn.param
ncnnbin = yolov8s-pose.ncnn.bin
ncnnpy = yolov8s-pose_ncnn.py
fp16 = 1
optlevel = 2
device = cpu
inputshape = [1,3,640,640]f32
inputshape2 =
customop =
moduleop =
############# pass_level0
inline module = ultralytics.nn.modules.Bottleneck
inline module = ultralytics.nn.modules.C2f
inline module = ultralytics.nn.modules.Concat
inline module = ultralytics.nn.modules.Conv
inline module = ultralytics.nn.modules.DFL
inline module = ultralytics.nn.modules.Pose
inline module = ultralytics.nn.modules.SPPF
inline module = ultralytics.nn.modules.Bottleneck
inline module = ultralytics.nn.modules.C2f
inline module = ultralytics.nn.modules.Concat
inline module = ultralytics.nn.modules.Conv
inline module = ultralytics.nn.modules.DFL
inline module = ultralytics.nn.modules.Pose
inline module = ultralytics.nn.modules.SPPF

---

############# pass_level1
unknown Parameter value kind prim::Constant of TensorType, t.dim = 2
unknown Parameter value kind prim::Constant of TensorType, t.dim = 2
unknown Parameter value kind prim::Constant of TensorType, t.dim = 2
unknown Parameter value kind prim::Constant of TensorType, t.dim = 2
unknown Parameter value kind prim::Constant of TensorType, t.dim = 2
unknown Parameter value kind prim::Constant of TensorType, t.dim = 2
############# pass_level2
############# pass_level3
############# pass_level4
############# pass_level5
make_slice_expression input 206
make_slice_expression self 207
make_slice_expression src 209
make_slice_expression input 210
make_slice_expression self 214
make_slice_expression src 216
make_slice_expression input 217
make_slice_expression self 221
make_slice_expression src 223
############# pass_ncnn
force batch axis 233 for operand 212
force batch axis 233 for operand 219
force batch axis 233 for operand pnnx_expr_35_sub(212,5.000000e-01)
force batch axis 233 for operand pnnx_expr_9_sub(219,5.000000e-01)
binaryop broadcast across batch axis 0 and 233 is not supported
binaryop broadcast across batch axis 0 and 233 is not supported
binaryop broadcast across batch axis 0 and 233 is not supported
binaryop broadcast across batch axis 0 and 233 is not supported
binaryop broadcast across batch axis 0 and 233 is not supported
select along batch axis 0 is not supported
select along batch axis 0 is not supported
slice with step 3 is not supported
slice with step 3 is not supported
slice with step 3 is not supported
slice_copy with step 3 is not supported
slice_copy with step 3 is not supported
slice_copy with step 3 is not supported
ignore Crop select_0 param dim=0
ignore Crop select_0 param index=0
ignore Crop select_1 param dim=0
ignore Crop select_1 param index=1

技穷了，请大神赐教！！！

[triple-Mu]

https://zhuanlan.zhihu.com/p/622596922
可以参考这个知乎。

[Rachel-liuqr]

https://zhuanlan.zhihu.com/p/622596922 可以参考这个知乎。

谢谢三木，问题解决了！！！

[Rachel-liuqr]

问题已解决。
YOLOv8-pose在ncnn框架下部署过程记录（包含ncnn、pnnx安装以及模型结构更改及转换）
https://github.com/Rachel-liuqr/yolov8s-pose-ncnn
如果有帮助的话，麻烦star，谢谢！

[glenn-jocher]

@Rachel-liuqr I have great news 😃! I've recently added official support for Ultralytics YOLOv8 NCNN export ✅ in PR ultralytics/ultralytics#3529 with the help of @nihui which is part of ultralytics==8.0.129. NCNN works for all tasks including Detect, Segment, Pose and Classify.

You can now export with CLI:

yolo export model=yolov8n.pt format=ncnn

or Python:

from ultralytics import YOLO

# Create a model
model = YOLO('yolov8n.pt')

# Export the model to NCNN with arguments
model.export(format='ncnn', half=True, imgsz=640)

Output is a yolov8n_ncnn_model/ directory containing model.bin, model.param and metadata.yaml, along with extra PNNX files. For details see https://github.com/pnnx/pnnx README.

To get this update:

• Git – Run git pull from within your ultralytics/ directory or run git clone https://github.com/ultralytics/ultralytics again
• Pip – Update with pip install -U ultralytics
• Notebooks – Check out the updated notebooks
• Docker – Run sudo docker pull ultralytics/ultralytics:latest to update your image

Please let us know if NCNN export is working correctly for you, and don't hesitate to report any other issues you find or feature requests you may have. Happy training with YOLOv8 🚀!

[watertianyi]

@glenn-jocher
Although ncnn has been converted, there is a lack of examples of using c++ to call ncnn. Can this be provided, such as how to display the results predicted by ncnn?

[watertianyi]

@triple-Mu
The following is the inference time of ncnn-gpu, ncnn-cpu and pt-ncnn that I tested. Is this normal? This is tested by yolo prediction code.
ultralytics/ultralytics#3917

[nihui]

针对onnx模型转换的各种问题，推荐使用最新的pnnx工具转换到ncnn
In view of various problems in onnx model conversion, it is recommended to use the latest pnnx tool to convert your model to ncnn

pip install pnnx
pnnx model.onnx inputshape=[1,3,224,224]

详细参考文档
Detailed reference documentation
https://github.com/pnnx/pnnx
https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx#how-to-use-pnnx

[nihui]

yolov8 example #5506

[nihui]

hi, yolov8 examples are updated with full support for detection, segmentation, classification, pose estimation and obb
https://github.com/Tencent/ncnn/tree/master/examples

android demo
https://github.com/nihui/ncnn-android-yolov8

detailed instruction (zh)
https://zhuanlan.zhihu.com/p/16030630352