RCNN Runner

[RaphaelKlink]

Hey,

i tried to run the RCNN Runner from this repository, but it always fails for me.
I used the Mask rcnn model from the onnx model, because i could not quantize the rcnn model from the release with the given python files.
https://github.com/onnx/models/tree/main/vision/object_detection_segmentation/mask-rcnn

Then i run it with firemarshal in linux, with the given preprocessed image.
rcnn_runner -m mask_rcnn_R_50_FPN_1x_opt_quant.onnx -i images/preprocessed.jpg -x 2 -d 0
The preprocessing is executed correctly, but then i gets stuck at the beginning of the execution.
The last line on my output is:
[I:onnxruntime:, sequential_executor.cc:157 Execute] Begin execution
And then i simply get nothing more. No output, no error or anything else. The core is still running as i can stop the execution with ctrl+c and then execute something else.
I also tried the CPU mode with the same result.

Did you ever run the RCNN Runner?
Do you have an idea where the problem is?

Greetings
Raphael Klink

[hngenc]

If you're not getting any errors, then it might just be very slow, but still working. How long did you keep the FPGA running? Would you be able to leave it running overnight?

Also, can you share your mask_rcnn_R_50_FPN_1x_opt_quant.onnx file?

[RaphaelKlink]

Hey,

i let it run for over an hour. I got the same results for int8 and fp32.

I used the imagenet runner before and that one took only seconds with the resnet 50 example.

I uploaded the onnx file to:
https://rackstation.iti.uni-luebeck.de:5001/sharing/A6bOv1njJ

Thanks for the help.

Greetings
Raphael

[hngenc]

Can you also share your rcnn_runner binary? I built one with these commands:

rm -rf ./build/
./build.sh --parallel --enable_training --for_firesim --config=Release --cmake_extra_defines onnxruntime_USE_SYSTOLIC=ON onnxruntime_S    YSTOLIC_INT8=ON onnxruntime_SYSTOLIC_FP32=OFF
cd ./systolic_runner/rcnn_runner/
./build.sh --parallel --enable_training --for_firesim --config=Release

Mine didn't seem to freeze, but when running the ONNX model you uploaded, I encountered this error:

terminate called after throwing an instance of 'Ort::Exception'
what():  This is an invalid model. Type Error: Type 'tensor(int8)' of input parameter (8_quantized) of operator (MaxPool) in node (8) is invalid.
/root/run-tests.sh: line 88:   229 Aborted                 ./ort_test -m mask_rcnn_R_50_FPN_1x_opt_quant.onnx -i images/preprocessed.jpg -x 2 -O 99 -t /root/traces/rcnn-raphael

I think the model you're using might be quantizing the MaxPool inputs, but our fork of ONNX Runtime might not support that yet (even though Gemmini's hardware should be able to support that). [Correction: This error seems to be saying that opset 8 doesn't support quantized inputs. Only opset 10+ does.]

But it would be good if I could also try running with your rcnn_runner directly, since it seems yours differs from mine. Also, can you make sure that you build your rcnn_runner with the --for_firesim option?

[hngenc]

Correction to something I posted above: The error I saw when running your model seems to be saying that opset 8 doesn't support quantized inputs. Only opset 10+ does. Our ONNX-Runtime fork can handle quantized inputs to maxpool, but they may be done on the CPU instead of on Gemmini.

Also, when compiling rcnn_runner binary, you might want to try applying this patch which will dump each executed kernel to stderr. That will make this issue easier to debug.