How to generate bbox, after the coremltools.predict(image) step of the coreml model

[fnuabhimanyu]

❔Question

Can someone guide me on how to generate bbox, after the coremltools.predict(image) step of the coreml model? What post processing needs to be done on the predicted values?

Additional context

Do I have to post process the predicted value using these lines:

y = x[i].sigmoid()
y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i].to(x[i].device)) * self.stride[i]  # xy
y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i]  # wh

Or are they included in the CoreML model.

[github-actions]

Hello @abhimanyu8713, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook , Docker Image, and Google Cloud Quickstart Guide for example environments.

If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom model or data training question, please note Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:

• Cloud-based AI systems operating on hundreds of HD video streams in realtime.
• Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
• Custom data training, hyperparameter evolution, and model exportation to any destination.

For more information please visit https://www.ultralytics.com.

[hearables-pkinsella]

I'm having this issue as well. I've converted the model to CoreML using:

model.model[-1].export = True

This gives me three multiarray outputs that are 1x3x13x13x6, 1x3x52x52x6 and 1x3x26x26x6. Ideally I would want the output to be (x1, y1, x2, y2, conf, cls).

If I set, model.model[-1].export = False, then the output is the correct shape but I run into an error when converting to CoreML:

CoreML export failure: node 2061 (expand) got 3 input(s), expected 2

[fnuabhimanyu]

Ya, I didn't set the flag to false, and had to do post processing on the prediction of the coreml model. And it worked

[hearables-pkinsella]

@abhimanyu8713 , would you be able to provide some code snippets for how you have done the post-processing?

[dlawrences]

Hi @hearables-pkinsella , all,

There is some guidance here. Hope this will help you!

[fnuabhimanyu]

Hi @hearables-pkinsella, you can refer this issue. Hope this helps you.

[ShreshthSaxena]

@abhimanyu8713 were you able to get correct bounding boxes after post-processing? Did you test it on an iOS device ?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[waheed0332]

Any update regarding coreml conversion. I'm getting an error when I set model.model[-1].export = True