Issues about deploy yolo master via ONNX-Runtime

[cydiachen]

Thank you for your excellent job.
I have carefully read your project and tried to export your model to deploy it on CPU via ONNX-Runtime (OPSET=13).
I managed to export the model successfully.
But it seems that the onnx version of YOLO-Master infers a image with a much slower speed than naive Pytorch cpu inference.
Could you provide any clues?

[isLinXu]

Hi there, thanks for the feedback and your testing!
This is actually a known trade-off when deploying MoE (Mixture-of-Experts) architectures to ONNX, especially on CPU.
Here are the key clues regarding the performance drop:

1. Conditional Execution vs. Static Graph
   PyTorch (Eager Mode): When running forward() in PyTorch, the code uses Python's dynamic control flow. It physically skips the computation of non-active experts.

• Result: You only pay the compute cost for the Top-k experts.
  ONNX (Graph Mode): Standard ONNX export often struggles to represent this "conditional skipping" efficiently. Depending on how the routing is implemented, the exported graph might be:
• Executing ALL experts: It might have collapsed into a "Dense" model where all branches are computed, and the result is masked later.
• Heavy Indexing Overhead: Even if it selects experts, the Gather, Scatter, and Slice operations required to route data dynamically create massive memory overhead on CPU, which outweighs the FLOPs reduction.

2. Verification Steps
   [ ] Check with Netron: Open your .onnx file in Netron. Look at the MoE block.
   [ ] Run ONNX Simplifier: Sometimes redundant Gather/Scatter ops can be fused.
   [ ] Profiling: If possible, check if the slowdown comes from NonMaxSuppression (post-processing) or the backbone itself.