Cannot evaluate multiple YOLO models at once

[Fleyderer]

Search before asking

• I have searched the Yolo Tracking issues and discussions and found no similar questions.

Yolo Tracking Component

Tracking

Bug

After running script with command:

python tracking/val.py --benchmark MOT17 --yolo-model yolov8x.pt yolo11x.pt --tracking-method botsort --reid-model osnet_x0_25_msmt17.pt  --source tracking/val_utils/data/MOT17/train

I am getting this error:

Standard Error:
 Traceback (most recent call last):
  File "/home/kozakmv/Programming/Python/boxmot/tracking/val_utils/scripts/run_mot_challenge.py", line 84, in <module>
    dataset_list = [trackeval.datasets.MotChallenge2DBox(dataset_config)]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kozakmv/Programming/Python/boxmot/tracking/val_utils/trackeval/datasets/mot_challenge_2d_box.py", line 124, in __init__
    raise TrackEvalException(
trackeval.utils.TrackEvalException: Tracker file not found: //MOT17-05.txt

At the same time, command with option --yolo-model yolov8x.pt or --yolo-model yolo11x.pt worked fine, so bug can be reproduced only while there are multiple detectors.

After debugging I've found that problem is in reassignment of source option:

opt.source = mot_folder_path / 'img1'

That's why at the first time path tracking/val_utils/data/MOT17/train becomes tracking/val_utils/data/MOT17/train/img1 and everything is ok, but second time it becomes tracking/val_utils/data/MOT17/train/img1/img1, while mot_folder_paths becomes not a list of directories, but list of .txt files and it is wrong.

P. S. I would like to add that main problem here is in idea of reassignment args, passed to the script. Sometimes it looks easier to change it and make things work, but later it reminds you why is it anti-pattern 😄

Environment

Doesn't matter

Minimal Reproducible Example

python tracking/val.py --benchmark MOT17 --yolo-model yolov8x.pt yolo11x.pt --tracking-method botsort --reid-model osnet_x0_25_msmt17.pt  --source tracking/val_utils/data/MOT17/train