[?] How to pass custom parameters to YOLO models in apply_model()

[oguz-hanoglu]

My aim is to use apply_model with a yolov11 model. I need to send some args to the model. Analyzing the document I see that kwargs can be used for this purpose.

**kwargs: optional model-specific keyword arguments passed through
                to the underlying inference implementation

My baseline is:

# Apply the model to the dataset
model = YOLO(
    "weights/best.pt"
)
dataset.apply_model(
    model,
    label_field="yolov11",
)

CASE1:

# Apply the model to the dataset
dataset.apply_model(
    model,
    label_field="yolov11",
    kwargs={"conf": 0.25, "iou": 0.1, "agnostic_nms": True, "imgsz": 512},
)

CASE2:

# Apply the model to the dataset
dataset.apply_model(
    model,
    label_field="yolov11",
    conf=0.25,
    iou=0.7,
    agnostic_nms=True,
    imgsz=512,
)

The two cases above result in the same label with the baseline showing that my arguments are not sent to the underlying model.
Can it be because here kwargs is not received but not used?

fiftyone/fiftyone/core/models.py

Line 52 in 67bdf7e

def apply_model(

[Burhan-Q]

The conf value can be passed directly to the apply_model method using the confidence_thresh argument. For everything specific to the Ultralytics model, I recommend using the following:

import fiftyone as fo
from ultralytics import YOLO

dataset = fo.load_dataset("path/to/dataset")
model = YOLO("yolo11n.pt")

# Set model overrides
custom_cfg = {"iou": 0.1, "agnostic_nms": True, "imgsz": 512}
for k,v in custom_cfg.items():
    model.overrides[k] = v

dataset.apply_model(model, label_field="yolov11", confidence_thresh=0.25)

Reproducible example

import fiftyone.zoo as foz
from fiftyone import ViewField as F

from ultralytics import YOLO

model = YOLO("yolo11n.pt")

ds = foz.load_zoo_dataset("quickstart")

cfg = {
    "classes":[0,2],  # limit inference to only "person" and "car" classes
    "imgsz":320, 
    "device":"cuda"  # only with GPU and PyTorch+CUDA installed
}
for k,v in cfg.items():
    model.overrides[k] = v

ds.apply_model(
    model,
    label_field="yolo11-predictions",
    confidence_thresh=0.25,
)

view = ds.filter_labels("yolo11-predictions", F("label") is not None)
len(view)
# >>> 84

# re-run without setting model overrides
model = YOLO("yolo11n.pt")  # load new model instance
ds.delete_sample_fields("yolo11-predictions")  # clear field

ds.apply_model(
    model,
    label_field="yolo11-predictions",
    confidence_thresh=0.25,
)
view = ds.filter_labels("yolo11-predictions", F("label") is not None)
len(view)
# >>> 199

If you re-run without setting the overrides you'll find that the resulting view will have significantly more samples where the label value is not None (199 vs 84).

[dangom13]

I still don't think the confidence_thresh parameter is working for yolo models. It does not seem to have an impact.

[brimoor]

@dangom13 appears to be working for me:

import fiftyone as fo
import fiftyone.zoo as foz
from ultralytics import YOLO

dataset = foz.load_zoo_dataset("quickstart", max_samples=10)
model = YOLO("yolo11n.pt")

dataset.apply_model(model, label_field="yolov11", confidence_thresh=0.9)

print(dataset.bounds("yolov11.detections.confidence"))
# (0.9024709463119507, 0.949521005153656)

dataset.apply_model(model, label_field="yolov11", confidence_thresh=0.5)

print(dataset.bounds("yolov11.detections.confidence"))
# (0.547297477722168, 0.949521005153656)