Skip to main content
dbb.

Results

500 photos across 25 dog breeds. 7 models.

Models ranked by first-choice accuracy
ModelFirst matchTop threeMedian latency
GPT-6.1 TerraMedium reasoningFirst match
83.8%
419/500
Top three
94.4%
472/500
Median latency4.66 s
GPT-6 AstraMedium reasoningFirst match
83.6%
418/500
Top three
95.6%
478/500
Median latency9.43 s
GPT-6.1 SolMedium reasoningFirst match
83.0%
415/500
Top three
95.2%
476/500
Median latency7.99 s
GPT-5 MiniMedium reasoningFirst match
82.0%
410/500
Top three
92.8%
464/500
Median latency12.77 s
GPT-4o MiniNo reasoning settingFirst match
79.0%
395/500
Top three
89.0%
445/500
Median latency2.01 s
GPT-6 Luna FastMedium reasoningFirst match
70.8%
354/500
Top three
91.2%
456/500
Median latency3.69 s
GPT-6 LunaMedium reasoningFirst match
70.0%
350/500
Top three
92.0%
460/500
Median latency5.26 s

Median latency is measured per request. Serving conditions varied between runs.

Accuracy by breed

Each breed has 20 photos.

Average gives each of the 7 models equal weight.

First match accuracy by breed
BreedAverageTerraAstraSol5 Mini4o MiniLuna FastLuna
Staffordshire Bull Terrier
50.7%7 models
60%12/20
75%15/20
80%16/20
55%11/20
40%8/20
30%6/20
15%3/20
American Pit Bull Terrier
47.9%7 models
45%9/20
45%9/20
50%10/20
75%15/20
65%13/20
40%8/20
15%3/20
English Setter
71.4%7 models
90%18/20
85%17/20
85%17/20
90%18/20
75%15/20
30%6/20
45%9/20
American Bulldog
61.4%7 models
95%19/20
70%14/20
65%13/20
40%8/20
70%14/20
45%9/20
45%9/20
English Cocker Spaniel
62.9%7 models
75%15/20
75%15/20
80%16/20
55%11/20
25%5/20
70%14/20
60%12/20
Miniature Pinscher
76.4%7 models
50%10/20
85%17/20
90%18/20
100%20/20
100%20/20
55%11/20
55%11/20
Scottish Terrier
56.4%7 models
65%13/20
75%15/20
70%14/20
45%9/20
60%12/20
45%9/20
35%7/20
Havanese
54.3%7 models
75%15/20
45%9/20
45%9/20
70%14/20
55%11/20
50%10/20
40%8/20
Keeshond
87.9%7 models
95%19/20
95%19/20
95%19/20
95%19/20
95%19/20
60%12/20
80%16/20
Japanese Chin
83.6%7 models
90%18/20
90%18/20
85%17/20
90%18/20
60%12/20
85%17/20
85%17/20
Leonberger
87.9%7 models
100%20/20
95%19/20
100%20/20
85%17/20
85%17/20
70%14/20
80%16/20
Soft-coated Wheaten Terrier
84.3%7 models
85%17/20
95%19/20
95%19/20
95%19/20
80%16/20
65%13/20
75%15/20
Chihuahua
79.3%7 models
85%17/20
75%15/20
65%13/20
75%15/20
90%18/20
85%17/20
80%16/20
Newfoundland
81.4%7 models
80%16/20
80%16/20
80%16/20
75%15/20
70%14/20
90%18/20
95%19/20
Samoyed
87.1%7 models
90%18/20
95%19/20
95%19/20
90%18/20
95%19/20
75%15/20
70%14/20
Shiba Inu
87.9%7 models
90%18/20
85%17/20
85%17/20
100%20/20
100%20/20
75%15/20
80%16/20
Basset Hound
91.4%7 models
95%19/20
100%20/20
90%18/20
90%18/20
80%16/20
90%18/20
95%19/20
Great Pyrenees
91.4%7 models
95%19/20
90%18/20
90%18/20
100%20/20
80%16/20
95%19/20
90%18/20
Boxer
83.6%7 models
85%17/20
85%17/20
85%17/20
85%17/20
90%18/20
75%15/20
80%16/20
Yorkshire Terrier
87.1%7 models
95%19/20
90%18/20
85%17/20
85%17/20
95%19/20
80%16/20
80%16/20
Beagle
93.6%7 models
90%18/20
90%18/20
95%19/20
95%19/20
100%20/20
95%19/20
90%18/20
German Shorthaired Pointer
85.0%7 models
85%17/20
90%18/20
90%18/20
80%16/20
80%16/20
85%17/20
85%17/20
Pomeranian
94.3%7 models
100%20/20
90%18/20
90%18/20
95%19/20
100%20/20
95%19/20
90%18/20
Pug
99.3%7 models
95%19/20
100%20/20
100%20/20
100%20/20
100%20/20
100%20/20
100%20/20
Saint Bernard
85.7%7 models
85%17/20
90%18/20
85%17/20
85%17/20
85%17/20
85%17/20
85%17/20

Methodology

Each model evaluated the same 500 photos once. A match requires a valid response with the reference breed first or among the top three. All attempts count.

Dataset credit: Oxford-IIIT Pet dataset (Parkhi, Vedaldi, Zisserman, and Jawahar, 2012), CC BY-SA 4.0. Photo copyrights remain with their owners.