Skip to main content
dbb.

Results

500 photos across 25 dog breeds. 10 models.

Models ranked by first-choice accuracy
ModelFirst matchTop threeMedian latencyEst. cost / scan
Claude Opus 5.5Medium reasoningFirst match
86.0%
430/500
Top three
95.0%
475/500
Median latency3.69 s$0.01484
GPT-6.1 TerraMedium reasoningFirst match
83.8%
419/500
Top three
94.4%
472/500
Median latency4.66 s$0.00274
GPT-6 AstraMedium reasoningFirst match
83.6%
418/500
Top three
95.6%
478/500
Median latency9.43 s$0.02846
GPT-6.1 SolMedium reasoningFirst match
83.0%
415/500
Top three
95.2%
476/500
Median latency7.99 s$0.00577
GPT-5 MiniMedium reasoningFirst match
82.0%
410/500
Top three
92.8%
464/500
Median latency12.77 s$0.00269
Claude Sonnet 5.5Medium reasoningFirst match
81.0%
405/500
Top three
89.4%
447/500
Median latency2.21 s$0.00743
Claude Haiku 5.5Medium reasoningFirst match
80.8%
404/500
Top three
90.6%
453/500
Median latency1.58 s$0.00037
GPT-4o MiniNo reasoning settingFirst match
79.0%
395/500
Top three
89.0%
445/500
Median latency2.01 s$0.00154
GPT-6 Luna FastMedium reasoningFirst match
70.8%
354/500
Top three
91.2%
456/500
Median latency3.69 s$0.00072
GPT-6 LunaMedium reasoningFirst match
70.0%
350/500
Top three
92.0%
460/500
Median latency5.26 s$0.00036

Median latency is measured per request. Serving conditions varied between runs. Some runs used prompt caching. Cost estimates use uncached model token rates for breed-analysis attempts, in USD at the recorded list rates.

Accuracy by breed

Each breed has 20 photos.

Average gives each of the 10 models equal weight.

First match accuracy by breed
BreedAverageOpusTerraAstraSol5 MiniSonnetHaiku4o MiniLuna FastLuna
English Setter
77.0%10 models
95%19/20
90%18/20
85%17/20
85%17/20
90%18/20
85%17/20
90%18/20
75%15/20
30%6/20
45%9/20
Staffordshire Bull Terrier
52.5%10 models
65%13/20
60%12/20
75%15/20
80%16/20
55%11/20
60%12/20
45%9/20
40%8/20
30%6/20
15%3/20
American Pit Bull Terrier
45.5%10 models
35%7/20
45%9/20
45%9/20
50%10/20
75%15/20
45%9/20
40%8/20
65%13/20
40%8/20
15%3/20
Soft-coated Wheaten Terrier
82.5%10 models
100%20/20
85%17/20
95%19/20
95%19/20
95%19/20
40%8/20
95%19/20
80%16/20
65%13/20
75%15/20
American Bulldog
61.0%10 models
60%12/20
95%19/20
70%14/20
65%13/20
40%8/20
70%14/20
50%10/20
70%14/20
45%9/20
45%9/20
English Cocker Spaniel
60.0%10 models
80%16/20
75%15/20
75%15/20
80%16/20
55%11/20
35%7/20
45%9/20
25%5/20
70%14/20
60%12/20
Miniature Pinscher
83.0%10 models
100%20/20
50%10/20
85%17/20
90%18/20
100%20/20
100%20/20
95%19/20
100%20/20
55%11/20
55%11/20
Keeshond
90.0%10 models
100%20/20
95%19/20
95%19/20
95%19/20
95%19/20
95%19/20
90%18/20
95%19/20
60%12/20
80%16/20
Scottish Terrier
60.5%10 models
70%14/20
65%13/20
75%15/20
70%14/20
45%9/20
75%15/20
65%13/20
60%12/20
45%9/20
35%7/20
Havanese
58.0%10 models
75%15/20
75%15/20
45%9/20
45%9/20
70%14/20
65%13/20
60%12/20
55%11/20
50%10/20
40%8/20
Japanese Chin
84.0%10 models
90%18/20
90%18/20
90%18/20
85%17/20
90%18/20
85%17/20
80%16/20
60%12/20
85%17/20
85%17/20
Leonberger
87.5%10 models
85%17/20
100%20/20
95%19/20
100%20/20
85%17/20
80%16/20
95%19/20
85%17/20
70%14/20
80%16/20
Boxer
87.5%10 models
95%19/20
85%17/20
85%17/20
85%17/20
85%17/20
95%19/20
100%20/20
90%18/20
75%15/20
80%16/20
Chihuahua
80.5%10 models
85%17/20
85%17/20
75%15/20
65%13/20
75%15/20
85%17/20
80%16/20
90%18/20
85%17/20
80%16/20
Newfoundland
82.5%10 models
75%15/20
80%16/20
80%16/20
80%16/20
75%15/20
95%19/20
85%17/20
70%14/20
90%18/20
95%19/20
Pomeranian
92.5%10 models
95%19/20
100%20/20
90%18/20
90%18/20
95%19/20
95%19/20
75%15/20
100%20/20
95%19/20
90%18/20
Samoyed
88.5%10 models
95%19/20
90%18/20
95%19/20
95%19/20
90%18/20
95%19/20
85%17/20
95%19/20
75%15/20
70%14/20
Shiba Inu
90.0%10 models
95%19/20
90%18/20
85%17/20
85%17/20
100%20/20
95%19/20
95%19/20
100%20/20
75%15/20
80%16/20
Basset Hound
91.5%10 models
100%20/20
95%19/20
100%20/20
90%18/20
90%18/20
80%16/20
95%19/20
80%16/20
90%18/20
95%19/20
Great Pyrenees
94.0%10 models
100%20/20
95%19/20
90%18/20
90%18/20
100%20/20
100%20/20
100%20/20
80%16/20
95%19/20
90%18/20
Yorkshire Terrier
88.5%10 models
90%18/20
95%19/20
90%18/20
85%17/20
85%17/20
90%18/20
95%19/20
95%19/20
80%16/20
80%16/20
Beagle
93.5%10 models
95%19/20
90%18/20
90%18/20
95%19/20
95%19/20
95%19/20
90%18/20
100%20/20
95%19/20
90%18/20
German Shorthaired Pointer
84.5%10 models
85%17/20
85%17/20
90%18/20
90%18/20
80%16/20
85%17/20
80%16/20
80%16/20
85%17/20
85%17/20
Pug
98.5%10 models
95%19/20
95%19/20
100%20/20
100%20/20
100%20/20
95%19/20
100%20/20
100%20/20
100%20/20
100%20/20
Saint Bernard
86.5%10 models
90%18/20
85%17/20
90%18/20
85%17/20
85%17/20
85%17/20
90%18/20
85%17/20
85%17/20
85%17/20

Methodology

Each model evaluated the same 500 photos once with the same benchmark prompt. A match requires a valid breed ranking with the reference breed first or among the top three. All attempts count.

Dataset credit: Oxford-IIIT Pet dataset (Parkhi, Vedaldi, Zisserman, and Jawahar, 2012), CC BY-SA 4.0. Photo copyrights remain with their owners.