# hotdogbench

1,000 food photos for one question: hot dog or not hot dog. 239 hot dogs, 500 random other foods,
and 261 look-alikes picked to be hard (sausages, sandwiches, burgers, burritos, churros and the like).

## Files

- `hotdogbench.parquet`: one row per photo: `id`, `is_hot_dog`, `category` (`hot_dog`, `other_food`, `lookalike`),
  `lookalike_class` (e.g. Bratwurst), the JPEG bytes in `image`, `width`, `height`,
  and the attribution columns below.
- `ATTRIBUTION.csv`: the same rows without image bytes: who took each photo, where it was published, its license,
  and what we changed.

## Where the photos come from

The photos and their hot-dog / food labels come from [Open Images V7](https://storage.googleapis.com/openimages/web/index.html)
by Google. Hot dogs are images with a human-verified positive "Hot dog" label; other foods are images with a
human-verified positive "Food" label and no "Hot dog" label, drawn at random from the validation and test splits.
Look-alikes are images with a human-verified positive label for one of 22 hot-dog-shaped or bun-adjacent foods
(Sausage, Bratwurst, Kielbasa, Chorizo, Sausage roll, Sandwich, Submarine sandwich, Bánh mì, Breakfast sandwich,
Hamburger, Cheeseburger, Burrito, Taco, Kebab, Egg roll, Spring roll, Churro, Pretzel, Baguette, Bread roll,
Bun, Chicken fingers) and no "Hot dog" label, drawn round-robin across those classes.

## Label review

Open Images' "Hot dog" labels are noisy: about one in five "hot dogs" we drew was really a sub, lobster roll,
burger, bánh mì or similar. Every photo was checked by three models (Jev, GPT-6 Luna, Claude Haiku); each photo where
two or more disagreed with its label was reviewed by hand and moved to its proper category (`review_tag = relabeled`).
Our rule: a hot dog is a cooked sausage that is ready for or in a bun. Photos of several cooked franks without a bun
count as hot dogs; raw sausages and corn dogs do not (`review_tag = multi_dog` marks these tricky cases).

## The bonus row

Row 1001 is not a photo: it is our own painting of a hot dog, after René Magritte's *The Treachery of Images*,
captioned "Ceci n'est pas un hot dog." It is labeled `is_hot_dog = true` and `scored = false`: benchmarks report
each model's verdict on it but leave it out of accuracy. It is licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) by MotherDuck.

## Licenses

- **Photos**: each photo is licensed by its photographer under
  [CC BY 2.0](https://creativecommons.org/licenses/by/2.0/). Credit for every photo (title, author, the
  photographer's page and the photo's Flickr page) is in `ATTRIBUTION.csv` and in the parquet file.
  Google notes it makes no representations about each image's license status; we re-checked each photo's
  Flickr page when building this set and kept only photos still shown as CC BY 2.0.
- **Labels**: Open Images annotations are by Google LLC under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
  We used a subset of them (which photos are hot dogs).
- **Changes we made**: photos were rotated upright where Open Images records a rotation, resized to at most
  1,024 px on the long side, and re-encoded as JPEG with metadata removed. Each row's `modifications` says what applied.

The photographers do not endorse hotdogbench or MotherDuck. If you are the author of a photo and want it removed
or credited differently, open an issue and we will update the set.