Photo ingredient recognition: how it works

A useful second look.
Not the final word.

From a photo to a reviewed pantry

Open My pantry, choose a clear kitchen photo, optionally adjust the crop, and allow online analysis. The app suggests visible foods and labels uncertain identifications. Correct each food name and its preparation state, select the ones you recognize, then decide whether to add them to your pantry or keep them as food notes.

What a photograph can and cannot tell you

A clear photo can provide evidence of visible foods. It is not a complete ingredient list for a cooked dish. Oil, salt, sugar, blended ingredients and hidden fillings may not be visible. Similar cheeses, herbs, powders and products in opaque packages can be difficult or impossible to distinguish. The tool therefore does not estimate calories, grams, freshness or allergy safety. Read original labels and verify food safety yourself.

No automatic ingredient substitutions

The recognition vocabulary is not restricted to our recipe library. Foods outside the 49 ingredient labels can be retained as exact food notes. They do not increase recipe-match percentages until you explicitly choose a matching recipe ingredient. Canned chickpeas do not silently become dried chickpeas; generic pasta does not silently become whole-wheat pasta. Cooked meal components are not automatically treated as available pantry supplies.

Take a better photo

Use good light without glare, keep the camera steady, and separate ingredients where possible. Include legible packaging for products that look alike. Crop closer for a label or small ingredient, while remembering that anything outside the crop is excluded from analysis. The app accepts JPEG, PNG and WebP up to 20 MB and 48 megapixels, then prepares a metadata-free JPEG no larger than 1600 pixels on the longest edge, which is sharp enough for the AI and quick to send. Convert HEIC/HEIF files to JPEG before selection.

Your consent and data

Photos are processed online only after consent. The browser strips metadata by redrawing the photo, and the app does not persist the uploaded image or raw model output. The selected provider is shown before submission. Provider policies and security checks also apply; read the privacy details. Recognition requires an internet connection. Manual pantry selection, recipes and saved plans continue to work when the relevant site assets are available offline.

For the website owner

This release includes a Cloudflare Pages advanced-mode _worker.js endpoint. Enable a Workers AI binding named AI, set PHOTO_RECOGNITION_ENABLED=true, and configure a Turnstile site key and secret for your actual hostname. Redeploy after changing bindings. Alternatively, set PHOTO_PROVIDER=openai and an encrypted OPENAI_API_KEY. No API key belongs in browser code. Check /api/vision/status and complete a real-photo acceptance test before advertising the service as operational.

How reliability is evaluated

Software tests cover upload validation, response validation, explicit confirmation, mapping rules, unsafe inputs and error recovery. They do not establish a model's recognition accuracy. Evaluate a consented, labeled photo set separately: clear ingredients, similar-looking foods, cooked dishes, sealed packages, poor lighting and non-food images. Report incorrect additions and missed visible foods, not an invented confidence percentage. A model change should repeat that evaluation.

Small photos and recovery

Images below 256 pixels on their shorter edge or 384 pixels on their longer edge are not sent for recognition. Below 480 pixels on the shorter edge or 640 on the longer edge, you must deliberately choose to try the limited-resolution image. These are conservative interface checks, not measured accuracy thresholds. A small image does not by itself explain a timeout.

A failed or unusable result saves nothing. Choose a closer photo or type the foods. Confirm each visible ingredient yourself; hidden ingredients, amounts, freshness and allergens cannot be established from an image.

How the AI checks work

Recognition asks two different Workers AI vision models at the same time: @cf/google/gemma-4-26b-a4b-it and @cf/meta/llama-4-scout-17b-16e-instruct. If both answer in time, their results are merged: foods both models saw are marked “Two AI checks agree”, and foods only one saw are kept and marked “Seen by one AI check”. If one model is slow or fails, the other’s answer is used after a short grace period, so a single slow model no longer ends in a timeout. If both fail, @cf/mistralai/mistral-small-3.1-24b-instruct is tried. Odd formatting in a model’s answer is repaired where it is harmless, and unreadable items are skipped with a visible note; nothing is invented. You still confirm every food, and nothing is saved automatically.

Owner: configuration and cost

All three models are available on the Workers AI free allocation of 10,000 neurons per day. From Cloudflare’s published per-token prices, one photo costs very roughly 50 neurons on Gemma 4 and 135 on Llama 4 Scout (an estimate that assumes about 3,000 input and 800 output tokens, not a measurement), so two checks allow around fifty photos a day on the free allocation. Optional variables: CF_VISION_MODELS (comma list that replaces the chain), CF_VISION_MODEL (tried first), CF_VISION_MODE=single (one model at a time with fallback, about half the cost), CF_VISION_TIMEOUT_MS, CF_VISION_GRACE_MS and CF_VISION_DEADLINE_MS. A leftover Moondream value in CF_VISION_MODEL is ignored because that model caused repeated timeouts; name it in CF_VISION_MODELS if you really want it. Provider details stay available behind Processing details in the scanner. To test the real models on your own account, set VISION_SELFTEST_TOKEN (16 or more characters) and call /api/vision/selftest with the header X-Selftest-Token; the endpoint answers 404 when the variable is unset.

A paid account can incur charges beyond the free allocation. This app neither verifies your account quota nor enforces a distributed billing cap. No real-image accuracy benchmark has been run for these models, and accuracy is not guaranteed: check /api/vision/status and test with your own photos.

Vision model contract · Free allocation and pricing

Where shall we go?

Use arrow keys to move. Enter to open. Escape to close.