Reading prices straight from the screenshot with vision models, across 100+ marketplaces

The price-monitoring product already stores a screenshot of every offer as evidence. Pulling title, price, instalments and seller out of each marketplace's HTML meant a different set of selectors per site, kept up to date by hand every time a layout changed.

I created the vision-extraction bot and the AI audit pass, and built the pipeline that combines OCR with an LLM.

A browser bot captures the product page and Gemini Vision returns title, prices, instalments and seller as JSON, guided by a prompt config per marketplace. Instead of writing those configs by hand, a Claude agent per marketplace read two screenshots and drafted its config, and a test harness checked each one against Gemini. A second pipeline runs PaddleOCR first and has an LLM structure the text through LiteLLM, falling back to another model when validation fails. An audit pass re-reads prices and seller names from the screenshots and writes corrections back to the database.

Every read now costs tokens and depends on a model provider, where a CSS selector cost nothing. I accepted that because a config is cheap to regenerate when a layout changes, and I kept the bill visible: flash-class models first, a fallback only on failed validation, and tokens and cost logged on every call.

158 marketplace configs drafted by agents and validated by a harness, structured price data read from the same screenshot that proves it, and an audit loop that corrects the database instead of a person doing it.