My wife sent me a link to a copycat recipe for the hash browns from a certain pancake chain. I imported it into Feastmark, the recipe app I built, mostly to show off.
It came back with one ingredient — “Finely shredded potatoes” — and four instructions. Step one was “Preheat your oven.” No temperature. Step two said to prepare the sheet pan with oil.
The actual recipe has four ingredients with quantities, six steps, and starts with “Preheat the oven to 375°F.” It uses half a cup of butter. There is no oil in it.
The diagnosis writes itself: the language model made things up. That’s the story everyone reaches for, because it’s usually true and it’s always available. It was wrong here, and the way it was wrong is the interesting part.
What Was Actually on the Page
Before touching any code, I pulled the page down and looked at what a scraper would see.
That page ships a complete schema.org Recipe block as JSON-LD — the structured data Google needs to render a recipe card in search results. Almost every WordPress food blog publishes one, because the recipe-card plugins do it automatically. It had all four ingredients with quantities, all six steps verbatim, prep time, cook time, and yield. Perfect, machine-readable, sitting right there in the HTML.
My scraper deleted it.
$('script, style, nav, footer, header, iframe, .ads, .comments').remove();That line is meant to strip noise before handing text to the model, and it’s a reasonable line to write. But JSON-LD lives inside <script type="application/ld+json">. That .remove() threw away the single best source on the page before anything read it. Worse, nothing in the codebase ever looked for structured data at all. I’d never written that path.
Then there was the second line:
const bodyText = $('body').text().replace(/\s+/g, ' ').trim().slice(0, 2000);The page body is 36,160 characters. I measured where the oven temperature first appears: character 5,042.
So the model’s entire input was the first 2,000 characters of a food blog. If you’ve ever scrolled one, you know exactly what lives there: the affiliate disclosure, the “why you’ll love this recipe” list, a jump-to-recipe menu, and several paragraphs about the author’s memories of diner breakfasts. The recipe card is thousands of characters further down, past the cutoff.
It Summarized Faithfully
Here’s the part that changed how I think about these failures.
Every wrong detail in that import traces directly back to text that was in the window.
“Finely shredded potatoes” wasn’t invented. It’s a bullet point from a section titled “What Makes These Hash Browns?” — a list describing texture, not ingredients. The model saw a bulleted list of food-sounding phrases under a recipe headline and did the reasonable thing.
“Prepare the sheet pan with oil” came from a sentence in the intro explaining that this version “comes together on a simple sheet pan instead” of a griddle. The oil was the model’s inference about what you do with a sheet pan. Sensible. Wrong.
And “Preheat your oven” with no temperature is exactly what you’d write if you knew a recipe was baked and had never been told how hot.
The model didn’t hallucinate. It performed competent summarization of marketing prose, because marketing prose was all I gave it. The failure was upstream, in a .slice(0, 2000) written long before any of this involved AI.
That’s the lesson I’d hand to anyone building extraction on top of a language model: when the output is confidently wrong, look at the input before you look at the model. A hallucination and a faithful summary of the wrong text produce identical-looking garbage. They have completely different fixes, and only one of them is the model’s fault.
The Fix Is Boring, Which Is the Point
Read the structured data first. Fall back to the model only when there isn’t any.
The new path parses the JSON-LD directly — handling @graph, arrays, HowToStep and HowToSection, ISO 8601 durations, and the four different shapes publishers use for recipeYield. If it finds a usable recipe, it returns it and stops.
Same URL, after:
4 ingredients — 6 Yukon gold potatoes, ½ cup unsalted butter,
2 tsp sea salt, 2 tsp garlic powder
6 steps — #1 "Preheat the oven to 375°F."
AI call made — noneZero model calls. It’s exact rather than approximate, it’s free, and it can’t confabulate, because there’s no generation step to confabulate with. The most reliable AI feature I shipped this month works by not calling the AI.
Pages without structured data still go to the model, now with a 12,000-character window that hoists the recipe card to the front instead of feeding it the intro.
I ran it against a spread of food sites. One of them fixed a bug I’d had open for days: a different recipe that had been importing 3 ingredients against 5 instructions — which I’d written off as “probably a partial parse of their markup” — came back with 8. Same root cause. I’d filed it as a separate, unrelated, low-priority mystery.
Two Things I Got Wrong Along the Way
I want these in here, because the tidy version of this story is a lie.
I nearly shipped a fix for a bug that didn’t exist. A tester reported that ingredients from video imports had no amounts — “salt: dash, yogurt: 10oz” was the request. I had a clean theory: the transcript tier only runs when extraction returns nothing, so a video whose caption yields ingredient names without amounts never triggers it, and we throw away the spoken audio that has the quantities.
Good theory. I pulled the transcript before building anything.
The video contains no amounts. None. The narrator names every ingredient — “salt, pepper, red chili powder, cumin, turmeric, ginger, garlic, yogurt and lemon juice” — and never says a single quantity. The only numbers in 1,003 characters of transcript are an air-fryer temperature and two cooking times. I checked the description: empty. The top 25 comments: nothing. The channel’s links: none.
The extraction had been perfect. The amounts don’t exist. Had I trusted the theory, I’d have built a whole retrieval tier that would have changed exactly nothing, and I’d have believed it worked because the thing I “fixed” would still look broken in the same way.
A fix can be invisible even when it’s correct. Extraction results are cached for thirty days. Deploy the fix, re-import the same link, get the identical broken recipe — because the request never reaches the new code. I caught this before shipping, but only barely, and only because I went looking for reasons the fix might not show up. Cache entries are now keyed by parser version, so a parser change retires the results it supersedes.
What I’d Take From This
The instinct to blame the model is strong and it’s lazy. It’s also self-serving, because “the AI is unreliable” is a much more comfortable conclusion than “I truncated the input at two thousand characters and forgot.”
Three things I’d do differently from the start:
Look for structured data before you write a prompt. A large fraction of the web publishes machine-readable versions of exactly what you’re trying to extract, because search engines demanded it. Reading it is cheaper, faster, and exact. Reaching for a model first is reaching for the least reliable tool available.
Print the model’s actual input when the output is wrong. Not the URL. Not the page. The literal string you handed it. Mine would have shown the affiliate disclosure and a jump-to-recipe menu, and the investigation would have been over in ninety seconds instead of an afternoon.
Test the hypothesis before you build the fix. Both times I was sure this session, I was sure about the wrong thing. Checking cost minutes. Building would have cost a day and left the bug in place.
The scraper is 60 lines. Two of them cost me a recipe, and neither had anything to do with artificial intelligence.
