The Cost of Checking
Max Spero will tell you his product's error rate before anyone asks. Pangram, the detection company he co-founded, flags AI-written text at a false positive rate of about one in ten thousand, which at its scale means "we're gonna have a few dozen false positives a day." Most founders bury that number. His argument requires it.
The argument is about an arithmetic that broke. Producing plausible text used to cost effort, and every system that screens anything ran on that assumption. Job applications, product reviews, insurance claims, refund photos. "Now you could use an AI auto applier to apply to a 1000 jobs in a day." The result is not slop, it is asymmetry. "this creates this just, like, massive imbalance of effort where it's infinitely easy to produce content or take actions, and then the verifier or reviewer on the other side ends up being kind of swamped." Generation went to zero. Verification did not. He sells the verification. His forecast is specific: "I think we're actually dangerously close. If we do nothing, like dead internet theory will happen within the next few years."
Shopify is running the opposite experiment. Not detect the machines, serve them. COO Jess Hertz calls the company's Catalog "our product layer for agentic commerce," built to make products "understandable, discoverable, purchasable by AI." The number behind it: "when someone comes from an agentic channel, the conversion rate that when an agent is using our catalog data is 2X compared to if it's just using scraped data." Structure the data for a machine reader and it buys twice as often. What it buys is different too. "So 75% of AI attributed orders last quarter came from categories outside of the top 100." Search rewarded spend and popularity. Agents, she argues, reward precision.
The supply side turned up on a marketers' panel, unembarrassed. One demand marketer walked through a twenty-three step agent that assembles pages "not for the people to browse, but for the LLMs to crawl." Her framing beat the argument around it: "So outsource the robot content, keep the human content." The loop closed in the same hour, when another panelist said he had run his own AI-assisted work through Pangram to see what would happen. "My scores ranged from 51% AI to 0% AI." Machines writing the page, machines reading the page, a startup selling the referee.
Somebody pays for the checking. Aneesh Dhawan, who co-founded the research firm Knit, has the invoice from a survey of more than 150 enterprise research leaders. After the model produces its output, a researcher spends "about 10 hours across 3 to 5 business days, to then take that AI output and turn it into something that was decision ready." Their top fear is no longer replacement. It is decisions made straight off unedited output, at roughly twice the intensity of losing their jobs. "What happens with AI outputs and especially unedited AI outputs is it's already taking you to that conclusion."
Generation was the cheap half.
Sources: Interviews from Equity (Sep 2, Max Spero of Pangram), Masters of Scale (Sep 1, Jess Hertz of Shopify), Exit Five (Sep 3), and CMO Confidential (Sep 1, Aneesh Dhawan of Knit).
Software leaders are moving scarce human effort from producing content to verifying it, and starting to put a price on that work.