AI Weekly: Upstream of the Gate

Upstream of the Gate

Ross Young built a benchmark to test whether a model could find vulnerabilities in code, ran it, and got a perfect score. The score was fake. "What had happened was the LLM had trained itself on the comments to know what the answers were to the benchmark questions. It wasn't actually looking at the code." His summary: "So it was like using the cheat sheet to a test, not actually, you know, passing the test."

Young's lesson is not that models cheat, but that a check has to sit somewhere the thing being checked cannot reach. He wants that formalized in the harness, the skills, hooks, permitted tools and boundaries around a model, with architecture requirements living there rather than in a review meeting. "you can think of it almost as policies as code." Co-host G Mark Hardy objected: "I would suggest that a sentence that you put in a markdown file isn't the same thing as an actual control."

Hardy is right, and the week's better answers agree. Sebastien Baehni, CTO at the compensation software firm beqom, runs a design rule. Rules first, statistical models second, language models last and advisory only, never touching the arithmetic. "as soon as the LLM is touching numbers, right metics, then it's where you starts to drift." Language models, as he puts it, are "builds to be plausible, that completes to be correct."

The engineering panel had the number behind the urgency. Per LinearB's own benchmarking, COO Dan Lines reports, "So 37% of agentic PRs get merged." HumanLayer's Dexter Horthy rejects the volume framing. "you probably don't have too many PRs. If you have this as a pain point, you probably have too many bad PRs, because a good PR is a joy to review." What fixed it at three companies was not a human gate at all. "they created a very tight gate of what can be auto merged." Size ceiling, excluded paths, checks green. Deterministic. Throughput rose, because people stopped lobbing slop at a gate that would bounce it.

He also volunteered a failure: "we ran a lights off factory for like 4 or 5 months" before the team rebuilt it from scratch. Warp's Aloke Desai puts the person earlier instead, treating an agent that stops for a spec review as correct behavior: "An agent realizes it needs a spec, it needs a human to review the spec, or an agent assigns a PR to a human to review is a feature, not a bug."

Nicole Beckwith has made this argument in security for years. "Building a detection is easy, knowing whether that detection is mapped to your environment, and whether that fidelity is strong is the hard part." Knowing the alarm at two in the morning is "the power shell script that your HR team is running every Tuesday" is institutional knowledge, and generation does not supply it.

Three weeks ago the question was who owns the agent. Now it is what the owner inspects, and the answer keeps coming back the same. Not the output.

Sources: Interviews from CISO Tradecraft (Aug 31, Ross Young and G Mark Hardy), The Modern People Leader (Sep 2, Sebastien Baehni of beqom), Dev Interrupted (Sep 1, Dexter Horthy of HumanLayer, Aloke Desai of Warp and Dan Lines of LinearB), and Cloud Security Podcast (Sep 1, Nicole Beckwith of Cribl).