Teleport spent three months using frontier models to search its code for security flaws. The operator report says 13 engineers found dozens of bugs and nearly twice as many high-severity problems in one quarter as the company had found across 2024 and 2025 combined. Those are company-reported results without a controlled productivity baseline, but the workflow lesson is more interesting than the headline count. Teleport’s engineering account
The team first built Conclave, a multi-stage system in which several agents generated, criticized and ranked findings. In practice, experienced security engineers using simple, targeted prompts often did better. One critical vulnerability began with an engineer giving a model a single file and asking it to solve a capture-the-flag challenge. As the models improved, finding plausible problems became easier; human verification and deduplication became the bottleneck.
This is a catch-up artifact from August, resurfaced this week by Simon Willison. Its value is the negative result. More orchestration did not automatically produce more useful security work. The scarce part became a skilled person who could choose a tractable boundary, distinguish an exploit from an alarming story and turn the finding into a repair.