Ten Out of Ten Thousand: Pen Testing, Triage, and Fixes at Machine Speed
A point-in-time security assessment can be out of date the moment the report arrives. Episode four of The Blind Spot is about what happens after that stops being an inconvenience and becomes the operating reality: attack, detection, and response all moving at machine speed.
Dave Gerry is the CEO of Bugcrowd, one of the largest offensive security platforms in the world, connecting customers with a community of roughly 700,000 security researchers. He has spent most of his career in application and product security, and he now runs a platform where, per Bugcrowd's own Inside the Mind of a Hacker report, 82 percent of the hackers are already using AI in their work.
Pen testing has split into two worlds
Dave's first move is to cut the AI and pen testing conversation in half, because they are really two different stories.
The first is human testing augmented by AI. Attack surface management handles the recon and scoping up front, large language models take over the report writing at the end, and the tester spends their skill where it counts: on methodology, on the paths through the application or the network that only a human notices. The second story is what Dave now calls agentic pen testing, fully automated testing that he frames as finally fulfilling the promise of what DAST was always supposed to be. He remembers his years at WhiteHat, where one of the largest R&D expenses was keeping the rules engine current as new threats emerged. AI fills the gap that rules-based engines never could.
The consequence is economic as much as technical. Nobody wakes up excited to buy their next pen test, and the report is stale the moment the dev team ships. What changes with agentic testing is the unit economics: the same pool of dollars that bought a handful of point-in-time human tests can now cover an estate of ten thousand assets continuously, with human testers reserved for the assets that matter most, the customer portal, the wire transfer flow, the crown jewels. It is the same shift we described in our own machine-speed threat analysis: the constraint is no longer finding problems, it is deciding and acting fast enough.
The 22-hour clock
Jason raises the question every compliance-driven program eventually hits: if a point-in-time assessment proves a control existed on the day somebody looked, what is it worth when exploitation runs machine to machine?
Dave's answer is honest on both halves. Attestations still carry weight, because your customers and your supply chain partners need some way to evaluate you, and an incident at a partner with an API into your platform blows back on you either way. But the regulatory landscape will stay behind the technology, and the math has changed underneath it. Watching the zero-day clock, Dave puts the gap between vulnerability detection and a working exploit at roughly 22 hours.
"You've got to be talking about response or remediation times in hours, not days anymore."
We have tracked the same collapse from the defender's side: discovery got faster, patching didn't, and threat campaigns like agentic ransomware and agentic exploitation of mid-market targets are already operating on that clock.
Ten out of ten thousand
The heart of the episode, and the line the title comes from, is Dave's diagnosis of where the hard problem has moved.
"Detection's the easy part. The hard part now is how do you take all of those vulnerabilities that have been identified, pick the ten from the ten thousand that actually matter?"
Vulnerability detection is commoditized. Homegrown AI, frontier models, open source models: they all find things. What separates programs now is triage, prioritization of where fixes have the biggest impact, and whether the downstream operators, security engineers and developers, get enough context to issue effective fixes quickly.
And then there is the part the industry has failed at for twenty years: actually fixing things. Dave is blunt that remediation is the biggest challenge facing the industry, and that the incentive problem underneath it has not moved. Developers are not paid to write secure code, they are paid to ship code that drives revenue. He sees AI-driven patching as the only plausible way through, but notes the new problem it creates: customers suddenly holding thousands of vulnerabilities they did not know existed three months ago, with legal liability and customer duty attached to fixing them fast.
Drowning in findings nobody is fixing?
Z Cyber turns vulnerability noise into a prioritized, board-ready remediation program with owners, timelines, and evidence.
What the underwater CISO does right now
Jason puts the practical question: what do you tell the CISO who does not have budget for an AI remediation tool? Dave's answer will sound familiar to anyone who has read our episode three conversation on building a program in 2026: get back to fundamentals. Secure coding practices, knowing your architecture, knowing which APIs you actually use and whether you test them effectively.
The encouraging half is that the defender has the same machine-speed capability at their fingertips as the attacker. Good-enough open source models are dramatically cheaper than frontier models or vendor tools, and good enough is often enough. The caveats are real, some of these models come out of China, so lock down anything that phones home and put controls around them, but a budget-constrained team is not out of the game. As with the budget conversation from episode two, the ask that gets funded is the one framed around clarity, not anxiety.
Prioritization when criticality changes in real time
Does prioritization itself change? Absolutely, Dave argues. Criticality can change in real time, and vulnerability chaining is getting easier than ever, which breaks the old habit of sorting by CVSS score or severity label and working down the list. You have to understand your estate, your architecture, and what you are actually trying to protect. That demands a deeper partnership between the CISO and the CTO, aligned on incentives and mission, a theme that goes straight back to episode one on the translation gap.
"Security is not the CISO's problem, security is the company's problem."
And the boring corners of the estate no longer get a pass. The legacy infrastructure nobody wanted to look at is exactly where machine-speed testing goes first, because, as Dave puts it, AI does not care how boring it is. The much-referenced 27-year-old OpenBSD bug is not the worst vulnerability ever found, but it made the point: AI is great at finding what humans miss, not as an indictment of the human, but because it moves faster, goes deeper, and does not get bored.
The failure is process, not people
Asked where programs actually fail across people, process, and technology, Dave starts by dismantling the answer leaders reach for first. The talent gap, he argues, is largely a recruiting problem. The industry recruits from computer science programs at tier one schools and calls everything else a shortage.
"If you've got motivation and you're coachable, you can be taught to do anything."
Bugcrowd has dropped degree requirements, and Dave says so while sitting on university cybersecurity program boards. Jason's own hiring history agrees: his best hires were the people who took a goal, taught themselves through labs and YouTube university, and came back having done the work, not the ones who recited a memorized OWASP top ten rather than admit they would need to look something up.
The real failure lives in process. Most organizations run processes that were not built for an AI-first era, and Dave's example is one every GRC team will recognize: a SaaS tool clears vendor review, then six months later ships AI capability, and nobody routes it back through diligence. Users see a productivity feature; the organization has just opened its customer data to an unvetted AI tool. It is the same exposure we mapped in our shadow AI discovery and governance guide and the reason third-party AI risk assessment cannot be an annual form. The answer is not buying more tools. It is adapting the process, then leveling up the team inside it.
"You're using AI. Shouldn't it be cheaper?"
Jason closes with the question every AI-powered service provider now gets from buyers. Dave's first answer is the honest one:
"I'm happy to show anybody our token bill and I can promise you it's not cheaper."
The fuller answer is that the question misunderstands what changed. Hackers on the platform were already using Burp, scanners, and homegrown automation harnesses before AI arrived; the fee bought outcomes then and buys outcomes now. What AI changes is the unit economics of coverage: the same aggregate spend now buys deeper, continuous coverage across the whole estate. And in a market where every vendor booth at RSA looks identical and every board is mandating something with AI in it, both men land on the same conclusion: you are no longer buying software, which is commoditizing by the month. You are choosing a partner you trust to have your back when something goes sideways, the same standard we apply in the GRC platform versus security advisor decision and our guide to evaluating advisory providers.
The one question
The Blind Spot closes every episode the same way: pick a side of the table, and name the one thing you wish the other side understood. Dave stays loyal to the operators and aims his answer at boards and investors: understand how hard the operating environment is right now. Submissions into Bugcrowd's platform are up 334 percent year over year, customers have more choice and more noise than ever, and everybody, board members included, answers to someone. His resolution is the one that runs through the whole episode: focus on doing right by customers and employees, make the long-term bets, and the end result takes care of itself.
Listen to the full episode
The full conversation runs about 43 minutes and is worth the time for any security leader rethinking testing, triage, or remediation for the machine-speed era. Watch or listen here:
Dave Gerry is the CEO of Bugcrowd, an offensive security platform connecting organizations with a global community of 700,000 security researchers. New episodes of The Blind Spot are released every two weeks. If your findings pile is growing faster than your fix rate, talk to a Z Cyber advisor.
Frequently Asked Questions
What is agentic pen testing?
Agentic pen testing is fully automated penetration testing carried out by AI agents rather than by a human tester working through a methodology. It handles reconnaissance, scoping, and continuous testing across an attack surface, and it fulfills much of what dynamic application security testing (DAST) always promised but could not deliver with rules-based engines. In practice it pairs with human testing: AI covers the breadth of an estate continuously, while human testers focus on the highest-value assets such as customer portals and payment flows.
How fast are vulnerabilities exploited after they are discovered?
The window between a vulnerability being detected and a working exploit being built has collapsed to roughly 22 hours. That means remediation timelines built around fixing a critical finding in seven days no longer match the threat, and response has to be measured in hours. It also means a point-in-time assessment or attestation can be out of date within an hour of being issued, because a new vulnerability or a new release can change the picture immediately.
How should security teams prioritize vulnerabilities when everything looks critical?
Detection is now the commoditized part of the problem. The harder work is triage: picking the ten findings out of ten thousand that actually matter, prioritizing where fixes have the biggest impact, and giving downstream engineers enough context to fix them quickly. A CVSS score or severity label alone is no longer a sufficient lens, because vulnerability chaining and real-time changes in criticality mean prioritization has to account for the estate, the architecture, and what the business is actually trying to protect.
Can a budget-constrained security team still take advantage of AI?
Yes. Defenders have the same machine-speed capabilities at their fingertips as attackers. Good-enough open source models are far cheaper than frontier models or vendor tools, and can be used effectively if they are run with the right controls, such as locking down anything that phones home and vetting where the model comes from. The bigger lever is still security fundamentals: secure coding practices, understanding the architecture and the APIs in use, and testing them effectively.
Who is Dave Gerry?
Dave Gerry is the CEO of Bugcrowd, one of the largest offensive security platforms in the world, connecting organizations with a global community of roughly 700,000 security researchers. He has spent most of his career in application and product security, including six years at WhiteHat Security, and he sits on the boards of several university cybersecurity programs. He is the guest on episode four of The Blind Spot, Z Cyber's podcast.
Subscribe for Updates
Get cybersecurity insights delivered to your inbox.
