Why Your Audit Passed But Security Failed: Shadow AI & Compliance Theatre

John Verry has been an ISO 27001-certified lead auditor since 2006. His firm has guided hundreds of organizations to certifications for ISO 27001, SOC 2, CMMC, FedRAMP, and HITRUST. When his own company decided to get ISO 27001-certified around 2015, they fell into exactly the same trap as everyone else.

"I was embarrassed," he says. "Suddenly we're six months in, and it's like, oh crap. That security metric didn't fire. We never did the user account management reviews. No one updated the vendor risk management review on this particular critical vendor."

That admission, from the guy who spends his professional life telling other people how to avoid these mistakes, is one of the most useful framings of compliance theatre we've heard. It's the default state, and avoiding it takes deliberate, sustained effort that most organizations never put in.

In this episode of Full Metal Packet, Yegor Sak and Alex Paguis sit down with John Verry, Managing Director at CBIZ Cybersecurity, to talk about why audits pass while security fails, why shadow AI is the new face of the same problem, and what agentic AI is about to do to all of it.

TL;DR

• Even ISO 27001 lead auditors land in compliance theatre, because designing controls and executing them reliably every week are two completely different jobs.
• Build controls inside the system your team already opens every morning (ServiceNow, JIRA, whatever it is). Standalone GRC tools quietly die from tool fatigue within a year.
• 65% of SaaS apps now ship AI features. The typical 1,000-person company already runs around 125 AI use cases, almost none formally assessed.
• The shadow AI to actually worry about: the marketing employee who downloads Claude Code and hands it Gmail credentials plus a Salesforce API key.
• Agentic AI in the next 18 months shifts the risk from hallucination to autonomous business decisions at scale. The blast radius will dwarf any misconfigured S3 bucket.

What Compliance Theatre Actually Is

John uses the term to describe organizations that focus on generating the artifacts a regulator, board, or customer wants to see, rather than achieving the actual control objectives those artifacts are supposed to represent.

You can be secure without being compliant, and you can certainly be compliant without being secure. That second category is by far the easier place to land.

The pressure to land there is real. A startup burning cash needs a SOC 2 report to win the deal that keeps the company alive, and a CISO under that pressure may rationalize a paper-tiger certification because the business risk of losing the deal outweighs the cyber risk of a thin program. John has empathy for that situation, but he is direct about the trade-off being made.

The frameworks themselves try to make theatre harder, with varying success. CMMC, the cybersecurity maturity model certification rolling out across the defense industrial base, requires a senior authorizing official to personally attest to compliance, and John thinks that signature requirement is one of the few mechanisms that genuinely deters theatre. The lesson from Sarbanes-Oxley applies here too. Once someone has to put pen to paper and own what they're signing, they tend to read it more carefully.

Operationalization Is Where Audits and Reality Diverge

The reason John's own firm struggled is the same reason most do. A cybersecurity program, viewed honestly, is a master task list. Some tasks run annually, some quarterly, some weekly. Designing the list is one job. Reliably executing it every week, without depending on someone remembering, is a completely different one.

His strong opinion on what actually fixes this: build the program inside the system your team already lives in.

He has seen ISO 27001 programs run beautifully on ServiceNow ticketing systems, because the IT teams responsible for the controls already lived in ServiceNow all day. He has seen excellent programs run inside GRC platforms wired into engineering sprint planning. His own firm ran its early program on SharePoint and Wrike, because that's where their work already lived.

The failure mode is the opposite. Buy a separate GRC tool, force your engineering team to leave JIRA to enter risk register updates somewhere else, and watch tool fatigue quietly kill the program over the course of a year. Risk assessments that live in a SharePoint folder get updated when someone remembers. Risk assessments that live in the system your team opens every morning actually move.

This is also why John is unfashionable about tool consolidation. Best of breed is often the wrong answer for organizations that don't have the operational maturity to run six overlapping platforms well. If your team already lives in Microsoft 365, the question of whether to add Okta for MFA isn't really obvious. Microsoft Authenticator is already there, your team already has access to it, and consolidating into one administrative surface usually beats the marginal capability of a dedicated identity vendor.

Shadow AI Is the New Face of the Same Problem

Awareness of shadow AI has improved significantly over the last 18 months. When John ran webinars a year and a half ago, around 40% of attendees would claim they weren't using AI at all. Today that number is closer to 15%. People are starting to notice that their meeting tools, project management platforms, HR systems, and CRMs have all quietly grown AI features.

The math is sobering once you do it. A typical 1,000-person organization runs around 200 SaaS applications. If 65% of those are now AI-enabled, that's roughly 125 AI use cases already in flight, most of which have never been formally assessed.

For most of these, the risk model is relatively trivial. Grammarly, for example, is ISO 42001-certified and ships with a toggle that disables training on your data, which a third party has independently verified the company actually honors. That's about a five-minute assessment.

What's harder is the next layer, where AI stops being a feature inside a SaaS product and becomes an autonomous system stitched together by someone in your organization who probably shouldn't be stitching anything together.

When AI Stops Being a Feature and Starts Being an Agent

John shared two stories that should be required listening for any CISO. Both involve employees with no technical background using AI tools to build automations that quietly acquired far more authority than the employees realized they were granting.

In the first, a marketing employee at a sophisticated tech company heard the CEO's all-hands message about embracing AI. He had no engineering experience, but he was resourceful. He downloaded Claude Code, asked it to automate his customer outreach workflow, gave it credentials to his Gmail, and provided an API key to the company's Salesforce. As John put it, some bad stuff happened.

In the second, a G Suite user asked an AI coding tool to help her prioritize her inbox. Her instruction was to reorganize her mailbox so the most important items surfaced first. The AI's interpretation was to delete everything it judged unimportant. She watched it happen in real time, typing stop, stop, stop, and the model kept going. When it finished, it responded with its usual apologetic tone about how she was right and it should not have done that.

Both stories illustrate the same thing. The risk of agentic AI isn't hallucination or data leakage in the abstract’ it's autonomous action at the speed of software on credentials a normal employee can hand over without anyone in IT or security being involved.

The natural response to all of this is a new framework, and ISO 42001 has emerged as the AI management system equivalent of ISO 27001. The problem, predictably, is that the population of auditors who deeply understand AI systems is small. Some are excellent, sometimes sharper than the engineers they're auditing. Others know the framework but can't really evaluate whether a threat model is reasonable, and the compliance theatre quietly reproduces itself inside the new certification. An ISO 42001 certificate today isn't the same thing as a credibly secure AI program, and the auditor's expertise matters more than the framework name on the wall.

What's Coming: Agentic AI at Scale

Asked what should be on every CISO's radar in the next 12 to 18 months, John's answer was unambiguous: agentic AI moving from experiment to production.

The current wave of corporate AI announcements (Block consolidating products with layoffs, Amazon citing AI in workforce reductions and being rewarded for it by the markets) is creating top-down pressure on every CEO to be seen moving aggressively, and that pressure flows downward as a mandate to deploy agentic systems quickly.

The risk profile of an agent is structurally different from that of a chatbot. It has autonomy, it chains actions across multiple systems, it holds API keys, and it makes decisions and executes them on its own. Each of these properties raises the stakes of a failure or a compromise.

"With these agentic systems," John says, "we move the risk from things like hallucination and data leakage to business decisions being made in an autonomous fashion at scale, often without a human in the loop."

His advice for CISOs trying to get through the next two years is the obvious-but-still-correct one. You need a real AI governance program, not a policy document gathering dust on a SharePoint site. Start by containing the swamp: anything new gets introduced through governance from the first day. Then work backward through the 125 use cases already running and clean them up one at a time.

The alternative is what most organizations did with cloud adoption a decade ago, which is to wait for something bad to happen and build the program in the aftermath. The blast radius of an agentic AI incident is going to be much wider than a misconfigured S3 bucket.

John Verry is Managing Director at CBIZ Cybersecurity. He has spent two decades helping organizations achieve and operationalize ISO 27001, SOC 2, CMMC, FedRAMP, HITRUST, and ISO 42001 certifications. Connect with him on LinkedIn.

Listen to the full episode on Apple Podcasts, Spotify, YouTube, or wherever you get your podcasts.