AI Risk Lives in the Leash, Not the Dog

Here's a thought experiment. Take the most capable AI model available right now and put it in a box with no internet connection, no tools, no ability to write to any database, no way to send an email or update a record. Ask it a question. It answers. Maybe brilliantly, maybe badly. Either way, nothing in the world changes except the text on your screen.

Now connect that same model to your customer database, your ticketing system, your refund-processing workflow, and your outbound email queue. Suddenly the stakes are completely different. Not because the model got smarter. Because you handed it a set of keys.

That's the core of an argument that's been circulating in enterprise AI circles, and it's one that deserves more attention than it's getting. The debate over AI safety tends to fixate on model capability — how powerful is this thing, how aligned is it, could it eventually deceive us? Those are real questions. But for the organizations already running AI agents in customer-facing environments, the more urgent question is simpler and scarier: what have we actually authorized this thing to do?

The Difference Between Suggesting and Acting

There's a meaningful line between an AI that generates text and an AI that executes actions. Most of the early anxiety about AI in business centered on the first category — hallucinations, wrong answers, biased outputs, chatbots saying something embarrassing. Those problems are real, but they're fundamentally about content. A human reads the output. A human decides what to do with it.

Agentic AI is different. When an AI agent is wired into your systems, it doesn't just suggest. It does. It looks up the customer record, updates the case status, triggers the refund, sends the confirmation email. The human is often not in the loop at all — that's partly the point. Speed and scale are the whole value proposition.

Which means the blast radius of a mistake is no longer just a bad sentence on a screen. It's a refund issued to the wrong account. A case closed that shouldn't have been. A customer's data accessed in a context that violates their expectations or your compliance obligations. The error doesn't wait for a human to notice it before it propagates.

Philipp Heltewig, Chief AI Officer at NiCE, put it plainly in a LinkedIn post: the models themselves only generate text and propose actions. What converts those proposals into real-world consequences is the infrastructure wrapped around the model — the tools it can call, the systems it can reach, the permissions it's been granted. 'AI won't kill us (on its own),' he wrote. 'Its harness could.'

That word — harness — is doing a lot of work here, and it's worth unpacking.

What a Harness Actually Is

Think of the model as the reasoning engine. It takes in context, figures out what makes sense to do next, and proposes an action. That's all it does. The harness is everything else: the set of tools the model can invoke, the data sources it can query, the external services it can call, and critically, the rules governing when it's allowed to do any of those things.

A well-designed harness doesn't just connect the model to capabilities. It constrains them. It says: this agent can look up customer records, but only for the authenticated user in the current session. It can issue a refund, but only up to a certain dollar threshold, and only after logging the action. It cannot modify account ownership fields. It cannot access records outside the current support ticket context.

The harness is, in other words, the thing that determines how much damage a mistake can cause. A model that reasons poorly but operates inside a tight harness makes small mistakes. A model that reasons well but operates inside a loose harness — or no harness at all — can make large ones, quickly, at scale.

This framing maps neatly onto how we think about humans in organizations. A new hire might have excellent judgment, but you don't give them admin access to every system on day one. You grant permissions gradually, based on role and demonstrated need. You log what they do. You build in approval workflows for consequential actions. We've spent decades developing these controls for human employees. We're only beginning to apply the same logic to AI agents.

The Demo-to-Production Gap Nobody Talks About Enough

There's a specific failure mode that keeps appearing as organizations try to move AI from proof-of-concept to production, and it goes something like this: a developer or a small team builds something genuinely impressive. A voice bot that handles complex queries. An agent that can navigate a multi-step support workflow. It works beautifully in testing. Leadership sees the demo and wants it deployed.

What the demo didn't have: concurrent load from thousands of simultaneous users, security review, compliance sign-off, audit logging, alerting when something goes sideways, integration with the full stack of enterprise systems, and any of the governance scaffolding that production environments actually require.

Heltewig made this point directly in a conversation at NiCE Interactions earlier this year. You can take a capable model, a JavaScript environment, and an afternoon, and build something that looks like enterprise-grade AI. 'But does that demo hold up when I'm getting bombarded with 20,000 concurrent calls?' he asked. 'Does it hold up under the security and compliance reviews? Does it have things like auditing, logging and alerting?' The answer, he said plainly, is of course not.

This isn't a knock on rapid prototyping — that's a legitimate and valuable practice. The problem is when the prototype becomes the production system because the demo was good enough to get approval, and nobody built the harness before go-live.

Governing What's Already Running

Here's the thing that gets lost in the frontier AI debate: most organizations aren't waiting for some future superintelligent system to become a problem. They already have AI agents running. Right now. Handling customer interactions, accessing customer data, executing transactions. And in many cases, the governance around those systems is... incomplete.

Michael Weis, CRO at Synergy Group AI, made a pointed observation in response to the broader safety conversation: 'Why aren't we governing what's already running in our environments?' It's a fair challenge. The industry spends considerable energy debating whether frontier AI development should be slowed down. Less energy goes into auditing the permissions of the agents already embedded in production contact centers.

Weis framed it as a reorientation of the question: 'Maybe the question isn't whether we need to slow AI down. Maybe it's whether we've put the controls in place to safely keep moving forward.' That's a more actionable frame for most organizations. You're not Anthropic. You're not deciding the pace of frontier model development. You are deciding what your deployed agents are allowed to do today, and whether you have visibility into what they're actually doing.

The practical checklist here isn't complicated, even if executing it is. Do you know which agents in your environment have access to which data? Can you reconstruct exactly what an agent did in a given customer interaction if something goes wrong? Are permissions enforced at the infrastructure level, independently of the model itself, so the model can't self-authorize access it wasn't granted? If the answers to those questions are uncertain, the harness needs work — regardless of how capable or well-aligned the underlying model is.

The Frontier Conversation and the Contact Center Conversation Are Actually the Same Conversation

Anthropic CEO Dario Amodei has been arguing publicly that AI capabilities are advancing faster than the industry's ability to understand and control them — that safety work, alignment research, and interpretability need more time to catch up. Some of Anthropic's own researchers have made even starker assessments about where the trajectory leads.

It's easy to look at that conversation and think it's happening in a different universe from the one where a contact center manager is deciding whether to let an AI agent process refunds autonomously. But the underlying dynamic is the same. In both cases, the question is whether the capability being deployed has outrun the governance around it. In both cases, the answer depends less on the intelligence of the system than on the controls surrounding it.

The frontier version of this question involves alignment research and interpretability tools that don't fully exist yet. The enterprise version involves logging, permissions, audit trails, and escalation paths — things that absolutely do exist and can be implemented right now. Which makes the enterprise version both less dramatic and more tractable.

Amodei's argument is essentially: we need more time to build the harness before the model gets much more powerful. The enterprise argument is: we need to build the harness before we expand what the model is permitted to do. Same logic, different scale.

The Platform Question Underneath All of This

There's a layer to this that doesn't get discussed enough in the harness conversation: the quality of the harness depends heavily on the platform it's built on. An AI agent isn't just a model plus permissions. It's a model plus permissions plus the underlying CX infrastructure that handles routing, data access, conversation history, analytics, and integration with the rest of the enterprise stack.

Point solutions — standalone AI tools that do one thing well — can demonstrate impressive agentic capabilities in isolation. The challenge comes when those tools need to fit into a broader CX environment. Who owns the customer record? How does a handoff from the AI agent to a human agent preserve context? Where does the conversation history live, and who can audit it? How do learnings from agent interactions feed back into improving future performance?

These aren't model questions. They're platform questions. And they're exactly the kind of questions that surface when an organization tries to move from 'we have an AI agent' to 'we have an AI-powered CX operation.' The harness, it turns out, isn't just about permissions and security. It's about the entire ecosystem the agent operates within.

Heltewig put it simply: 'AI agents are really only as good as a platform that they stand on.' That's not a vendor pitch disguised as insight — it's a genuine architectural observation. An agent with great reasoning and terrible integration is going to create operational headaches that no amount of model fine-tuning will fix.

What Customers Actually Experience When the Harness Fails

Step back from the architecture for a second and think about what happens on the other end of a misconfigured AI agent. A customer calls about a billing dispute. The agent has access to their account, their payment history, their contact information. It has the ability to issue credits, update records, flag cases for review. Most of the time, this works fine.

But the harness was configured loosely. The agent can access not just the current account but linked accounts. It can issue credits without a dollar cap. It can close cases without human review. Now imagine the agent misidentifies the customer's intent, or retrieves the wrong account, or interprets a request to 'remove the charge' as authorization to cancel the service. The action executes before anyone notices. The customer's service is gone. Their account shows a credit that shouldn't exist. A linked family member's account has been modified.

None of that happened because the model was too smart or too dumb. It happened because the permissions were wrong. The harness failed.

Customer trust in AI-assisted service is genuinely fragile. People are willing to interact with AI agents when those interactions work. When they don't — especially when they don't in ways that affect money, access, or personal data — the damage to trust is significant and often lasting. That's a business problem, not just a technical one. And it's a problem that better harness design directly addresses.

The Unglamorous Work That Actually Matters

None of this is as exciting as debating whether AI will surpass human intelligence. Auditing tool permissions, implementing logging infrastructure, enforcing access controls at the platform level, building escalation workflows — this is unglamorous, operational work. It doesn't make for compelling conference keynotes.

But it's the work that determines whether AI agents in customer experience create value or create liability. The model is almost beside the point. A mediocre model inside a tight, well-governed harness will cause fewer problems than a brilliant model operating with unconstrained access to your systems and your customers' data.

The organizations that get this right won't necessarily have the most impressive AI. They'll have the most disciplined approach to what their AI is allowed to do. That discipline — boring as it sounds — is where the real safety work happens for most businesses. Not in the model weights. In the leash.