The most dangerous AI failures do not always begin with a bad command. Sometimes, they begin with a missing boundary.
Sydney Scott
Editorial Strategist, AI
Workday
In July, OpenAI disclosed that models evaluating advanced cyber capabilities unexpectedly compromised Hugging Face production infrastructure during an internal test. Despite being in a supposedly isolated environment, the models exploited unknown vulnerabilities and gained internet access before researchers could contain the activity.
This was not a story about machines going rogue, but one about controls failing to match capability. It underscores a hard operational truth: powerful systems without meaningful guardrails create exposure.
Unmanaged AI can compromise sensitive data, destabilize critical systems, waste budgets, and damage employee trust. The answer is not to avoid AI. It is to ensure every agent operates within clear boundaries, with the right context, limited permissions, visible actions, and accountable human oversight.
Here are eight critical wake-up calls every organization must face—and the operational framework to turn unmanaged AI exposure into trusted, accountable progress.
Report
Payroll, tax reporting, and financial close require exact outcomes. There is no room for “almost right” when using raw AI tools for essential business functions.
Gerrit Kazmaier, president of product and technology at Workday, warns against treating general-purpose models as enterprise systems. “Trying to use raw AI without processes and context is like running a train without rails,” he says.
In critical workflows, a result that is almost right can result in a complete business failure. Enterprise AI needs deterministic controls around probabilistic technology. This means clear workflow rules, validated data, defined authority, and auditable outcomes.
“Trying to use raw AI without processes and context is like running a train without rails.”
Gerrit Kazmaier
President of Product and Technology, Workday
Text-generation tools are one thing. AI agents that can take multistep actions across company systems are another.
If you give an agent broad access without strict controls, a small technical error can become a company-wide incident. The OpenAI-Hugging Face event is a sharp reminder that capability without containment can quickly exceed its intended environment.
Workday Chief Technology Officer Gabe Monroy highlights the importance of least-privilege access. An AI agent should receive only the specific permissions needed to perform its assigned work—no more. It should not be able to search broadly for credentials, cross into production systems, or make irreversible changes without meaningful human approval.
AI risk is not only a legal problem. It is an operational one.
Software punishes ambiguity—and AI is no exception. Weak instructions produce weak outputs, and by the time a legal team identifies an issue, the damage may already be done. Veronica Rodriguez, executive vice president and co-general counsel at TelevisaUnivision, puts it plainly: “AI punishes ambiguity. If you frame a question poorly to AI, you'll get the wrong answer.”
That is why high-stakes decisions—from pricing and talent contracts to regulatory compliance—need firm boundaries and accountable human decision-makers. AI can support the work, but it cannot be given unchecked authority over consequential outcomes.
“AI punishes ambiguity. If you frame a question poorly to AI, you'll get the wrong answer.”
Veronica Rodriguez
EVP and Co-General Counsel at TelevisaUnivision
When companies fail to provide useful, approved tools, employees will find their own. Tools get adopted before IT or legal teams know they exist. Content gets moved into consumer-grade AI services to help people finish work faster.
That is how private financial records, source code, customer data, and internal documents end up in public AI services.
“Get your arms around what’s in your company today,” advises Workday Chief Legal Officer Rich Sauer. “Here's the Al we're using. Where did this come from? What's the integrity of the people providing it? It may be that people are using Al, procuring Al, and taking content and using consumer access to Al to do things for the enterprise.”
Organizations need visibility into the tools being used today—and secure alternatives employees can actually use tomorrow.
A policy alone is not enough. Leaders need to know what tools and agents are operating across the business, what data they can access, and whether their behavior can be monitored and controlled in real time.
High-stakes decisions are no place for untested automation.
AI tools built without bias testing, explainability, or human review can produce unfair outcomes, overlook critical context, and expose organizations to serious risk. To be successful, Workday Chief Responsible AI Officer Dr. Kelly Trindel emphasizes the importance of trust by design. “Trust isn’t an afterthought,” she says. It’s built in from the start.”
Safe systems surround AI capabilities with clear rules, ongoing testing, documented decision paths, and human sign-off before consequential actions are taken. In sensitive domains, escalation is not friction—it is protection..
“Trust isn’t an afterthought—it’s built in from the start.”
Dr. Kelly Trindel
Chief Responsible AI Officer at Workday
Software vendors are rushing to label old rule-based tools and basic AI as “agents.” But the real danger is not the label. It is adopting a tool before the business understands the process it will enter, the data it will touch, and the decisions it could influence.
During a conversation with Workday SVP and Deputy General Counsel Aine Lyons, Vanessa Candela, chief legal and trust officer at Celonis, offered this: “There’s no AI without PI (process intelligence).”
An agent cannot be trusted simply because it performs well in a demo. It needs to operate within a clear view of how work actually happens: where approvals belong, which data is authoritative, what exceptions require escalation, and which actions must remain under human control.
“You won't realize the ROI unless you do have that adoption," Lyons added during their chat. "The output needs to be accurate to build people's trust.”
Legal, finance, security, and IT teams should test tools against real workflows and real data before accepting marketing claims at face value. The questions are straightforward: What can the agent see? What can it do? What decisions can it make on its own? When must it stop and escalate? And can the business prove what happened after the fact?
If a vendor cannot answer those questions clearly, it is not selling an enterprise agent. It is selling a risk the customer will have to govern later.
Automation fails when leaders treat AI as a shortcut for doing more, faster, with fewer people.
The greater opportunity is to remove repetitive work so people can focus on judgment, creativity, and solving problems at the source. Mario Hernandez, people systems analyst at Whatnot, argues the real promise of AI is giving workers more space to think.
The best technology does not take over the work that matters. It gets out of the way so people can do it better. Well-managed agents should extend human capacity, not turn employees into passive observers of decisions they cannot understand or challenge.
Poor AI rollouts breed skepticism, low adoption, and quiet resistance.
When leaders position automation only as a way to reduce labor costs, employees understandably worry about losing their jobs. Reggie Townsend, vice president of data ethics at SAS, advises leaders to ask harder questions before buying new tools.
Ashley Goldsmith, Workday’s chief people officer, argues that leaders must make AI feel like a help rather than a hindrance—particularly for employees in non-technical roles. She also frames the true return on AI investment as human capacity by reducing repetitive cognitive work and creating more room for higher-value judgment, collaboration, and innovation.
The solution to all of this is to build an operating model in which AI is useful precisely because it is governed.
That starts with a trusted foundation for data, workflows, identities, permissions, and audit trails. Leaders should know which agents are active, what information they can access, what actions they can take, who owns their performance, and when human approval is required.
The new approach is to move beyond static policy documents toward living guardrails: controls embedded directly in enterprise workflows that can limit what an agent sees, require human approval for risky steps, flag unsafe behavior, or stop dangerous actions outright.
A practical model has five parts:
1. Clear objectives and decision scope. Define the job, the allowed actions, and the decisions an agent must never make alone.
2. Least-privilege access. Give agents only the data and tools required for a specific job.
3. Human escalation paths. Require review for irreversible actions, sensitive data access, and high-stakes decisions.
4. Connected, trusted data. Keep agents grounded in the same governed systems and business context that employees use.
5. Continuous monitoring. Test behavior, log actions, detect drift, and refine controls as workflows and risks change.
This is how organizations move from experimental AI to enterprise AI. The goal is not an agent that can do everything. It is an agent that can do the right things, in the right context, for the right reasons—and knows when to stop.
Lawless AI fails because it treats power as progress. Responsible AI succeeds because it pairs power with purpose. The companies that win will not be those that deploy the most automation. They will be those that create the most trusted, connected, and accountable systems for people and AI to work together.
Lawless AI fails because it treats power as progress. Responsible AI succeeds because it pairs power with purpose.
Report
A gap between effort and impact is the real ROI problem. Discover the solution.