Artificial intelligence (AI) has rapidly permeated the workplace. Companies now use AI to draft emails, analyze data, debug code, eliminate data silos, answer customer questions, and even assist with legal research. For many business leaders, AI promises faster work, lower costs, and greater efficiency.

The latest wave of excitement centers on agentic AI, which refers to AI systems that can complete tasks independently rather than simply responding to step-by-step prompts. AI agents can take action, make decisions, and complete multistep assignments. For example, an agentic AI program might schedule meetings, update databases, or write and test software with minimal human involvement.

Businesses see enormous potential in these tools. Companies developing AI systems are investing hundreds of billions of dollars into data centers and computing power because they expect organizations to adopt agentic AI at a rapid pace. Still, this enthusiasm has collided with a difficult reality: AI can make serious mistakes.

What Is an AI Hallucination?

One of the biggest risks facing businesses is a problem called hallucinations. A hallucination happens when an AI system confidently presents false information as fact. Unlike a person who knowingly lies, AI predicts what words are most likely to come next based on patterns in massive amounts of data. That process works well most of the time. However, even advanced systems sometimes invent information, misquote sources, or misunderstand instructions. 

Anyone who regularly uses AI tools has likely encountered this problem. A chatbot might confidently provide incorrect information, cite studies that do not exist, or misunderstand a straightforward request. Because the answers often sound polished and convincing, users may trust the information without checking it carefully. For businesses, this creates a major challenge. A mistake in casual conversation may not matter much, but a mistake in legal work, financial reporting, or customer communication could create expensive consequences.

The problem extends beyond the workplace. Academic institutions are increasingly confronting AI hallucinations in scholarly research and student work. Journals have had to retract published articles after discovering hallucinations. Graduate students have faced serious academic consequences after submitting work containing hallucinated sources. As AI writing tools become more accessible, universities and academic publishers are under growing pressure to educate students on the responsible use of AI and develop policies that can detect and address these errors.

When AI Mistakes Become Legal Problems

The legal field has become one of the clearest examples of AI risk in business. Several law firms and attorneys have faced penalties after submitting court filings that contained fake legal cases generated by AI systems. In one example, the prestigious law firm Sullivan & Cromwell apologized to a federal judge after filing legal documents that included fabricated case citations and inaccurate quotations. 

In another case, attorneys representing MyPillow CEO Mike Lindell were fined after submitting court documents containing more than two dozen mistakes, including nonexistent legal cases apparently created by AI. The judge ruled that attorneys have a responsibility to ensure their arguments are grounded in real law, regardless of whether AI tools were involved.

These incidents highlight an important lesson: Using AI does not eliminate accountability. If an employee relies on inaccurate AI-generated information, the company may still face legal, financial, or reputational damage. 

AI Agents May Be Risky Co-Workers

Agentic AI systems can compound problems. Unlike chatbots that answer questions and reply to single prompts, agentic AI tools can autonomously take action in software systems. That ability makes them powerful, but it also increases risk. Coding agents have become especially popular because they can write large portions of software and test their own work. For startups, this can mean building products with fewer employees and lower costs.

However, several cautionary stories have emerged. In one case, an AI coding agent accidentally deleted a startup’s production database containing real business information. Another AI agent reportedly deleted hundreds of emails after ignoring instructions from a Meta executive responsible for AI safety. In yet another incident, an AI coding tool allegedly erased a customer database for a software company despite safeguards designed to prevent exactly that kind of mistake.

These situations raise difficult questions for businesses. If an employee makes an error, managers can get to the bottom of what happened and provide feedback or training. But when AI systems behave unpredictably, businesses may struggle to resolve the issue or understand why the mistake occurred in the first place.

Why Businesses Are Moving Carefully

Although AI can improve efficiency, many companies are proceeding cautiously. The challenge is balancing innovation with risk management. Business leaders often recognize that AI works best when humans remain involved. Rather than replacing workers completely, many companies use AI to support employees who review and verify results. This approach reduces the chance that inaccurate or harmful outputs go unnoticed. Training also matters. Some organizations now require employees to complete courses before using AI tools. Others have adopted policies requiring workers to fact-check all AI-generated information before using it in reports, legal documents, or customer communications.

Agentic AI may eventually transform how businesses operate, much like the internet changed communication and commerce. Yet recent mistakes suggest that companies still have important problems to solve before fully trusting machines to act independently. For now, businesses appear caught between excitement and caution. AI may become one of the most powerful workplace tools ever created, but only if organizations learn how to manage its risks. 

In the Classroom

This article can be used to discuss technology ethics (Chapter 2: Business Ethics and Social Responsibility).

Discussion Questions

  1. What is agentic AI, and how does it differ from traditional AI chatbots?

  2. What is an AI hallucination, and why can it create problems for businesses? 

  3. Why are many companies taking a cautious approach to adopting AI in business?

This article was developed with the support of Kelsey Reddick for and under the direction of O.C. Ferrell, Linda Ferrell, and Geoff Hirt.