MHI Logo

The Risks of Using AI in Risk Assessment: Understanding AI’s Limitations in Determining Warehouse Risks and Safety

SOLUTIONS COMMUNITY

Warehouse manager considering AI in risk assessment while holding a tablet, with digital charts and chat messages overlaid in front of pallet racking.

Artificial Intelligence or AI can do a lot of things and make our lives easier in many ways. However, it is important to be mindful that AI has considerable limitations when it comes to applications involving human and machine safety. . One of those applications is performing risk assessments and/or creating safety procedures in the warehouse. In this article, we will explain these limitations to help people understand what AI can and cannot do to assess the safety of warehouse facilities. 

What is AI? (No, Really, What is It?)

Today, when people think about  AI, they tend to associate it with Generative AI like LLMs when in reality it is a broad term encompassing many different categories with different applications and with unique strengths and weaknesses. .  The AI algorithms that beat you at chess, control your autonomous taxi, or predict the 3D structure of how proteins will fold are distinct from the LLM’s powering your chatbot. This article will focus on two AI subsets, Generative AI (GenAI) and Large Language Models (LLMs) which have the most focus and attention in the media and by far the type of AI systems that the general public regularly interacts with. 

Generative AI is defined as “A type of artificial intelligence that can learn from and mimic large amounts of data to create content such as text, images, music, videos, code, and more, based on inputs or prompts.”1 GenAI programs are fed large amounts of training data (text, images, music, etc.) which it then uses to create new content for an end user. It does this when users give these programs specific prompts or inputs telling the AI agent what to do or what the user wants. When you use AI to create an image or computer code, you are using GenAI. 

Large Language Models were the first AI models developed, and they were trained using large amounts of text (books, articles, newspapers, etc.). These models process that information for language generation. LLMs can process text in the following ways: generating, summarizing, translating, and analyzing. LLMs are the basis for many of the AI applications we already use like ChatGPT and Gemini.2

We need to acknowledge that GenAI and LLMs are powerful tools that are changing the way we work and how we develop and implement automated systems. However, we also need awareness of its weaknesses and limitations, especially if using it for warehouse safety and risk assessments. 

Using AI for Safety

AI can be very useful for researching and organizing safety standards, such as OSHA or electrical standards, and putting that information together so companies can draft procedures and develop risk assessments for their facilities. It is also well suited for summarizing information, which can save huge amounts of time by sifting through more information than even a team of humans could and it can be helpful by breaking down the niche technical jargon and legalese typically found in safety standards that can be difficult for lay people to understand. 

However, some companies are going beyond research and using AI to create safety manuals and protocols for their warehouses without confirming this information with safety professionals first. Other companies are taking AI even further and using it to generate their own risk assessments. While this can be an alluring strategy to save companies money, it could lead to significant safety lapses without proper oversight and fact checking by experts. 

Caution Ahead: AI’s Limitations 

When you understand how GenAI and LLMs work, it is easier to understand the technology’s limitations, so let’s examine how AI functions in order to learn its weaknesses.

Limitation 1) GenAI does not ‘think’ 

GenAI and LLMs are designed to generate words in a logical order using general information that has already been provided. Put another way, the AI model takes words and text that it has been trained on and puts them together in the most plausible way to answer a given prompt from the user. 

While the knowledge and information contained in the LLM’s training data is vast, it isn’t necessarily deep in every niche. The frontier of capability and intelligence of AI models is very “jagged” unlike human intelligence. We are accustomed to assessing other humans’ intelligence by observing their performance in one task and extrapolating to assume they have similar capability in many other unrelated tasks, but this very explicitly does not apply to LLM’s. The very impressive AI capabilities that have been shown in domains like software development or proving mathematical theorems makes it very easy to overestimate AI’s capabilities in other applications. The domains that are well suited to LLM’s like programming and mathematics have vast amounts of highly structured training data with easily verifiable outputs,  but this does not automatically translate to other knowledge domains like safety risk assessment and complex technical standards which have far less representation in the training data and have many grey areas and room for interpretation using human judgement.  

Although AI can quickly search for general safety standards, it may not have the training to provide specific answers for specific applications or situations, especially when it comes to fast-changing technologies like warehouse automation. When it comes to AI, the more technical or niche the topic being asked, the more likely it will produce an incorrect answer. 

Another example, while AI might have been trained using federal standards, it may overlook differences in standards for a specific state.Also, even including a specific state in the prompt does not guarantee a good result because AI models frequently and unpredictably mix and match  between states without explanation. If you ask for electrical regulations in Minnesota, AI might confidently give you an answer using the standards for Georgia or from a mix of several states without clarifying or indicating. 

The Lesson: This is why it is imperative to consult with a safety expert or professional, either on staff or from an outside agency, to make sure AI is providing the right information and including all relevant information for your situation.

Limitation 2) Because of how AI is trained, its responses appear confident and authoritative 

GenAI and LLMs are designed to generate the most plausible next word in their responses based on what other words and texts have been used in their training data. If the materials used to train AI were written or spoken in a confident manner (e.g., research papers or encyclopedia entries), the answer AI provides will also tend to be confident, even when the model is uncertain or does not know.

LLM’s are also fine-tuned using human feedback to help give the best sounding answer. However, the humans evaluating the output don’t necessarily know if the answer is true or not, just whether it sounds plausible and is worded in a pleasing way.  Both of these reasons  are why AI answers appear so confident and trustworthy to a layperson, even when the answer is wrong.

The Lesson: Even though AI may provide a confident and authoritative answer, the end user still must confirm the information and verify the sources AI used before accepting its response. 

Limitation 3) AI hallucinates

Hallucination is the term used when LLM’s make up answers that are not true or factual. Even the creators of OpenAI admit that there is no way to fully prevent this. That’s because AI was designed to provide answers. Therefore, when an AI model doesn’t know something, in an effort to provide an answer, it will fill in the gaps and make up a very plausible sounding lie. Unlike humans, AI does not say, “I don’t know,” because it wasn’t trained to do that. AI was designed to provide answers, therefore answers, both good and bad, are what the user will get. 

AI frequently hallucinates imaginary standards and regulations and refers to them as if they were real, or will apply a certain regulation to a completely different application which can have serious consequences. For example, giving an answer about safely storing hazardous chemicals in a warehouse while using regulations for a completely different type of material.   

The Lesson: Once again, end users need to follow up and verify with human experts the answers and sources they receive in an AI generated response to avoid basing their safety initiatives on hallucinations. 

Limitation 4) AI is designed to make you happy

It’s true. AI was designed to make users happy, to give us what we want. Think of it as a form of that old saying, “the customer is always right.” Technically, it’s called Reinforcement Learning with Human Feedback. When training GenAI and LLM models, humans provided feedback to the AI agent telling the agent if it did a good job or not with its response. With each new response, the AI model’s  goal is to get more positive feedback from its human reviewer. This effort to please is also what often leads to AI hallucinations and overconfidence. 

The Lesson: The best way to minimize AI sycophancy is for the user to provide neutral and unbiased prompts and inputs without leading language or unintentionally indicating your desired outcome. For example, if you ask GenAI a question this way, “I’d like to add more racks to my existing storage facility. Is this design OK?” Even if the design is not OK, AI will tell you it is because the AI model assumes you wanted that answer. 

Avoiding bias in AI prompts can be difficult, but not impossible. Sometimes simply having an awareness of bias before inputting prompts helps. Another technique is creating several different ways to ask your question and examining each for bias before inputting or inputting each question individually to see what kind of responses you get. Avoiding tags, like ‘is this OK’ or ‘should I do this,’ is another way to avoid sycophancy. Another good option is to craft a multiple-choice prompt, where AI has to decide between more than one item. That way you are not forcing it to agree with you. 

Limitation 5) GenAI is probabilistic, not deterministic 

When you ask GenAI or LLMs a question, they are not just looking up information in a database and regurgitating an answer. There are strategies like Retrieval-Augmented Generation (RAG) that help search for relevant and up-to-date information, but they are still generating their own answers based on the probabilities of the words it chooses going in a particular order. It’s where GenAI gets its name. 

That means that two people can ask an AI agent the exact same question and get two different answers. AI models are also static. Even though AI companies do updates and post-training on a regular basis, the world is dynamic and information changes constantly, which leads to what is called in the industry “model drift.” Also called model degradation, model drift means that as new information and data becomes available, the AI model has difficulty correlating its training data with the new data. Technologies like automatic storage and retrieval systems (ASRS), autonomous mobile robots (AMRs), or lithium-ion batteries, are dynamic and constantly evolving. AI programs might not have the latest information on these systems to provide current safety responses.

The Lesson: Knowing how GenAI and LLMs operate allows the end user to provide better prompts, verify the answers and sources given, and then double checking everything with an authoritative person or known field expert to ensure the response given is the one that will best help the company. 

How Unverified AI Use Effects Risk Assessments

While the obvious negative effects of misusing GenAI without oversight could be unsafe or even dangerous warehouses, there are other, subtler effects that could impact safety. Even if these issues don’t affect safety, they could affect the bottom line. 

1) Getting wrong information

If you trust the answer AI provides you without doing any verification, you could potentially implement incorrect safety measures or overlook significant safety risks in your facility. 

2) Not getting enough information 

Even if you verified the answer as good information, how do you know that answer covered everything that could go wrong in your warehouse? Unless you are a safety expert, you don’t. This could lead to blind spots in safety measures and programs.

3) Getting irrelevant information 

Responses could include information that doesn’t apply to your facilities. For example, AI could include information on cold storage even if your facility doesn’t use it. Information that is irrelevant will lead companies down rabbit holes that take time and resources away from doing what is needed to make the warehouse safe.

Another way irrelevant AI information can lead you astray is in risk probability. All types of equipment and machinery carry some kind of risk, but the probability of a specific risk happening could be high or low depending on how you use the equipment. AI might tell you to mitigate an issue that has a low probability of occurring while ignoring a high-probability factor, causing you to implement protections against things that are unnecessary or not protecting against high-risk situations at all. 

4) Getting bogged down in too much information 

Like irrelevant information, too much information also requires people to sift through pages and pages of details instead of moving forward. 

Understanding AI’s strengths and weaknesses can help you maximize the benefit to your organization and avoid potential pitfalls which could negate the significant efficiencies that can be realized when fully using AI to its strengths. 

Case Study: What Can Happen Relying on AI Alone

The following is a real situation involving a real company:  

A company installing automated car wash equipment decided to use an AI agent to create their safety programs and procedures. Because of the prompts input by the company, the agent didn’t find any safety standards that were specific to “car washes.” As a result, people at the company assumed that standards for “car washes” didn’t exist. The company continued using AI to create their own risk assessments and safety manual, which included items such as developing procedures for storing hazardous chemicals and where fire extinguishers were located. Then OSHA inspected the facility. The OSHA representative produced a 40-page report of violations and abatements resulting in issues that needed fixing and fines to be paid. In the end the company hired an outside firm to help them get out of this difficult situation. It was an expensive lesson. 

Ask AI, then Verify with a Pro

This article isn’t to say you can’t use AI at all. With the right prompts, AI agents can point companies in the right direction for safety regulations and risk assessment areas. AI can be a starting point, especially for small to mid-size companies that don’t have the budgets for in-house safety departments. However, companies should not rely on AI alone. 

Companies can look to public organizations for help with risk assessments. They can contact their local OSHA office or state and local government workplace safety agencies. These organizations have agents to help with questions or find areas that AI can overlook. And they can often provide these services for low or no cost. Other outlets include electrician or mechanical unions, especially if you have union members on staff. 

If a company doesn’t have an in-house safety department or officer, they can partner with a safety specialist, a company or consultant that has expertise in the type of warehousing that company does. The members of MHI’s Solutions Community have that level of expertise. Contact a member company today to learn your options for having a professional risk assessment done. 

Don’t leave the safety of your warehouse and the people working in it to AI alone. That is a risk not worth taking.   

Contributor: Nathan Hibbs of Beckhoff Automation LLC 

1Harvard University

2IBM

View more blogs by:

Related Content


MHI Logo

8720 Red Oak Blvd., Suite 201

Charlotte, NC 28217

Follow Us

Follow Us on LinkedInFollow Us on TwitterFollow Us on YouTubeFollow Us on FacebookFollow Us on Instagram

Copyright MHI © 2026 | All Rights Reserved