How to Build an AI Incident Response Plan
An AI tool can fail without being “hacked.” It may expose confidential data, invent a damaging claim, make inconsistent decisions, call the wrong tool, send content to the wrong audience, or continue an automated action beyond its intended scope.
A conventional cyber incident response plan remains essential for compromised accounts, malware, credential theft, and data breaches. An AI incident response plan extends that capability to failures involving models, prompts, training or retrieval data, human oversight, automated tools, and third-party providers.
The goal is not to predict every failure. It is to make the first hour less chaotic, preserve evidence, limit harm, and create a disciplined path back to safe operation.
What counts as an AI incident
An AI incident is an event in which an AI system causes, contributes to, or materially increases the risk of harm, policy violation, service disruption, or loss of trust.
Examples include:
• a chatbot revealing confidential or personal information;
• an AI assistant sending an unauthorized message or changing a live system;
• a model producing false claims that are published without review;
• biased screening or prioritization affecting people unfairly;
• prompt injection causing a tool to ignore approved instructions;
• a third-party model changing behavior after an update;
• a retrieval system citing the wrong document;
• copyrighted, explicit, hateful, or unsafe content reaching users;
• repeated model errors that disrupt operations;
• a vendor outage affecting a critical workflow;
• an employee using an unapproved tool with business data; or
• a near-miss where a control prevents harm but exposes a weakness.
NIST’s AI Risk Management Framework organizes work around Govern, Map, Measure, and Manage. Its Generative AI Profile identifies risks including confabulation, privacy, information integrity, information security, harmful bias, human–AI configuration, intellectual property, and value-chain integration.
An incident plan should cover both actual harm and credible near-misses. Near-miss reporting helps a business fix controls before the next event reaches a customer.
Why a normal cyber plan is not enough
AI incidents can cross several disciplines at once.
A leaked prompt may be a privacy and security issue. A false product claim may create legal, customer, and reputation risk. An autonomous agent may trigger operational changes without a traditional intrusion. A model update can create quality drift even when every account remains secure.
The evidence also differs. Useful records can include:
• user prompts and model outputs;
• system instructions;
• retrieved documents and citations;
• model and version identifiers;
• temperature or configuration settings;
• tool calls and approvals;
• account and access logs;
• human-review decisions;
• vendor status information;
• evaluation results;
• timestamps and affected users; and
• changes to policies or prompts.
A cyber plan may already define incident leadership, communications, evidence handling, recovery, and reporting. Build on it rather than creating a disconnected process. Add AI-specific scenarios, evidence, owners, and stop controls.
The RESET response framework
The original RESET framework provides a memorable response sequence.
R — Recognize and record
Treat reports seriously without assuming every poor output is a major incident. Capture what happened, where, when, who or what detected it, the affected system, and the possible impact.
Preserve the exact output and surrounding context. Screenshots can help, but machine-readable logs are more useful when safely available. Record the model name, version, prompt, system instructions, retrieved sources, tool calls, and human approvals.
Do not ask employees to copy sensitive data into an unapproved channel for convenience. Use the existing secure incident process.
E — Escalate by impact
The person receiving the report should not decide alone whether the issue is harmless. Use documented severity criteria and notify the incident lead.
Escalate immediately when the event may involve:
• exposure of personal, confidential, regulated, or authentication data;
• unauthorized access or system changes;
• physical safety;
• discrimination or material impact on a person;
• harmful public content;
• legal or contractual notification duties;
• a large number of users;
• continued autonomous action;
• evidence tampering; or
• an unknown scope with a credible severe outcome.
Legal, regulatory, or contractual reporting timelines vary. Qualified counsel and relevant professionals should assess obligations.
S — Stop or contain
Containment should reduce harm while preserving evidence and essential operations.
Possible actions include pausing the workflow, disabling tool access, revoking an exposed credential, switching to a human-only process, restricting affected users, removing public content, or asking the provider to suspend a feature.
Avoid making broad changes without a record. Deleting logs, rewriting prompts, or updating the model immediately can erase evidence needed to understand the cause. If continued operation creates immediate danger, prioritize safety and document the emergency action.
Containment is not a final fix. A disabled chatbot may stop new harm while leaving exposed data, published content, affected customers, or connected systems unresolved.
E — Examine evidence
Build a timeline. Separate confirmed facts from assumptions. Determine:
• the first known occurrence;
• the affected systems, data, users, and decisions;
• whether the event is continuing;
• the model, provider, version, and configuration;
• the role of prompts, data, tools, permissions, and human review;
• controls that failed or succeeded;
• whether similar events occurred earlier;
• vendor or dependency involvement; and
• the worst credible remaining impact.
AI incidents rarely have one cause. A harmful output may combine an ambiguous policy, weak retrieval, excessive permissions, missing approval, and a model limitation. Fixing only the final prompt can leave the system vulnerable.
T — Transition safely
Recovery means more than switching the system back on.
Define acceptance criteria. Test the corrected workflow with representative and adversarial cases. Restore the minimum necessary permissions. Add temporary monitoring. Confirm that affected stakeholders received required communication. Obtain approval from the designated owner.
Then schedule a post-incident review. Record root and contributing causes, actions, owners, deadlines, residual risk, and lessons for other systems.
Build the plan before an incident
Start with an inventory. List every AI system used in business operations, including embedded AI features in software that may not be marketed as separate tools.
For each system, document:
• owner and business purpose;
• provider and model;
• users and access level;
• input and output data;
• connected tools and permissions;
• decisions influenced;
• human-review requirement;
• fallback process;
• logging available;
• vendor contact;
• known limitations; and
• shutdown or pause method.
Prioritize systems that affect people, money, safety, legal rights, confidential data, public communication, or irreversible actions. The plan can be lighter for low-impact brainstorming tools, but every tool needs an owner and reporting route.
Define the response team. A small business may have one person in several roles, but name the responsibilities:
Incident lead: coordinates decisions, timeline, and status.
System owner: understands purpose, configuration, and users.
Security lead: handles access, credentials, logs, and technical containment.
Privacy or legal contact: assesses data, contract, and notification duties.
Communications owner: prepares accurate internal, customer, and public messages.
Vendor contact: opens and tracks provider cases.
Executive decision-maker: accepts residual risk and approves restart for high-severity incidents.
Publish one simple reporting channel. Employees should know that reporting a mistake promptly is expected and that hiding an event creates more risk.
Create severity levels
Use impact-based levels.
Severity 1 — Critical
Actual or credible severe harm, large data exposure, physical safety risk, unauthorized high-impact actions, widespread public harm, or uncontrolled continuing behavior. Pause the system, activate leadership, security, legal, and communications immediately.
Severity 2 — High
Material customer, employee, privacy, security, or operational impact with limited scope or effective containment. Assign an incident lead and begin rapid investigation.
Severity 3 — Moderate
Incorrect or harmful behavior with low immediate impact, a contained policy violation, or repeated degradation that could grow. Fix within a defined period and monitor.
Severity 4 — Low or near-miss
Minor isolated error or control success with no material impact. Record it, correct the weakness, and look for patterns.
Severity can change as evidence develops. Use the highest credible level when the scope is unknown and a serious outcome is plausible.
Prepare evidence and communication
Create an AI incident record with:
Incident ID:
Date and time detected:
Reporter:
System and owner:
Model/provider/version:
Description:
Severity and reason:
Affected users, data, or decisions:
Current status:
Containment actions:
Evidence locations:
Known facts:
Open questions:
Vendor ticket:
Legal/privacy review:
Communications issued:
Recovery criteria:
Approval to resume:
Corrective actions:
Post-incident review date:
Communication should be factual, timely, and limited to confirmed information. Avoid blaming a vendor or employee before evidence supports the conclusion. Do not call an event “fully resolved” while monitoring or notification work remains.
If customers are affected, explain what happened, what information or function was involved, what the business has done, and what the customer should do. Legal and regulatory requirements vary, so use qualified advice for high-stakes communication.
Run a tabletop exercise
A tabletop exercise tests the plan without creating a real incident.
Choose a realistic scenario: an AI support bot reveals a private order detail; an agent sends unauthorized emails; a content tool publishes a false health claim; or a provider update changes a critical workflow.
Give the team information in stages:
1. A user reports the problem.
2. Logs show it happened more than once.
3. The model vendor confirms a recent update.
4. A screenshot is spreading publicly.
5. The scope remains uncertain.
Ask who leads, who can pause the system, where evidence is stored, when legal or security is called, how fallback service continues, and what message is sent.
The exercise should produce concrete fixes, not a score. Update contacts, permissions, logging, templates, and restart criteria.
Copyable AI incident response template
Purpose
This plan governs suspected or confirmed incidents involving AI systems, outputs, data, integrations, automated actions, or providers.
Report
Employees report immediately through [channel] and include system, time, output, affected user, and any visible evidence. Sensitive data must not be copied into unapproved tools.
Authority
[Role] may pause any AI workflow when credible harm is possible. [Role] approves restart for Severity 1 or 2 incidents.
First actions
1. Protect people and stop ongoing harm.
2. Preserve logs, prompts, outputs, configuration, and approvals.
3. Assign severity and incident lead.
4. Contain access, automation, publication, or affected accounts.
5. Notify security, privacy/legal, communications, leadership, and vendor as required.
6. Establish a timeline and investigation questions.
7. Use a documented fallback process.
8. Validate corrective controls before restart.
9. Monitor and communicate.
10. Complete a post-incident review.
Do not
Do not delete evidence, hide mistakes, publish speculation, repeatedly retry harmful outputs, restore full permissions without approval, or assume a vendor fix resolves every local control failure.
Frequently asked questions
Is every hallucination an AI incident?
No. Impact, context, repetition, and control failure matter. A harmless brainstorming error may be routine, while a false statement sent to a customer or used in a decision may require incident handling.
Who should be allowed to shut down an AI system?
At least one clearly named role and backup should have authority to pause high-risk workflows without waiting for a long approval chain.
Should prompts and outputs be retained?
Preserve relevant evidence through approved secure processes, subject to privacy, security, legal, and retention requirements. Do not collect more sensitive data than necessary.
When should a vendor be contacted?
Contact the provider when its model, platform, outage, security, logging, or update may contribute to the event. Maintain your own investigation and controls rather than relying only on the vendor’s conclusion.
How often should the plan be tested?
Test after material system changes and on a recurring schedule proportionate to risk. A six-month tabletop is a practical starting point for important AI workflows.
