AI Incident Evidence
Checklist
A structured checklist for preserving evidence after an AI security incident — covering containment, evidence preservation, root cause analysis, and remediation.
Educational notice: This checklist is a general framework for AI incident evidence preservation. Your organization may have specific legal, regulatory, or contractual obligations that require additional steps. Coordinate with your legal team and compliance officers before an incident occurs — not during one.
Synopsis
AI security incidents differ from traditional application security incidents. The evidence includes prompts, model outputs, tool call chains, RAG retrieval context, MCP server interactions, and authorization decisions that may have been made by a model rather than a human. This four-phase checklist ensures you preserve the right evidence, reconstruct the timeline, identify the root cause, and document everything for compliance and legal purposes.
Who Needs This Checklist and At What Level
Not every organization needs the full four-phase evidence preservation plan. Use this guide to determine your required level of preparation.
Level 1: Full evidence plan
Level 2: Core evidence plan
Level 3: Basic evidence plan
Important: If you use AI in hiring, lending, healthcare, or any consequential decision — regardless of company size — you need Level 1. The data sensitivity and regulatory exposure determine the level, not just company size. A 10-person startup using AI to screen resumes in NYC needs the full evidence plan because NYC LL 144 applies.
Monitoring & Tracking to Set Up Before an Incident
Evidence preservation only works if you are already logging the right data. Set up these monitoring capabilities proactively — you cannot retroactively capture logs that were never enabled.
Prompt and response logging
Tool call audit trail
RAG document provenance tracking
Model version and configuration history
User access and authentication logs
Data egress monitoring
Bias audit and decision logs (hiring AI)
Candidate notification records
AI in Hiring: Evidence, Notification & Disclaimer Requirements
AI in hiring is one of the most heavily regulated AI use cases. Multiple jurisdictions enforce specific notification, audit, and evidence preservation requirements. If you use AI to screen, rank, score, or evaluate candidates, these obligations apply to you.
NYC Local Law 144 (AEDT) — In effect since July 2023, actively enforced
Who: Any employer or employment agency using an Automated Employment Decision Tool (AEDT) to substantially assist in screening candidates for hiring or promotion in New York City.
Bias audit: Annual independent bias audit by a qualified, independent third-party auditor. The vendor cannot audit their own tool. Audit must include selection rates, impact ratios, and scoring distributions across protected categories.
Public disclosure: Summary of bias audit results must be publicly available on the employer's website. Include the audit date, auditor name, and key findings.
Candidate notice: Notify candidates at least 10 business days before the tool is used. Notification must state that an AEDT will be used and describe the tool's function. Notice can be in the job posting, on the careers website, or via email — but it must be specific and actionable.
Penalties: $500 for the first violation, $1,500 for each subsequent violation. Each day the tool runs without a current bias audit is a separate violation. Each candidate who does not receive notice is a separate violation. A single non-compliant hiring season can result in tens of thousands of dollars in fines.
Evidence to preserve: Bias audit reports, candidate notification records, AEDT configuration and version history, selection decisions with timestamps, and all data provided to or generated by the AEDT.
Illinois — Two active laws
AI Video Interview Act (AIVIA, in effect since 2020): Requires notice before the interview that AI will analyze the video, explanation of what characteristics the AI evaluates, written consent from the applicant, and deletion of the video within 30 days of applicant request. Videos may not be shared except with those evaluating the candidate.
HB-3773 (effective January 1, 2026): Amends the Illinois Human Rights Act — using AI that results in discrimination in employment decisions is a civil rights violation. Imposes affirmative notice requirements when AI is used for recruiting, hiring, promotion, or other employment decisions. Unlike many AI statutes, this law focuses on discriminatory effect, not just intent.
Evidence to preserve: Video interview consent records, AI analysis results, notification records, deletion request logs, and all employment decision data showing AI involvement.
California FEHA Regulations — In effect since October 1, 2025
Who: All employers covered by the Fair Employment and Housing Act (generally 5+ employees) using automated decision systems (ADS) in employment decisions.
Requirements: Unlawful to use ADS in a way that discriminates on a protected basis. Anti-bias testing and proactive efforts are central evidence in discrimination claims. Employers must preserve automated-decision system data — including data provided by or about applicants, data reflecting employment decisions, and data used to develop or customize the ADS — for 4 years.
Vendor liability: California extended liability to AI vendors under an agency theory (Mobley v. Workday). If a vendor's tool disparately impacts protected groups, both the employer and the vendor are on the hook.
Evidence to preserve: All ADS data (inputs, outputs, scoring criteria, training data), anti-bias testing results, documentation of testing quality and scope, and records of responsive actions taken when risks were identified.
Colorado SB 189 — Effective January 1, 2027
Who: Deployers (employers) and developers of automated decision-making technology (ADMT) used in consequential decisions affecting Colorado residents, including employment.
Requirements: When an adverse decision is made using ADMT, provide disclosures within 30 days including a plain-language explanation of the decision, the role of the ADMT, instructions for requesting further information, and rights including meaningful human review. Records must be retained for at least 3 years. 60-day pre-enforcement cure period sunsets January 1, 2030.
Evidence to preserve: ADMT decision records, disclosure records, human review documentation, and all data supporting the adverse decision.
EU AI Act — Annex III: High-Risk (Employment)
Classification: AI systems for recruitment, selection, promotion, termination, task allocation, and performance monitoring are classified as high-risk under Annex III. There is no de minimis carve-out for small employers.
Article 5 prohibitions (in effect since February 2025): Emotion recognition AI in the workplace is banned. AI interview tools that infer enthusiasm, confidence, or cultural fit from facial micro-expressions are illegal in the EU.
Article 50 transparency (in effect): Candidates must be informed when they are interacting with an AI system.
Article 26 deployer obligations (high-risk regime, August 2026 or December 2027 per Digital Omnibus): Human oversight with authority to override, inform workers' representatives before deployment, retain automatically generated logs for at least 6 months, fundamental rights impact assessment (FRIA) where required.
GDPR Article 22: Right not to be subject to solely automated decisions with legal or similarly significant effects. Candidates must be offered human review of any solely automated rejection.
Evidence to preserve: System logs (6-month minimum), human oversight records, FRIA documentation, candidate notification records, worker representative consultation records, and conformity assessment documentation.
Other State Requirements
Texas HB 149 (effective January 1, 2026): Prohibits AI deployed with intent to unlawfully discriminate. Narrower than other states — disparate impact alone does not establish a violation. Stricter restrictions on government use of AI in employment.
Maryland HB 1202 (in effect since October 2020): Requires consent before using facial recognition technology in hiring. Maintain consent records as evidence.
Pending legislation: Washington, New Jersey, Massachusetts, DC, and several other states have introduced bills that would impose bias audit requirements, disclosure mandates, or impact assessment obligations for AI hiring tools. The trajectory is clear: what NYC introduced is becoming a national standard.
Required Candidate Notifications & Disclaimers — Checklist
Use this checklist to ensure compliant candidate notifications across jurisdictions:
- Job posting disclosure: State that AI will be used in the screening/evaluation process. Required by NYC LL 144 (10 business days before use) and EU AI Act Article 50.
- Pre-assessment notification: Before any AI-administered assessment (test, video interview, resume scoring), notify the candidate of: (a) AI use, (b) what the AI evaluates, (c) how results will be used, and (d) their right to request a human review (EU AI Act, Illinois AIVIA, Colorado SB 189).
- Video interview consent: Obtain explicit written consent before AI analysis of video interviews. Include right to request deletion within 30 days (Illinois AIVIA).
- Adverse decision disclosure: When AI contributed to a rejection, provide: (a) notice that AI was used, (b) plain-language explanation of the decision, (c) the role of the AI system, (d) instructions for requesting more information, and (e) right to human review (Colorado SB 189, EU AI Act, GDPR Article 22).
- Biometric data disclaimer: If any biometric data is collected (facial analysis, voice patterns), provide a separate biometric data notice and obtain explicit consent (Illinois BIPA, Maryland HB 1202). Do not use emotion recognition AI — it is banned in the EU and high-risk everywhere else.
- Data retention notice: Inform candidates how long their data and AI-generated assessments will be retained (4 years under California FEHA, 3 years under Colorado SB 189, 6 months for logs under EU AI Act).
- Bias audit availability: In NYC, the bias audit summary must be publicly accessible. Include a link or reference in the job posting or careers page.
- Vendor disclosure: If the AI tool is provided by a third-party vendor, disclose the vendor name and provide contact information for data requests (GDPR Article 13/14, various state laws).
Regulatory Notification Timelines — Quick Reference
AI incidents may trigger statutory notification deadlines. Missing these deadlines can result in separate fines and penalties. Assess obligations early in Phase 3 — do not wait until the investigation is complete.
| Regulation | Who must notify | Deadline | Notify whom |
|---|---|---|---|
| HIPAA (PHI breach) | Covered entities and business associates | 60 days from discovery to HHS; affected individuals without unreasonable delay | HHS Secretary, affected individuals, media (if 500+ in a state) |
| GDPR Article 33 | Data controllers and processors | 72 hours from awareness | Relevant supervisory authority; affected data subjects without undue delay if high risk |
| EU AI Act Article 73 | Deployers and providers of high-risk AI systems | 15 days for serious incidents; 48 hours for incidents causing harm to health or safety | Relevant market surveillance authority and competent authority |
| CCPA / CPRA | Businesses handling California resident data | Without unreasonable delay; no later than 90 days from discovery | Affected California residents; California Attorney General (if 500+ residents) |
| State breach laws (general) | Entities handling state residents' PII | Varies by state: 30–90 days from discovery | Affected residents; state Attorney General (thresholds vary) |
| NYC LL 144 (hiring) | Employers using AEDTs in NYC | No incident notification, but bias audit must be current at all times | NYC DCWP (audit compliance); candidates (10 business days before use) |
| Colorado SB 189 (ADMT) | Deployers of ADMT in consequential decisions | 30 days from adverse decision to provide disclosure | Affected individual |
| SEC (public companies) | Public companies subject to Reg S-K Item 1.05 | 4 business days from determination of material cybersecurity incident | SEC via Form 8-K |
Important: These are general guidelines. Consult your legal team for jurisdiction-specific requirements. Some states have shorter deadlines or additional notification content requirements. The clock starts at discovery, not at full investigation completion.
Evidence Chain of Custody
Preserved evidence is only useful if its integrity can be proven. A broken chain of custody can render evidence inadmissible in legal proceedings or regulatory investigations.
Hash and timestamp everything
Append-only evidence storage
Access logging on evidence
Custody transfer documentation
Incident Communications: Who to Notify and What Not to Say
Communication during an AI security incident can make or break the response. Poor communication can trigger regulatory penalties, litigation, and reputational damage that exceeds the incident itself.
Internal notification order
External notification protocol
What NOT to do
Privilege considerations
Post-Incident Report — Required Sections
The post-incident report is the deliverable that regulators, legal counsel, and executives will review. Include all of these sections to ensure completeness.
- Executive summary: One-page overview — what happened, when, impact, root cause in one sentence, remediation status. Written for non-technical executives.
- Timeline of events: Chronological log from first anomalous event through containment, evidence preservation, root cause analysis, and remediation. Include timestamps in UTC.
- Incident scope: What data was affected, how many users were impacted, which systems were involved, what tools were called, what MCP servers were connected.
- Root cause analysis: The complete chain — trigger, propagation, impact, detection. Identify whether the cause was adversarial input, configuration error, code defect, or vendor issue. Include evidence references (log IDs, hash values).
- Evidence inventory: List of all preserved evidence with hashes, storage locations, access logs, and chain of custody documentation.
- Regulatory notifications: Which regulators were notified, when, what was communicated, and what deadlines were met. Include notification content and delivery confirmation.
- Remediation actions: What was fixed, what controls were added, what tests were run to verify the fix, and what remains open. Include target dates for open items.
- Lessons learned: What went well, what went poorly, what would be done differently. Include process improvements, tooling improvements, and training needs.
- Risk register updates: Which risk register entries were created, updated, or closed as a result of this incident. Reference the AI Risk Register.
- Sign-off: Incident commander, security lead, legal/compliance officer, and executive sponsor. Include date and signature for each.
AI in Lending: Evidence & Compliance Requirements
AI used in credit decisions, loan underwriting, and insurance pricing is subject to fair lending laws. If you use AI to evaluate, score, or decide loan, credit, or insurance applications, these obligations apply.
ECOA / Regulation B — Adverse Action Notices
Who: Any creditor using AI (including third-party models) for credit decisions — including fintech companies, banks, credit unions, and alternative lenders.
Requirements: When AI contributes to a credit denial or adverse action, you must provide an adverse action notice within 30 days stating the specific principal reasons for denial. "The AI said no" is not a valid reason — you must identify the specific factors (e.g., insufficient credit history, high debt-to-income ratio). If the AI model is a black box, you have a compliance problem.
Evidence to preserve: Model inputs used for each decision, model output (score, recommendation, factors), adverse action notice sent to applicant, and the model version and configuration at time of decision.
Fair Credit Reporting Act (FCRA)
Who: Entities using AI to generate credit scores or consumer reports, including CRAs and users of consumer report information for credit decisions.
Requirements: If AI-generated scores are used as consumer reports, FCRA requires reasonable procedures to ensure maximum possible accuracy. Consumers have the right to dispute inaccurate information. If AI contributes to an adverse action, the user must provide the consumer with the credit score used and key factors.
Evidence to preserve: AI-generated scores, input data, dispute records, reinvestigation results, and adverse action notices with score disclosure.
CFPB Guidance on AI in Credit Decisions
Adverse action explanations: The CFPB has confirmed that creditors using complex AI models must still provide specific, accurate reasons for denial. Citing "complex algorithms" or "proprietary models" is not sufficient. The creditor must understand and explain what factors drove the decision.
Anti-discrimination: AI models must not result in disparate impact on protected classes. Even if the model does not use protected characteristics as inputs, proxy variables (zip code, name, occupation) can create discriminatory outcomes. Regular fair lending testing is required.
Evidence to preserve: Model explainability documentation, fair lending testing results, disparate impact analysis, proxy variable audit, and all decision records with factor explanations.
State Insurance AI Laws
Colorado Insurance Regulation 7-42-17: Requires insurers using AI for underwriting or pricing to maintain a governance framework, perform bias testing, and document decision logic. Evidence of the AI governance program and testing must be available to the Colorado Division of Insurance on request.
Other states: Connecticut, Illinois, and several others have introduced similar insurance AI accountability requirements. The National Association of Insurance Commissioners (NAIC) has adopted a model bulletin on AI use in insurance.
Sample Incident Walkthrough
A realistic scenario showing how the four phases connect in practice.
The scenario
An internal Streamlit AI assistant for a law firm allows paralegals to query case documents using RAG. The assistant has an MCP tool that sends emails on behalf of the user. At 2:47 PM on a Tuesday, a paralegal reports that the assistant sent an email containing confidential case strategy to an opposing counsel's address — without the paralegal requesting it.
Common Mistakes in AI Incident Response
Restarting the application before preserving state
Treating it as a traditional security incident
Not preserving RAG retrieval context
Failing to capture MCP tool call records
Not documenting the authorization chain
Delaying regulatory assessment
Critical: Do NOT restart, redeploy, or "fix" the application before preserving evidence. Once state is lost, the incident chain may be unreconstructable.
Phase 1: Immediate Containment (0–1 hour)
- Disable the affected AI application or feature (stop the bleeding)
- Revoke API keys and access tokens associated with the incident
- Preserve the current application state — do NOT restart or redeploy
- Capture running process list, memory state, and active connections
- Snapshot any cloud resources (VMs, containers, serverless functions)
- Notify the incident response team and designate an incident commander
- Begin a contemporaneous incident log (who, what, when, why)
- Preserve all browser tabs, Streamlit sessions, and user screens if applicable
Phase 2: Evidence Preservation (1–4 hours)
- Export all application logs (API calls, model interactions, tool executions)
- Preserve all prompts and model outputs from the incident window
- Export Streamlit Session State snapshots if the application is still running
- Capture all tool call records: function name, arguments, results, authorization decision, approver identity
- Export authentication and authorization logs (who was logged in, what roles, what RLS filters were applied)
- Preserve RAG retrieval logs: query, retrieved chunks, chunk metadata, tenant filters applied
- Export MCP server logs: tool discovery, tool calls, server responses
- Capture model API request/response logs including timestamps, model version, token counts
- Preserve any cached data that may have been involved (global cache, session cache)
- Export database audit logs and transaction records for the affected time window
- Capture network flow logs (egress connections, especially to external APIs or model vendors)
- Preserve any uploaded documents or files associated with the incident
Phase 3: Root Cause Analysis (4–24 hours)
- Reconstruct the timeline: first anomalous event, detection, containment, resolution
- Identify the attack vector: direct injection, indirect injection, tool abuse, auth bypass, data leakage, supply chain
- Determine which user or system identity was involved
- Identify which data was accessed, modified, or exfiltrated
- Determine whether RLS or tenant isolation was bypassed and how
- Identify which tools were called and whether authorization was properly enforced
- Determine whether the model output was the cause or a symptom
- Check for prompt injection patterns in user inputs or retrieved documents
- Review MCP server logs for anomalous tool definitions or calls
- Assess whether the incident was caused by configuration, code, or adversarial input
- Document the complete chain: trigger, propagation, impact, detection, containment
- Identify any compliance obligations triggered (breach notification, regulatory disclosure)
Phase 4: Remediation & Documentation (1–7 days)
- Implement immediate fix (patch, configuration change, tool removal, access restriction)
- Conduct negative-access testing to verify the fix
- Test for similar vulnerabilities in other AI applications or integrations
- Update the AI risk register with the new finding and remediation status
- Document the incident in a formal post-incident report
- Prepare evidence package for legal, compliance, or regulatory purposes
- Notify affected parties (customers, users, regulators) if required
- Update security controls: tool allow-lists, RLS rules, approval workflows, monitoring rules
- Conduct a post-incident review with all stakeholders
- Update the incident response playbook with lessons learned
- Schedule follow-up security review for 30 days post-remediation
- Consider whether the incident warrants disclosure under TRAIGA, EU AI Act, or other applicable regulations
Download the AI Incident Evidence Checklist
FAQ
How is an AI security incident different from a traditional application security incident?⌄
Traditional incidents involve network intrusions, malware, or access control failures. AI incidents additionally involve: prompt injection (the attack vector is natural language, not code), RAG poisoning (the data source is the attack vector), tool abuse (the model is manipulated into taking actions), and model-driven authorization bypass (the application trusts model output for access decisions). The evidence is different too — you need prompts, model outputs, retrieved chunks, and tool call chains, not just network logs.
What if we don't have logging set up when an incident occurs?⌄
This is unfortunately common. Preserve what you can: Streamlit Session State if the app is still running, any cloud provider logs (API gateway, load balancer), database audit logs, and the model vendor's API logs if accessible. Document the logging gap as a finding in your post-incident report and implement evidence-grade logging as a P0 remediation item. The incident itself is evidence that your logging was insufficient.
Who should be on the AI incident response team?⌄
At minimum: an incident commander (coordinates response), an AI engineer (understands the application architecture), a security analyst (preserves evidence and performs analysis), and a legal/compliance officer (assesses regulatory obligations). For incidents involving PHI or financial data, add the relevant data owner. For MCP-related incidents, add the engineer responsible for the MCP server integration.
How long should we retain AI incident evidence?⌄
Follow your organization's retention policy, but at minimum: 7 years for incidents involving PHI (HIPAA), 6 years for EU AI Act compliance documentation, and until any litigation or regulatory inquiry is fully resolved. AI incident evidence should be retained longer than traditional IT evidence because AI-related liability (bias, discrimination, hallucination harm) may surface months or years after the incident.