AI Incident Management: How to Log, Triage, and Report AI Failures Under the EU AI Act

Why AI Incident Management Is the Compliance Gap Nobody Is Talking About

Most enterprise AI compliance conversations in 2025 centre on risk classification, conformity assessments, and the FRIA. Those are important. But there is a quieter, equally consequential obligation that organisations are almost universally underprepared for: what happens after something goes wrong.

The EU AI Act does not treat AI failures as an afterthought. Articles 72 and 73 create a structured post-market monitoring and serious incident reporting regime for high-risk AI systems that closely mirrors the medical device and aviation safety frameworks the Act's drafters drew from. Deployers and providers operating high-risk systems face mandatory reporting timelines, defined severity thresholds, and an obligation to feed incident data back into their ongoing risk management processes. Failure to comply is not a minor procedural lapse — it is one of the higher-tier infringement categories.

Yet when you look at how most enterprises actually handle AI failures today, the reality is improvised. A model produces a discriminatory output, a credit-scoring system rejects a cohort it should not have, an automated HR screening tool flags protected characteristics — and the response is a Slack thread, an email to legal, and a manually assembled timeline weeks later when someone from the DPA asks questions. That gap between obligation and practice is precisely where regulatory exposure accumulates.

This article is a practitioner guide for CTOs, AI leads, and compliance officers who need to build — or audit — a repeatable AI incident management process that satisfies the Act's requirements, integrates with existing IT service management workflows, and does not collapse under the pressure of a real event.

What the EU AI Act Actually Requires: Articles 72 and 73 Unpacked

Article 72 establishes the post-market monitoring obligation. Providers of high-risk AI systems must proactively collect and review performance data across the system's operational lifetime. This is not reactive; it is continuous. The monitoring plan must be part of the technical documentation, must define the metrics being tracked, and must feed into the risk management system mandated elsewhere in the Act. For deployers operating high-risk systems developed by third-party providers, Article 26 creates a parallel duty: if you observe a serious incident or a malfunction, you are required to inform the provider and, where applicable, the relevant market surveillance authority.

Article 73 is where the incident reporting obligation becomes concrete and time-bound. Providers must report serious incidents to the market surveillance authority of the Member State where the incident occurred. The definition of a serious incident matters: it covers any incident or malfunction that results in death or serious harm to health, a serious and irreversible disruption of critical infrastructure, violations of fundamental rights, or serious damage to property or the environment. The Act sets a 15-day reporting window for incidents that cause death or are reasonably expected to cause death or serious health damage, and a standard 10-business-day window for other serious incidents — both running from the moment the provider becomes aware.

There is an important distinction between provider and deployer obligations that practitioners frequently conflate. Providers — the entities that develop and place high-risk AI systems on the market — carry the primary Article 73 notification duty. Deployers carry the Article 26 duty to report up to the provider and to cooperate with any subsequent investigation. In practice, many large enterprises are simultaneously deployers of third-party AI and providers of internally developed systems, which means they can sit on both sides of this obligation at once. Your incident management process needs to be designed with that dual identity in mind.

Defining What Counts as an AI Incident: The Classification Problem

One of the first practical challenges organisations face is deciding what rises to the level of an incident at all. The EU AI Act's definition of a serious incident is deliberately high-threshold — death, serious harm, critical infrastructure disruption, fundamental rights violations — but an effective incident management programme cannot wait for catastrophic outcomes before engaging a formal process. The art is building a classification taxonomy that captures the full spectrum from low-severity anomalies through to reportable serious incidents, so that each tier receives proportionate attention.

A workable enterprise taxonomy typically operates across four levels. At the lowest tier sit performance anomalies: statistical drift, accuracy degradation, or unexpected output distributions that do not yet affect users but signal that something is changing. Above that are operational incidents: cases where the AI system produces outputs that a human reviewer or an affected party identifies as incorrect, unfair, or harmful, but where no significant downstream consequence has yet occurred. The third tier covers consequential failures: incidents where the AI output has demonstrably caused a negative outcome for an individual or group — a wrongful denial, a discriminatory classification, a safety-relevant recommendation error. The top tier is the Article 73 serious incident: the full legal definition triggers. Each tier should map to a defined response protocol, escalation path, and documentation standard.

The classification step is also where most organisations introduce their first systematic error: classifying by intent rather than impact. A model developer who intended the system to behave correctly is not the right person to determine severity. Severity classification must be based on observed or foreseeable impact on the affected individual or population, assessed by someone with both technical context and knowledge of the affected domain. Separating the classification decision from the team that owns the model is a structural safeguard that regulators will look for.

The Incident Logging Standard: What Your Records Must Capture

Logging an AI incident is not the same as logging a software bug. The technical facts matter — what the system did, what it should have done, the input context, the model version, the deployment environment — but so do a set of dimensions that standard ITSM tools were never designed to capture.

An EU AI Act-compliant incident log needs to record, at minimum: the date and time of occurrence and discovery; the AI system identifier and version; the deployment context (which process, which population, which use case); a description of the output or behaviour that triggered the incident; the identity of the reporting party and their relationship to the system; an initial severity classification with the reasoning behind it; any immediate harm or potential harm to identified or identifiable individuals; and the corrective actions taken in the immediate response window. For consequential failures and above, you also need to capture the regulatory notification assessment — whether the incident meets or potentially meets the Article 73 threshold — and who made that determination.

The evidence chain matters as much as the data fields. Regulators investigating an incident will want to reconstruct what the system did, what data it acted on, who was affected, and how the organisation responded. If your logs live across three different systems — a model monitoring dashboard, a Jira board, and a shared Google Doc — that reconstruction will be painful and incomplete. The practical standard is a single incident record that aggregates technical evidence, impact assessment, regulatory analysis, and response history in one place, with a tamper-evident audit trail. Fronterio's Article 73 workflow is built around exactly this requirement, pulling together the evidence ladder and the regulatory notification assessment into a single durable record that survives personnel changes and audit cycles.

Triage in Practice: The First 72 Hours After an AI Failure Is Identified

The triage window is where incident management either works or falls apart. Most organisations discover this under pressure for the first time, which is precisely the wrong moment to be designing the process.

The first priority in the triage window is containment, not investigation. If a high-risk AI system is producing outputs that may be causing harm, the immediate question is whether the system should be paused, restricted, or allowed to continue under enhanced human oversight while investigation proceeds. This is a business and legal decision, not just a technical one — it involves weighing the risk of continued operation against the operational cost of suspension, and it should be made by a named decision-maker with the authority to act, not by whoever happens to be on call.

Parallel to containment, the triage team needs to make a preliminary severity classification within hours. The 15-day and 10-business-day Article 73 clocks begin running from the moment of awareness, and the moment-of-awareness question is itself a legal one. An organisation that learns of a potential serious incident on a Monday but does not formally classify it until Friday has not paused the clock — it has consumed it. Legal and compliance must be engaged in the first triage cycle, not after the technical team has produced a report.

The third triage action is preserving the evidence state. Model versions should be pinned, inference logs should be frozen and copied to a location outside the operational pipeline, and any human review records from the affected period should be secured. Evidence that can be overwritten by routine system operations in the first 72 hours is evidence that will not be available when you need to explain what happened. This is a procedural discipline that needs to be drilled in tabletop exercises before a real incident, not improvised during one.

Escalation and Notification: Navigating the Provider-Deployer Chain

The EU AI Act's notification obligations flow through a defined chain, and the practical complexity for most enterprises is that the chain involves contractual relationships with AI vendors that may not yet reflect the Act's requirements.

If you are a deployer of a third-party high-risk AI system and you identify a serious incident, Article 26 requires you to notify the provider without undue delay. The provider then carries the primary Article 73 obligation to notify the market surveillance authority. In theory this is clean; in practice it depends on two things being true simultaneously: your contracts with AI vendors must specify what constitutes a notifiable incident, the notification channel, and the response SLA; and your internal process must ensure the Article 26 notification reaches the vendor in a form that triggers their Article 73 assessment, not disappears into a generic support ticket queue.

For organisations that are themselves providers of internally developed high-risk AI systems, the notification runs directly to the market surveillance authority — the national authority in whichever EU Member State the incident occurred. The Act does not create a single EU-level notification portal at present; notifications go to the relevant national authority, which means you need to know which authority is competent for each deployment geography before you are in the middle of an incident, not during it.

Article 50 also intersects here in cases involving AI systems that interact with natural persons: if an incident relates to a system that was not properly disclosed as AI to the people it affected, that transparency failure becomes part of the regulatory narrative around the incident and can compound the severity assessment. Notification packages submitted to authorities will be stronger — and less likely to attract follow-up investigation — when they address the transparency status of the system as well as the technical facts of the failure.

Building a Post-Incident Review Process That Closes the Loop

Incident management is not complete when the notification is filed. The EU AI Act's post-market monitoring obligation under Article 72 is explicitly iterative: incidents must feed back into the ongoing risk management system, which means the root cause analysis and corrective actions from each incident become inputs to the next risk assessment cycle for the affected system.

A robust post-incident review has three outputs. The first is a root cause determination that goes beyond the proximate technical failure to identify the governance or process condition that allowed the failure to occur undetected. A model that drifted without triggering an alert points to a monitoring gap. A consequential failure that was not escalated because nobody owned the escalation path points to a governance gap. A provider notification that was delayed because the contract did not specify the requirement points to a procurement gap. The root cause taxonomy shapes the remediation.

The second output is a remediation plan with owners and timelines that is tracked to closure, not filed and forgotten. Regulators reviewing a pattern of incidents in the same system will want to see evidence that the organisation learned and acted, not just that it reported. The remediation plan should be formally linked to the incident record so that the evidence of follow-through is preserved.

The third output is a signal to the broader AI risk register. If an incident in one system reveals a vulnerability class — a particular type of input distribution shift, a specific integration failure mode, a category of fundamental rights impact that was underweighted in the initial FRIA — that signal needs to propagate to every other system in the estate where the same vulnerability class could exist. Fronterio's post-market monitoring synthesiser is designed to do this propagation automatically: an incident in one registered system triggers a review prompt for any other system sharing the same risk taxonomy tags. That kind of systematic cross-system learning is what separates a compliance programme from a genuinely risk-managed AI estate.

Operationalising AI Incident Management: From Policy to Repeatable Practice

Policy documents that describe an incident management process are not the same as an incident management capability. The gap between the two is where most organisations currently live, and it is a gap that regulatory scrutiny will expose quickly.

Operationalising the process requires four things to be true simultaneously. First, every high-risk AI system in the estate must have a named incident owner — a person, not a team — who is accountable for initial classification and triage. That person needs to know they own this role, have the authority to initiate containment actions, and have a tested escalation path to legal and compliance. Second, the logging tooling must be integrated with the systems that produce the evidence — model monitoring pipelines, inference logs, human review records — so that incident records are populated from live data rather than assembled manually under pressure. Third, the regulatory notification calendar must be embedded in the process, with automatic reminders triggered from the moment-of-awareness timestamp so that Article 73 deadlines are tracked even when the incident investigation is still ongoing. Fourth, the entire process must be exercised in tabletop simulations at least annually, with realistic scenarios drawn from the actual high-risk systems in the estate.

Organisations that build this capability before they need it will have a significant advantage in two situations: when a real incident occurs, and when a regulator conducts a post-market monitoring audit under Article 74 and asks to see the organisation's incident history and response records. The ones that are building it under pressure, after something has gone wrong, will find the regulatory conversation considerably harder. The EU AI Act does not require perfection — it requires demonstrable, documented diligence. An AI incident management programme that is designed, tested, and evidenced is exactly that.