Fronterio vs Fairly AI: Why LLM Red-Teaming Alone Isn't EU AI Act Compliance

Fairly AI's Strength: Adversarial Red-Teaming for LLMs in the Pipeline

Fairly AI built a product that solves a specific problem extremely well. If you are shipping a customer-facing LLM application and you want continuous assurance that your system prompt cannot be trivially jailbroken, that your retrieval-augmented generation does not leak PII under crafted probes, and that your agent refuses to comply with harmful instructions, their Asenion platform runs thousands of adversarial tests against your model on every pull request and fails the build when the attack success rate crosses a threshold. It is red-teaming at the cadence of CI, and that is a genuinely valuable primitive.

The technical depth is serious. Fairly's attack generation goes beyond the public lists of jailbreak prompts to include synthesised adversarial suffixes, multi-turn social-engineering chains, and targeted probes for known failure modes of specific model families. The platform reports pass/fail rates by attack category, tracks regression across model versions, and integrates as a native step in GitHub Actions or GitLab pipelines. For engineering teams whose models are under active development, this is the kind of tool that legitimately prevents production incidents.

The question for an AI governance buyer is whether this particular primitive is the binding constraint on their compliance posture. For a small number of organisations — those building bespoke LLM products with an active model pipeline — it is. For the majority of organisations deploying AI today, it is not. They are not building models. They are procuring Microsoft Copilot, configuring a Salesforce Einstein feature, rolling out an AI applicant tracking system from a vendor, or connecting Google Gemini to their workspace. The LLMs they use are operated by the provider. The red-team surface belongs to OpenAI, to Anthropic, to Microsoft — not to the deployer. A CI-based red-teamer does not have anything to attach to.

This is not a criticism of Fairly. Their tool is aimed at a specific buyer, and for that buyer it is excellent. It is a claim about what most AI governance buyers actually need, which turns out to be a very different set of capabilities.

The Classic-ML Gap: Tabular Drift, Credit Scoring, and HR Screening

Fairly's design centre is generative AI. That design centre is intentional — the founders correctly identified that GenAI risk was the category-expanding wave, and they went deep on it. The downstream effect is that classic machine learning — tabular-data systems, credit-scoring models, insurance pricing engines, HR screening tools — falls outside the product's natural scope.

Classic ML does not fail the way LLMs fail. An LLM fails by generating toxic output under adversarial prompts or by leaking information from its context window. A credit-scoring model fails by drifting over time as the distribution of applicants shifts away from the training data, by producing disparate approval rates across protected groups in ways that violate Equal Credit Opportunity regulation, or by being calibrated against base rates that no longer reflect reality. You cannot detect these failure modes with a prompt-injection attack. You need statistical drift detection, disparate-impact testing, and calibration monitoring — the kind of observability Holistic AI specialises in and that Fronterio addresses through its agent activity logs and operational monitoring report synthesiser.

This matters because many high-risk AI systems under the EU AI Act Annex III list are classic-ML systems, not LLMs. Article 6 explicitly scopes high-risk designation to AI used in recruitment and employment decisions, creditworthiness assessment, insurance pricing for natural persons, educational assessment, access to essential public services, and law-enforcement decisions. A large share of these remain classic-ML deployments. A governance platform that only covers the LLM attack surface leaves the tabular-ML attack surface uncovered — and for Annex III organisations, the tabular systems are usually the higher-risk deployment.

Fronterio does not try to be a deep statistical auditor — that is Holistic's position, and trying to compete there would sacrifice the adoption-first focus. What Fronterio does is treat every high-risk agent, regardless of whether it is an LLM or a classic-ML system, as a governed object with the same obligations tracker, the same FRIA requirement, the same human oversight plan, the same log retention, and the same incident reporting workflow. The obligations are regime-driven, not model-family-driven, which is how the EU AI Act actually works.

The Organisational Obligations Fairly Doesn't Solve

Even if the LLM red-team gate is running perfectly, a deployer is not compliant with the EU AI Act. The regulation imposes obligations that no pipeline test can discharge. Five of them are worth walking through in detail because they account for most of the actual compliance work.

Article 4 is AI literacy. Every organisation that operates AI systems must ensure sufficient AI literacy among staff who deploy or are affected by the systems. This is a training programme obligation. You need a defined curriculum, completion tracking per employee, evidence of effectiveness, and a refresh schedule as the AI portfolio evolves. A CI gate does not do any of this. Fronterio tracks training completion per user, mails weekly reminders to non-completers rate-limited to once per fourteen days, and advances the Article 4 obligation as the completion rate crosses thresholds.

Article 26(5) is operational monitoring. Deployers must monitor the operation of high-risk systems against the instructions of use, identify risks, and report them to the provider. This is a continuous operational discipline that requires log retention, anomaly detection, and a reporting channel back to the provider. Fronterio captures activity logs per agent, retains them for the required window, and surfaces override rates, complaint keywords in consultant messages, and incident spikes through the weekly operational monitoring report.

Article 27 is the Fundamental Rights Impact Assessment. For Annex III use cases in HR, insurance, credit, public-service access, education, and similar, a FRIA must be completed before the system is put into operation and notified to the relevant market surveillance authority. This is a structured written assessment, not a test result. Fronterio ships a guided FRIA wizard with locale-aware templates, a scoping helper that correctly identifies which agents trigger Article 27, and PDF export for notification.

Article 72 is post-market monitoring. For high-risk systems, providers and deployers must maintain ongoing performance monitoring and produce periodic reports. Fronterio's operational monitoring synthesiser runs weekly on Monday mornings, aggregates activity logs, incident records, complaint-keyword scans, and human-override rates, and produces a drift signal categorised as stable, warning, or alert. The report lands as a draft for a human to sign off, not as an auto-published document.

Article 73 is serious incident reporting. When a high-risk AI system causes a serious incident, the provider or deployer must report to the relevant market surveillance authority within fifteen days, and for the most serious incidents within forty-eight hours. Fronterio's incident workflow computes the reporting deadlines automatically from the classification — starting at the strictest statutory window by default, with the Act's deadlines ranging from two days to fifteen — and warns as each deadline approaches, plus a breach alert. Idempotency is built in so re-running the check does not re-send warnings.

None of these obligations are deliverable by a CI/CD red-team gate. They are the substance of deployer compliance, and they are where most of the operational work sits.

Governance at the register, not in the pipeline

The Fairly comparison does not end with compliance paperwork, because the two products meet different moments of the same lifecycle. Fairly's control point is the build: a red-team gate in CI that a new model version must pass before it ships. Fronterio's control point is the register: every agent — bought, built, or discovered — enters one governed record with a risk classification, an approval workflow, guardrail policies, and an audit trail that survives the people who wrote it.

The register-first approach matters because most deployers do not own the build. A mid-market company's AI estate is mostly vendor systems — Copilot, ChatGPT, a scattering of department tools — plus a handful of agents assembled on someone else's runtime. There is no pull request to gate. What there is, is a lifecycle: propose the agent, classify its risk under the EU AI Act, attach the human-oversight plan, approve it, monitor it in operation, and record every override and incident against it. Fronterio runs that lifecycle, and the guardrail policies attached to an agent's record — blocked actions, approval requirements, escalation rules — are the governance contract its operators are held to.

For the agents a customer does run on external platforms, Fronterio offers governed agent handoff to four supported targets — Azure AI Foundry, Copilot Studio, Claude Managed Agents, and a custom webhook — with a pre-handoff compliance check: agent approved, risk classification set, conformity assessment recorded, human oversight plan documented, and transparency requirements met. The MCP Server lets external systems query an agent's guardrails and governance state at runtime. Either way, the point is the same: Fronterio meets customers where their agents actually run, rather than requiring them to shift left into a red-team gate that presumes the customer owns the build.

When to Still Pick Fairly AI

Fairly has a clear, earned position. Pick Fairly when you are an engineering-led organisation building a generative-AI product, your model pipeline is active with frequent updates, and your primary risk is that a model update introduces a regression in safety or jailbreak resistance. In that environment, continuous adversarial testing catches problems before they ship, and the CI/GitLab integration surface is where your engineers already work. Running Fairly against every pull request is the right operational choice, and the alternative — running red-team tests manually on a release cadence — is both slower and less thorough.

Pick Fairly also when your organisation does not yet have a formal compliance function. If there is nobody whose job is to produce Article 27 FRIAs or track Article 4 literacy completion, a compliance-first governance platform will feel like overhead. Fairly's positioning inside the engineering organisation means adoption is easier — the tool is installed by the engineers who are already shipping the model, and the compliance paperwork can wait until the organisation hires for it.

A realistic mid-market trajectory is to run Fairly and Fronterio together. Fairly owns the red-team gate in CI. Fronterio owns the risk register, the obligations tracker, the literacy programme, the FRIA workflow, the incident-reporting deadlines, and the operational monitoring reports (Article 26(5)). They do not overlap. The engineering team uses Fairly, the compliance lead uses Fronterio, and the two products meet at the agent registration point: an agent that passes the Fairly gate enters the Fronterio register with its Article 15 robustness evidence already attached.

For organisations that do not need the red-team gate — deployers rather than builders — Fronterio alone is enough. Start on the Free plan to find where AI creates value, move to Pro for the deployer toolkit, and add Fairly later when and if the organisation starts building its own models. See /features/governance for the full governance surface.