External AI evaluators enter the SME supplier file

External AI evaluation is moving from expert vocabulary into the supplier file. That is the useful reading for SMEs watching the AI Office build the European Centre for Algorithmic Vulnerability Assessment, or ECAVA, and open work on qualification requirements for external evaluators.

On 24 and 25 June 2026, the first international ECAVA Forum met in Brussels around AI evaluation. The next practical marker is already dated: the AI Office has invited expressions of interest for a 15 July 2026 workshop on external evaluator qualification requirements. This is not a new obligation for ordinary AI buyers. It is a sign that model assurance is becoming a formal market layer.

Buyers will feel it through vendors

Most SMEs will not evaluate a general-purpose model. They will buy a tool that depends on one, or a sector product that wraps one, then rely on the vendor’s statements about safety, documentation and change control.

That is where external evaluation matters. If the model layer is tested by qualified third parties, or if the provider claims to follow AI Act guidance for general-purpose AI models, the downstream buyer needs a way to record that claim. If the vendor cannot name the underlying model or explain how material model changes are communicated, the buyer has found a procurement gap.

The question is not “has Brussels certified this product?” It has not. The question is narrower and more useful: what model sits underneath, what evaluation evidence exists, who performed it, and what part of that evidence can the SME retain?

Supplier-file field	Minimum useful answer
Model provider	The company or organisation providing the base model.
Model family and version	Enough detail to know when the model changes.
System vendor	The seller wrapping the model into the business tool.
Evaluation claim	Internal, external, benchmarked, red-teamed or not stated.
Evidence retained	Public report, conformity note, vendor attestation or contract exhibit.
Change notice	How the buyer is told when the model or safety controls change.

Source: Regulation (EU) 2024/1689 and European Commission AI Office material on ECAVA, external evaluators and GPAI provider guidance. Last verified 2026-06-26.

That table is deliberately small. A ten-person manufacturer does not need a model-risk department to ask whether its maintenance assistant is built on a named model with a documented update path.

ECAVA is not a help desk

ECAVA’s public role is evaluation capability, not SME support. The forum language is about methods, ecosystems and expert cooperation. The workshop call is about qualification requirements for external evaluators. Those are upstream questions.

For buyers, upstream still matters because the AI Act splits responsibilities across the value chain. Art. 53 sets obligations for providers of general-purpose AI models. The buyer of an ordinary tool may sit far away from that legal role, but still depends on its output. If the model provider changes safety mitigations, documentation or terms of use, the SME’s operational risk can change without anyone touching the local workflow.

The lazy control is a supplier-file entry, not a second law degree:

• identify the model provider behind each material AI tool;
• keep the vendor’s latest AI Act or model documentation;
• ask whether evaluation is internal, external or unspecified;
• record how the vendor notifies model changes;
• schedule a re-check when the final guidance or evaluator requirements change.

That is enough to stop model provenance from disappearing into a sales demo.

Evaluation claims need verbs

“Responsible AI” is not an evaluation claim. Neither is “enterprise-grade” or “secure by design”. A useful claim has a verb and an object: tested against which risk, assessed by whom, documented where, updated how often, passed through to which customer.

This distinction matters more as external evaluators become visible. Some vendors will have real model-evaluation evidence. Some will have a policy page. Some will have a benchmark table that says little about the SME’s actual workflow. Procurement should not treat those as the same thing.

A buyer can ask four plain questions:

1. Which general-purpose model does this system use today?
2. What safety, cybersecurity or misuse evaluations are documented?
3. Were any evaluations performed by an external evaluator, and under what qualification route?
4. What written notice do we receive when the model, guardrails or data-handling terms change?

If the vendor cannot answer, the answer is “unknown”. That is allowed at the inventory stage. It is not allowed to disappear.

The contract should carry the file

The worst version of this work is a questionnaire that never enters the contract. It produces a clean spreadsheet and no durable right to information.

For material systems, the supplier file should map back to contract language. The buyer needs a right to receive updated documentation, notice of material model changes, a route for incident information, and enough record retention to answer its own internal governance questions. The point is not to push provider obligations onto a small vendor. The point is to stop every change in the model layer from becoming invisible to the deployer.

That is especially important where AI is embedded in staff, customer, safety or regulated workflows. A harmless drafting assistant and a customer-triage system do not deserve the same paperwork. But both deserve a named model, a named vendor owner and a dated evidence note.

Do the small thing before the big market arrives

External AI evaluation will likely become noisier before it becomes clearer. New evaluator labels, methods, workshops and guidance will arrive faster than most SMEs can interpret them.

The small response is still useful. Add one supplier-file page for every AI tool that matters. Name the model provider. Save the vendor documentation. Record any evaluation claim. Ask for change notices. Re-check the file when the AI Office publishes final qualification expectations or new GPAI guidance.

That does not make an SME an AI evaluator. It makes the SME a better buyer of systems that depend on evaluated models.