The web is not a free AI training set

A public webpage is visible; it is not therefore a free input to an AI training pipeline. That is the useful starting point in the European Data Protection Board’s new draft guidance on web scraping for generative AI.

On 8 July, the EDPB opened Guidelines 03/2026 for consultation until 30 October. They do not prohibit web scraping, and they are not final law. They do make the uncomfortable boundary plain: a crawl that collects, stores, organises or retrieves personal data is processing under the GDPR. A public profile, forum post or company page can still contain information about identifiable people.

For a small company, the practical question is not whether the crawler can reach a page. It is whether the company could explain the dataset after the model team, the customer or a regulator asks what it contains.

Public is not purpose-free

The EDPB describes web scraping as large-scale automated extraction that often happens without the individuals’ awareness. That matters because a dataset is not defined only by its source. It is also defined by what the controller does next: collecting it, retaining it, combining it, training on it and serving a model that may reproduce patterns from it.

The GDPR’s ordinary disciplines still apply. Art. 5 requires a specified purpose, data minimisation and accuracy; Art. 6 needs a lawful basis. The EDPB’s draft applies those questions to a setting where an engineering team can otherwise mistake technical access for permission.

That distinction is useful in practice. A company collecting product specifications from manufacturers’ pages for a narrow supplier-search tool has a different file from one harvesting named experts’ posts, biographies and contact details to train a general-purpose assistant. The first question is still not whether the pages are indexed by a search engine. It is what personal data enter the collection, why they are needed, how long they remain useful and what the model is allowed to do with them.

Keep the trail beside the data

The Board recommends using reliable sources, recording timestamps and validating data before they are used for AI training. This is less glamorous than a new model evaluation, but it is the shortest route to an answer when an entry is challenged or a source changes.

An SME does not need a grand data-governance programme to start. It needs a small dataset record that travels with the crawler and the training job.

Question	Evidence to retain
What is the defined use?	A one-sentence purpose and the model feature it supports.
Where did the material come from?	Source domain, URL pattern, collection date and timestamp.
Why is personal data included?	Data categories, lawful basis and a necessity assessment.
What was checked before use?	Source reliability, sampling or validation method, and exclusions.
How can the data leave the system?	Retention period, deletion path and owner.

Source: EDPB Guidelines 03/2026 and Regulation (EU) 2016/679, Arts. 5–6. Last verified 2026-07-12.

The point is not paperwork for its own sake. A timestamp can show that a profile page was later corrected. A source field can make it possible to honour an objection or remove a faulty record. A short purpose field can stop a collection built for one narrow search feature from quietly becoming reusable training material for everything else.

Sensitive data changes the route

The difficult cases are rarely the clean product catalogue. Scraped pages can contain health information, political opinions, religious beliefs, trade-union membership or other special-category data. A crawler does not need to be looking for those fields to encounter them.

The EDPB is explicit: processing special-category data is prohibited in principle. Where it occurs, a controller needs both a lawful basis under Art. 6 and an exception under Art. 9(2). The Board notes a Court of Justice ruling that may matter for incidental or residual collection, but it also says there is no general escape from Art. 9; the facts and safeguards must be assessed case by case.

That is a reason to design collection rules before a crawl begins. Exclude sources that predictably centre on sensitive personal information. Filter known categories where technically possible. Sample the output before training. Give someone the authority to halt the job when the dataset drifts beyond its original purpose.

Removing a visible name after the fact is not a substitute for that work. In related July guidance, the EDPB frames anonymity around three tests: whether a person can be singled out, linked across data, or inferred from the information. A dataset that fails any of those tests deserves further analysis rather than an “anonymous” label.

Transparency has a practical limit

Scraping is often invisible to the people represented in the data. The EDPB therefore puts special weight on purpose limitation and transparency. It recognises that individual notices may be impossible or involve disproportionate effort in some designs, but that is not a switch that turns the rest of the GDPR off.

For a small firm, the sensible response is to reduce the amount that needs explaining. Do not collect every page because storage is cheap. Do not turn a one-off research crawl into a permanent reservoir by default. Do not assume an outsourced data provider has already answered the legal-basis question: ask for its sources, collection dates, categories, exclusions and deletion route in writing.

This is especially important for teams buying a model, enrichment service or dataset rather than building a crawler themselves. The controller’s file should show the product purpose and the supplier’s answer, not merely an invoice and a promise that the data was “public”.

A useful test before the consultation closes

The EDPB is still asking for feedback on the draft until 30 October. That means businesses should not present the guidance as settled interpretation. It does not mean waiting is the low-risk option. The questions in the draft are already a good pre-flight check for any dataset that will train, fine-tune or ground a generative system.

Start with one live dataset. Name its purpose. Trace ten records to their sources. Identify personal and special-category data. Check the lawful basis, the retention rule and the way a bad record can be removed. If those answers cannot fit on one page, the dataset is probably not ready for a wider model pipeline.

The web remains a valuable research surface. It becomes a liability when an organisation treats visibility as a substitute for accountability.