The EU AI Act Applies Beyond the AI You Buy

TL;DR

My argument in short: Many boards may think the EU AI Act is only relevant to the AI they bought, but it is so much more. One example outside that category is the AI they cannot list. For most organizations, demonstrating governance becomes difficult if they cannot answer, "Which AI systems do we operate?"

AI is Introduced Through Procurement, Updates, and Usage

Unlike AI capabilities that can appear through software updates, personal data inventories are generally more stable. Data maps are painful to build, but they stay roughly true. AI systems do not behave that way. You can have an organization with a chatbot someone piloted, a summarization feature switched on in an existing SaaS license, or a module that a vendor added in last release notes. None of these passed through a procurement process.

This is the “Shadow IT” problem all over again. Can organizations inventory AI embedded in software they already license, including capabilities introduced through vendor updates? In my view, the answer today for most organizations is no.

An inventory is not a one-off project. It is a process, and it needs a trigger such as change management. If a new AI capability in an existing product is introduced, then it should raise a flag, in a very similar way that a new data flow can raise a privacy review.

Invisible AI and Shadow AI

This is the same issue as Shadow IT, but with a focus on AI. You can have two categories:

Shadow AI: Users who choose AI without approval/governance

An employee copies or pastes a contract, a customer list, an email, or source code into a public AI because it potentially saves an hour of work. This is the familiar shadow IT pattern. It is a confidentiality risk. Additionally, there are so many public AI services that blacklisting is an unwinnable battle.

Invisible AI: Users who use AI without knowing it

While the first category is not an easy one to solve, this one will be even harder, and I think it is also an underestimated issue. One tool starts summarizing calls. Another tool drafts replies to emails, scores sales leads, creates excel files, etc. The person initiating the activity or clicking the button often might not have the slightest idea a model is involved, and the organization may not know either.

For some products there is no setting to turn the AI off. It is bundled into the product, enabled by default, or can only be disabled at a higher tier.

Below are some actions you could potentially take to tackle this risk:

• Establish inventories: Record every system in the inventory, classify the use, and document a risk decision with a named owner. Make sure you establish a process for discovering new ones. The easiest AI to inventory is the AI everyone already knows about. When asking organizations for an AI inventory, the first version is usually a list of items such as ChatGPT, Copilot, Gemini, Claude, etc. After a few iterations, it becomes a larger list, including tools such as call analyzing tools, recruitment tools, etc.
• Ask vendors three questions before renewal: Which features use AI? Where can each be switched off (tenant, group or user level)? And What are the data protections that can be enabled?
• Specify contractual language: Opt-out rights, change notice for new AI features, and data use terms belong in the agreement.
• Add compensating controls: Restrict what data can be used in the tool, limit who has access, and monitor.
• Make AI visible: The AI Act's transparency and AI literacy duties point in the same direction — people should know when AI is involved.
• Conduct AI Assessments: Like Data Privacy Assessments in GDPR, you can do AI Assessments on projects and technology to understand if AI is being used or not. Obviously this will need to be repeated on a regular basis and when major changes occur. For certain high-risk systems, a Fundamental Rights Impact Assessment (Article 27) will also need to be performed.

Shadow AI is a visibility challenge. Without visibility, demonstrating governance, accountability, transparency, or risk management becomes significantly harder.

Four (Potentially Dangerous) Assumptions

1. "We don't build AI, so this isn't ours." Most companies deploy. AI literacy obligations (Article 4 of the EU AI Act) have applied since 2 February 2025, and transparency obligations (Article 50 of the EU AI Act) generally began to apply in August 2026. A company can be in scope without having written a single line of code.
2. "Our vendor is compliant." A company’s ability to comply may depend on documentation and work done by third or fourth parties that it does not control. Contracts need to ask for documentation explicitly.
3. "The EU AI Act deadline moved, so we can wait." The postponement to 2 December 2027 for high-risk AI systems is not a change of direction. It does not freeze or stop the duties that already apply.
4. “We only customised it a bit.” A small change can have a big consequence. A company that puts its own name on a high-risk AI system, makes a substantial modification to it, or uses a system for a purpose that makes it high-risk can be treated as the provider of that system (Article 25). Providers carry the heaviest obligations, such as conformity assessment, technical documentation, and post-market monitoring.

All these statements above are treating regulation as something that happens to the legal department, and are focused on compliance. There are enough real-world cases demonstrating that being compliant doesn’t mean you are secure. Security doesn't make you compliant either, because the AI Act's product safety obligations (roles, documentation, and conformity) are something no security program produces on its own.

Fines and the Lasting Effect

Fines are there to help enforcement. Learning from GDPR, we notice that fines tell you little about what happened inside the organizations that were never fined.

The penalty ceilings are worth knowing, but they should not be the main driver: up to €35 million or 7% of global turnover for prohibited practices, and €15 million or 3% for most other infringements.

The lasting effect of EU regulation is a change to the organizational processes, not the fines.

Timeline

An inventory, a classification method, named owners, and contract language. None of these can be created and set up in a month, and all of them are needed before a regulator or a customer asks. The challenge is not meeting the final deadline, but rather establishing visibility, ownership, and governance before those deadlines arrive.

What Can We Do Next ?

A security leader does not need to own the AI Act to take actions. Below is a list of actions and tasks that can be initiated:

• Build an AI Inventory:
• Combine procurement records, logs, and a simple survey. Include embedded features and tools people use on their own, not only named AI products. Accept that it will be incomplete and will require continuous maintenance and updates.
• Find the AI you cannot switch off:
• For each tool, note which features use AI and whether they can be disabled at tenant, group, or user level. Log the ones that cannot be, with an owner and a documented risk log.
• Add an AI flag to change management:
• A new AI capability in an existing tool should start a review, test, and monitoring before it is applied.
• Update vendor contracts and due diligence:
• Ask for provider documentation, incident notification, opt-out options, data-use terms, and change notice for AI features.
• Reuse what exists; do not reinvent the wheel:
• GDPR records, NIS 2 governance, and Cyber Resilience Act processes already provide structure and expertise. Extend them before creating a new function.
• Consider usage of AI:
• Your users will want to use AI. Make them aware of which AI they can use for which kind of data.

Are We There Yet?

I truly don’t think we are there yet. We are learning as we go, and as the adoption of AI matures, so will we. However, there are some items we cannot ignore, which we should be doing right now. AI adoption is linked to cybersecurity because it introduces new attack surfaces, dependencies, and data exposure risks.

Knowing which AI systems you operate, why they are used, who owns them, and who is accountable for them are governance questions. And this is the foundation of AI security: You cannot defend, monitor, or respond to things that you do not know exist.

Questions you might ask yourself:

• Could I list every AI system in my organization? Including the ones inside software we already license?
• Do people within the organization know which of their everyday tools contain AI, and which of those they can safely use with what kind of data?
• For the AI you cannot disable, who accepted that risk, and is it written down?
• Who would sign the answer to a regulator or a large customer, and do they know it?