Artificial intelligence is fundamentally changing how organizations detect cyber threats and accelerate investigations. It helps security teams process massive volumes of telemetry. It also introduces vital questions about accountability and operational control.
Recent security incidents highlight why AI governance has become an urgent operational requirement. Security researchers recently exposed malicious models and authentication token leaks on public repositories like Hugging Face, proving that open AI hubs introduce new attack vectors. At the same time, threat actors actively attempt to manipulate training datasets or telemetry feeds to blind detection models and trick automated response systems. Organizations also risk accidental data leakage, exposing sensitive incident telemetry or proprietary intelligence when integrating commercial LLMs without strict boundary controls. Finally, attackers craft adversarial inputs specifically designed to bypass security controls and hijack autonomous agents to disrupt automated SOC workflows.
When an AI system contributes to a security decision, organizations need to understand more than its final recommendation. They need to know what information it used and the specific actions it performed. They must also know how its conclusion can be audited and what happens if the system is manipulated or becomes unavailable.
These questions are taking center stage as Europe and international regulators develop policy frameworks for trustworthy AI and cybersecurity.
Europe is building an interconnected policy framework
The European Union has not created a single piece of legislation governing every aspect of digital security. Instead, several complementary instruments address different parts of the security lifecycle:
- NIS2 focuses on cybersecurity risk management, incident handling, and organizational governance.
- DORA addresses digital operational resilience specifically within the financial sector.
- The Cyber Resilience Act (CRA) regulates the security of products with digital elements throughout their lifecycle.
- The AI Act introduces a risk-based framework for specific uses of artificial intelligence.
- The Cloud and AI Development Act (CADA), proposed in June 2026 under the European Technological Sovereignty Package, establishes a cloud sovereignty framework setting strict rules on data residency, supply chain controls, and AI training data restrictions for sensitive environments.
The European Commission presents these instruments as interlocking parts of a wider cybersecurity architecture.
This regulatory foundation is complemented by policy initiatives targeting the intersection of AI and security operations.
In July 2026, the European Commission published its Action Plan on Cybersecurity and Artificial Intelligence. The plan focuses on the safe use of advanced AI, the reinforcement of Europe’s cyber defense capabilities, and the development of local AI testing environments. It also refers to cooperation with ENISA on secure access to advanced AI systems for organizations in critical sectors.
The plan connects existing legislative pillars rather than replacing them. This creates a clear mandate for defenders: how should organizations deploy AI to strengthen capabilities while preserving strict control in sensitive environments?
The AI Act is risk based, not a blanket rule for every cyber tool
The AI Act uses a risk-based approach. The classification of an AI system depends entirely on its intended purpose and the context in which it is used. The European Commission distinguishes between limited categories of high-risk systems and the much broader range of AI systems that do not fall into that category.
This distinction matters for cybersecurity.
An AI system used in a Security Operations Center (SOC) is not automatically a high-risk system simply because it analyzes security data or supports an analyst. The relevant question is how the system is intended to be used, what decisions it influences, and whether it falls within one of the categories identified by the regulation.
The AI Act’s provisions on critical infrastructure focus on AI systems used as safety components in the management and operation of those infrastructures. That does not mean every cybersecurity platform or AI SOC is automatically covered by this category.
Its requirements for high-risk systems still provide a useful reference point for evaluating trustworthy AI in sensitive environments. They highlight foundational principles that remain relevant beyond the strict legal classification of a particular tool.
Trust starts with traceability
The AI Act includes record-keeping requirements for high-risk systems. These systems must automatically record relevant events throughout their lifecycle to support traceability and post-market monitoring.
For security operations, traceability has a concrete operational meaning.
A security analyst should never receive an unexplained verdict from an AI system. The investigation must preserve an auditable record of:
- The data the system examined
- The queries or actions it performed
- The observations that supported the conclusion
- The areas where uncertainty remained
- The presentation of the final result to the analyst
This is different from asking an AI system to disclose every internal mathematical step of its model. The objective is practical: preserving an auditable record of the investigation. A conclusion is far more useful when it can be verified, especially in regulated environments where an organization must explain how it detected and responded to an incident.
Human oversight must be an operational capability
The AI Act addresses human oversight. For high-risk systems, the people responsible for oversight must be able to understand the system’s capabilities, monitor its operation, and interpret its outputs. They should also be able to disregard, override, or interrupt the system in appropriate circumstances.
In cybersecurity, this raises the critical distinction between nominal and effective human oversight.
"Saying that an analyst remains 'in the loop' is not enough if that analyst cannot access the evidence behind a recommendation or has no practical way to correct the result."
Effective oversight requires specific conditions. Analysts need:
- Visibility into the evidence used by the system
- Sufficient context to assess the recommendation
- Clear boundaries around the actions an agent can perform
- The ability to correct or reject a proposed conclusion
- A safe way to stop or suspend an automated process
The purpose is not to prevent automation. It is to ensure that automation remains governable. For a concrete example of how to govern automation safely, review our engineering team's guide on what you need to know before putting AI agents in a production SOC.
AI systems require robust protection
The security of an AI system cannot be separated from the security of the environment in which it operates.
The AI Act identifies robustness and cybersecurity as lifelong requirements for high-risk systems. It points to risks such as data poisoning, adversarial inputs, confidentiality attacks, and model vulnerabilities.
These risks directly affect AI used in security operations.
In a SOC environment, an AI agent relies on telemetry, threat intelligence, asset information, identity data, and detection rules. If one of these sources is manipulated or becomes unavailable, the quality of the investigation drops.
Organizations evaluating AI for cybersecurity must look beyond the quality of the system’s answers. They must ask how the platform protects:
- The data sources used during an investigation
- The models and components on which it relies
- The interfaces through which it acts
- The logs needed to review its behavior
An AI system that accelerates investigations but cannot resist manipulation introduces a distinct operational risk.
From data sovereignty to tech autonomy
The debate around digital sovereignty often focuses on data location and reliance on external providers. These remain important questions. Sekoia’s positioning on strategic autonomy highlights data control, technological independence, and local resilience.
The growing use of AI introduces another dimension: control over the decisions that systems help produce.
"An organization may know where its security data is stored, but still have limited visibility into the model interpreting it or the agent recommending an action. This demands a broader analytical concept: tech autonomy."
This reflects the direction set by CADA, which formalizes EU cloud sovereignty assurance levels and restricts AI training data from leaving European jurisdiction for sensitive activities.
Tech autonomy can be evaluated through four pillars:
- Infrastructure: Where does the system operate?
- Data: Which information can it access, where is it stored, and how is it protected?
- Models: Which models determine system behavior, and who trains them?
- Agents: Which actions can the system take, and under whose control?
This concept extends the sovereignty debate beyond hosting. The question is no longer only where data resides. It is also who controls the intelligence interpreting it and the evidence left behind.
The global push for trustworthy AI
While Europe leads with binding horizontal legislation, AI governance is accelerating globally across key international jurisdictions:
United States: The Executive Order on Safe, Secure, and Trustworthy AI and the NIST AI Risk Management Framework (AI RMF) establish rigorous testing, red-teaming, and cybersecurity standards for dual-use foundation models.
United Kingdom: The UK advocates a sector-led, pro-innovation framework backed by the AI Safety Institute, focusing on severe risk evaluation and technical safeguards.
Global standards (ISO/IEC): International frameworks like ISO/IEC 42001 establish standardized AI management systems that align risk handling across multinational organizations.
For international companies operating across multiple regions, these global initiatives reinforce a unified requirement: security software vendors must provide transparent, auditable, and resilient AI capabilities regardless of where security operations are physically conducted.
Questions for organizations adopting AI in the SOC
As AI becomes standard in security operations, organizations should ask vendors and internal teams a consistent set of questions to bridge the gap between compliance and capability:
- Can the system show the evidence behind an AI-generated conclusion?
- Are investigations and agent actions recorded in an auditable format?
- Can analysts correct, reject, or replace a proposed verdict?
- Which actions require human validation?
- How does the system respond to manipulated or unreliable inputs?
- Are customer environments and data logically separated?
- Is customer data protected from training general-purpose third-party models?
- Where does AI processing take place?
- What happens if a model, provider, or service becomes unavailable?
These questions provide a practical way to evaluate whether an AI system is suitable for a sensitive operational environment. To see how these principles apply in practice, read our technical breakdown on how Sekoia handles customer data and ensures AI transparency.
A framework still taking shape
The European framework for AI and cybersecurity continues to develop.
The Commission’s implementation guidance for the AI Act reflects an evolving timeline for different categories of high-risk systems. The new Action Plan on Cybersecurity and Artificial Intelligence adds a specific focus on secure AI capabilities, testing, and European technological capacity. Concurrently, proposals like CADA will proceed through inter-institutional negotiations, gradually establishing explicit legal criteria for cloud and AI sovereignty across EU Member States.
ENISA has also published recommendations for authorities, defenders, and service providers on developing operational capabilities needed to address machine-speed threats. This reinforces the idea that the policy debate is moving beyond whether AI should be used. It is increasingly concerned with how AI can be used safely in environments where failure carries serious consequences.
Organizations face a twofold challenge: understanding the specific legal obligations that apply to their operations, and establishing clear internal expectations around evidence, governance, and business continuity.
Conclusion
Europe’s emerging AI and cybersecurity framework does not provide a single answer for every use case in security operations. The AI Act is risk based, and system classification depends on purpose and context.
However, the regulatory momentum brings vital principles into focus: traceability, human oversight, robustness, and lifecycle cybersecurity.
These principles offer a baseline for evaluating AI in the SOC. A trustworthy system should produce useful results and make those results reviewable, while remaining resilient when its data or operating environment is challenged.
Asking your vendors the right questions about evidence, data separation, model training, and agent control is no longer optional. It is a mandatory step to protect your operations. The next stage of confidence in AI for cybersecurity depends on whether organizations remain able to govern and sustain the critical operations that AI helps them perform.
Resources and further reading
- European Commission: Strengthening Europe's tech sovereignty (CADA initiative page)
- Strategic autonomy: where you get to choose
- Navigating the NIS2 Directive: key insights for cybersecurity compliance
- Mandating security by design: Sekoia's blueprint for the EU Cyber Resilience Act
- Navigating DORA compliance with Sekoia
