What is machine learning in cybersecurity?

Machine learning is a branch of artificial intelligence in which a system derives its behaviour from data rather than from rules a person wrote. A model is shown many examples, works out the distinguishing pattern itself, and can then make judgments about data it has never seen. In cybersecurity, that capability is embedded in most modern security tooling: it classifies files as malicious or benign, filters phishing from legitimate mail, baselines normal behaviour for a user or a device, groups related events into single incidents, and scores what an analyst should look at first. It's also among the most oversold ideas in the industry, and the two facts are connected. Machine learning genuinely solves some security problems well, performs partially on others, and is routinely claimed for things it cannot do. A security leader assessing a product, or an analyst deciding how much weight to give a model's output at two in the morning, needs to know which is which. This page sets out how machine learning works, where it demonstrably earns its place in security, where the claims outrun the capability, and the limits that matter once a model is in production.

Key takeaways

Machine learning derives behaviour from data, not from written rules. A model is trained on examples and infers the pattern itself, then applies it to new data.
It's strongest where labeled examples are abundant. Malware classification and phishing filtering work well because millions of confirmed good and bad examples exist.
It's weakest where intent matters. A model recognizes patterns in data; it does not reason about whether an action was hostile, which is the question security actually asks.
Anomalous is not the same as malicious. Most unusual activity in an organization is legitimate, and this single distinction explains most disappointment with model-driven detection.
A model's output is a probability, not a verdict. How it's presented to an analyst, and whether it can be explained, determines whether the detection is actionable.

How machine learning works

A model is trained on a dataset, adjusting internal parameters until its outputs match the answers it's supposed to give, or in the absence of given answers, until it finds structure in the data. What it actually reasons about are features: the specific measurable properties extracted from raw data, such as a file's size, the frequency of a login, or the entropy of a string. Feature selection matters enormously, because a model can only consider what it has been shown.

Once trained, the model performs inference. Given new data, it produces an output, typically a classification or a score. That output is a probability rather than a fact, and where the decision threshold sits determines how many false positives and false negatives the system generates. Deep learning is a subset of this, using multi-layered neural networks that handle complex data such as raw binaries well, at the cost of needing more of it and producing decisions that are considerably harder to explain.

The types of machine learning used in security

Approaches are grouped by what the training data looks like and how the model receives feedback. The distinction determines which security problems each can address.

Types of machine learning in cybersecurity
Type	How it learns	Security application
Supervised learning	From labeled examples, where each item carries the correct answer.	Malware classification and phishing detection, where large sets of confirmed good and bad examples exist.
Unsupervised learning	From unlabeled data, finding structure without being told what to look for.	Anomaly detection and clustering related alerts, where no labels are available for what you're seeking.
Semi-supervised learning	From a small labeled set alongside a large unlabeled one.	Common in practice because confirmed labels are expensive to produce in security while raw telemetry is abundant.
Reinforcement learning	By trial and feedback, adjusting according to whether outcomes were rewarded.	Response tuning and automation, although it is less established in production security than the other approaches.

The labeling problem shapes what's practical. Supervised learning needs confirmed examples, and in security a confirmed example usually means an incident somebody investigated and documented, which is slow and expensive. That's why unsupervised anomaly detection is so widely deployed despite its difficulties: it doesn't need labels.

What machine learning is actually good at in security

Vendor material on this subject tends toward lists of applications presented as uniformly successful. They aren't. The table below sets out where machine learning demonstrably outperforms the alternative, where it helps but needs support, and where the claim is stronger than the capability.

What machine learning is actually good at in security
Application	How well machine learning actually performs
Malware classification	Strong. Millions of labeled samples exist, the problem is well defined, and models generalize to variants that signatures miss. This is the clearest success case.
Phishing and spam filtering	Strong. The data is mature and high-volume, and the cost of an occasional false positive is tolerable. This is among the most effective applications in security.
Alert clustering and deduplication	Strong. Grouping related events into single incidents reduces volume without requiring the model to judge whether anything is malicious.
Prioritization and scoring	Useful. Ordering a known set of alerts by likely importance helps analysts spend their time more effectively rather than finding something new.
Behavior analytics on identity	Partial. It is valuable when combined with context about how attacks proceed, but produces persistent noise when used to generate alerts on its own.
Detecting novel attack techniques	Weak and frequently overclaimed. Models detect deviations from what they learned, not hostile intent. A technique with no precedent in the training data is not reliably recognized.
Autonomous response without review	Weak. A probabilistic output driving consequential action without human judgment transfers risk rather than reducing it, particularly given the false positive rates involved.

The pattern is consistent and worth internalizing. Machine learning performs well where the question is a classification with abundant examples, and poorly where the question requires reasoning about intent. Security asks both kinds of question, which is why models belong in a detection stack rather than constituting one.

Why anomalous does not mean malicious

The mechanism behind the partial and weak entries above is worth stating explicitly, because it accounts for most of the gap between what model-driven products promise and what security teams experience.

An anomaly detection model learns what normal looks like and flags departures from it. The logic seems sound, since attacks are unusual. The problem is the reverse direction. Attacks are unusual, but the overwhelming majority of unusual activity is not an attack. A developer working an unfamiliar shift, a finance process that runs four times a year, an application deployed last week, an employee travelling: all genuinely anomalous, all entirely legitimate. A model that flags anomalies faithfully will therefore surface far more benign oddity than malicious activity, and the ratio worsens in exactly the dynamic environments that most need good detection.

Two conclusions follow. Anomaly detection produces value when it feeds something that adds context, not when it generates alerts directly, because the question that matters is not whether behaviour is unusual but whether it's unusual in a way consistent with how attacks actually proceed. And when evaluating a product, ask what it does with an anomaly once found rather than how well it finds them, which is the easier problem and the one every vendor has solved.

The limits worth knowing before trusting a model

Four issues come up repeatedly once models are running in production:

A baseline can learn a compromise as normal. If a model builds its picture of normal from an environment that already contains an intrusion, that intrusion becomes part of the baseline and will not be flagged. Activity predating the training is effectively invisible, which matters when dwell times run to months.
Explainability determines whether an alert is actionable. An analyst needs to know why a detection fired in order to decide what to do. Models producing a score without a reason leave them reconstructing the logic, and unexplainable detections get ignored over time, which is the worst available outcome.
Models degrade as environments change. Accuracy drifts as infrastructure, applications, and working patterns evolve, so model-driven detection needs ongoing maintenance rather than one-time deployment.
Models can be manipulated. Adversarial machine learning covers crafting inputs a model misclassifies, and poisoning training data so it learns the wrong thing. An attacker who understands what a model examines can often work out how to look ordinary to it.

None of this argues against machine learning in security. It argues for knowing what a model was trained on, what it can explain, when it was last retrained, and what happens downstream of its output. Those four questions separate a capability from a source of noise.

Why detection needs more than machine learning

Security products increasingly describe themselves as AI-driven, with the implication that a good enough model removes the need for anything else. The analysis above suggests why that doesn't hold. Models are strong at recognizing patterns they have seen many examples of, and correspondingly weak at what security detection most needs: reasoning about intent, recognizing a technique with no precedent, and explaining themselves to an analyst who has fifteen minutes to decide whether an alert matters.

Sekoia is a European cybersecurity vendor, and it's worth being straightforward about the design choice that follows, because it differs from how this category is usually marketed. Detection on the Sekoia AI SOC platform rests primarily on explicit behavioural rules mapped to the MITRE ATT&CK framework and on cyber threat intelligence produced by an in-house Threat Detection & Research (TDR) team, with machine learning applied where it genuinely outperforms a rule rather than as the foundation. The reason is the explainability point above. A rule mapped to a documented technique tells an analyst what was detected and why, which is what makes an alert something they can act on. Intelligence about how a particular threat actor operates tells them what to expect next. A probability score tells them neither, however accurate it is.

For a security team assessing claims in this area, the useful questions are specific and unglamorous. What was this model trained on, and does that resemble our environment? When an alert fires, will the analyst be told why? How often is it retrained, and by whom? And what happens to an anomaly after it's found? A vendor answering those clearly is describing a capability. One answering by describing how advanced its algorithms are is describing a marketing position.