The MITRE Engenuity ATT&CK Evaluations 2024 outcomes are out and, with them, one other 12 months of distributors claiming victory. As a reminder, these evaluations don’t have any winners or losers — simply candy, candy information.
Living proof, MITRE ATT&CK tracks and checks on strategies that might be utterly benign, even one thing so simple as T1059.004, which launches a Unix shell. Relying on the person, this might be a completely regular exercise — however it is also an attacker. Equally, T1059.002, utilizing AppleScript, might be completely respectable and was truly used within the check to generate benign noise.
If a vendor says that it achieved 100% on the evaluations, it’s doubtless doing a number of of the next:
Manipulating the outcomes by solely exhibiting elements of outcomes that they really feel profit them
Turning on settings within the product which are unrealistic for a real-world setting in order to seem more practical
Treating the outcomes as a contest as a substitute of a studying alternative and an opportunity to enhance the product
As long as you take a look at these evaluations as informative information, not offering winners or losers, you will get actual worth out of the outcomes. With all that silliness apart, let’s get into what it’s good to know.
The analysis broke new floor with macOS.
The evaluations targeted on two adversary situations: ransomware concentrating on Home windows and Linux (CL0P, LockBit) and DPRK concentrating on macOS.
Vary working techniques
Home windows
Home windows Server 2022
Home windows 11
Linux
Ubuntu 22.04.x LTS
macOS
OS: macOS Sonoma 14.x
Arch: Apple Silicon
The give attention to macOS is a brand new addition to the evaluations. It’s thrilling to see one of these analysis cowl macOS, because the capabilities that instruments have on this OS are usually extra of a black field than the extra well-tested capabilities on Home windows and Linux.
The evaluations happen over a number of days per vendor. They kick off with detection rounds, then permit a day for configuration modifications and retests (which may embrace deploying further detection guidelines, gathering further telemetry, making modifications to the UI, and so on.). The safety spherical is executed final. All emulations had been performed post-compromise to look at the detection and safety capabilities as soon as an adversary gained entry.
Background noise and alert quantity make the detection outcomes particularly helpful.
One attention-grabbing hurdle MITRE launched this time is background noise and false positives. On this spherical, MITRE generated further alerts to function background noise and tracked false positives. This checks the product’s potential to solely discover actually malicious habits and never alert on benign exercise. It additionally makes it harder for distributors to crank up the detection capabilities to alert on every part, which has skewed vendor outcomes up to now.
MITRE additionally launched a “quantity” metric. This was a much-needed addition, as up to now, some distributors issued hundreds of alerts in a single situation, which, in observe, results in a lower-quality analyst expertise. Now, the outcomes present precisely what number of alerts had been triggered for every situation and the severity of these alerts.
Safety micro-emulations give extra granular outcomes.
There was a separate emulation plan for protections (although nonetheless targeted on ransomware) than detections this 12 months, which helped maintain the check practical. As well as, MITRE examined protections through micro-emulation plans, which MITRE defines as compound behaviors involving a brief collection of associated ATT&CK strategies which are incessantly used collectively in real-world assaults.
As an alternative of working everything of the emulation finish to finish, MITRE bundled a choose few strategies collectively. For instance, Take a look at 1 checked out enumeration and exfiltration through batch script and rclone (a mixture of added noise [T1059.003, T1105, T1021.001] and precise exercise [T1560.002 and T1048.003]). This isn’t the total scope of the assault, however it’s a collection of steps which are widespread in attacker exercise.
Utilizing micro-emulation plans is essential when testing preventive controls — as a substitute of getting an assault blocked from the very begin. This allows you to see precisely how efficient the device is at blocking every portion of an assault. It’s essential, nonetheless, to keep in mind that anticipating a device to dam each micro-emulation plan is unrealistic, as sure actions shouldn’t be blocked in isolation. For instance, archiving collected information after which exfiltrating it, as talked about above, will not be essentially malicious. Some prevention strategies depend on understanding person habits or indicators of compromise. Additional, because of the constraints of the check, the testing doesn’t contemplate locking down person account permissions primarily based on use case or a few of the tuning that occurs over time with analytics about typical person exercise.
It’s nonetheless troublesome to know what to do with the outcomes.
The MITRE crew has put a variety of work into making the outcomes consumable through a really easy-to-use outcomes web page that allows you to examine and distinction completely different distributors, see screenshots of their capabilities, and clearly see alert quantity. We extremely suggest trying by means of this web page. With that stated, we will probably be releasing a extra in-depth report within the coming months that gives extra full particulars on the analysis outcomes and tips on how to use them.
Keep tuned and should you’re a Forrester consumer e book an inquiry or steering session with me if in case you have extra questions.
