Photo credit: www.darkreading.com

MITRE ATT&CK Evaluations Highlight Cybersecurity Challenges and Progress

In 2025, a global fintech company is anticipated to encounter sophisticated cyber assaults targeting its hybrid cloud infrastructure. The attackers will focus on the company’s Active Directory, as well as employees’ LinkedIn profiles and shared code repositories, leveraging these vectors to execute their malicious intentions.

Rather than a straightforward prediction, this scenario is designed as part of the MITRE ATT&CK Evaluations. This annual assessment pits various cybersecurity vendors against evolving techniques employed by advanced threat actors. MITRE, a government contractor, conducts these evaluations to help participating firms identify areas for improvement in their detection, protection, and response capabilities amid real-world attack simulations. For cybersecurity professionals, the evaluation results serve as a crucial indicator of their readiness to counter sophisticated cyber threats.

While many vendors may flaunt their ratings from these evaluations, the essential goal extends beyond simply achieving better scores. According to Lex Crumpton, a principal cybersecurity engineer at MITRE, the emphasis is on enhancing security measures for companies and the products offered by vendors. “The ATT&CK Evaluations are more of a collaborative effort focused on adversary emulation and purple teaming,” Crumpton notes. “We establish a testing environment and the vendors are unaware of the specific techniques that will be chosen based on that scope document.”

The MITRE ATT&CK Framework is a well-established taxonomy of tactics and techniques used by cybercriminals. Each year, it facilitates testing of security tools against the backdrop of the latest cyber threats. For instance, in 2024, the evaluations focused on methods associated with notorious groups such as LockBit and Cl0p ransomware gangs, as well as threats linked to North Korean state actors, who have increasingly turned to ransomware as a funding mechanism for their national objectives.

The 2024 evaluations simulated various ransomware attacks affecting both Windows and MacOS platforms, as detailed in a statement from December 2024.

For 2025, one segment of the evaluations, known as the Managed Services Evaluation, will specifically examine “cloud-based attacks, response/containment strategies, and post-incident analysis,” based on MITRE’s scenario outline.

Companies can leverage the ATT&CK Evaluations in a couple of significant ways, as highlighted by Greg Young, Trend Micro’s vice president of cybersecurity. “For a company’s purchasing decisions, this is an important data point — but it should not be the only factor considered because the range of MITRE’s testing is relatively narrow,” he states. “Moreover, these evaluations can inform the operations of the company’s security operations center and enhance their red teaming efforts — prompting them to analyze what adversaries are currently employing.”

Creating More Realistic Cyber Threat Scenarios

The ATT&CK evaluations draw upon cyber threat intelligence from analysts around the world. MITRE combines insights gathered from its own in-house threat intelligence team with information sourced from the broader Cyber Threat Intelligence (CTI) community. This comprehensive data collection informs the selection of adversaries used in the evaluations. A specialized red team creates tools that mimic contemporary techniques used by these selected adversaries, while a blue team evaluates the effectiveness of these techniques in response to the attacks.

MITRE undertakes distinct rounds of testing. The managed service round involves a black-box testing environment, where the evaluated vendor is given minimal information aside from the general threat type. Conversely, in the enterprise round, vendors receive detailed technical scopes and insights about potential adversaries, including whether they originate from nation-states or employ varying tactics.

Like many organizations, MITRE has received some criticism regarding its evaluation scenarios. Crumpton elaborates, “One of the most significant pieces of feedback this year was due to the introduction of false-positive noise, which included benign user activities. Some vendors contested that certain benign actions could be misclassified as malicious.”

Driving Security Improvements Through Evaluations

During the evaluations, vendors are rated on their performance. However, the primary goal is to provide insights that help both vendors and businesses strengthen their security postures. Crumpton explains, “Ultimately, our mission is to enhance the tools available. If we find a technique that your tool fails to detect while emulating an adversary, our aim is to assist you in improving your detection capabilities.”

Defenders can also learn from the ATT&CK evaluations by developing playbooks aimed at identifying and shielding against the threats examined in the tests. Young from Trend Micro points out that during the evaluation, MITRE logs activity and takes screenshots to provide organizations with a detailed overview of attacks as they progress, mapping these maneuvers against the ATT&CK Framework.

“Understanding that adversaries are employing specific techniques, such as lateral movement or targeting particular resources, is invaluable for companies as they design their defenses,” Young remarks. “I believe there is significant merit in examining the ATT&CK framework itself, perhaps even more than the evaluations, though the emphasis may vary based on individual objectives.”

Source
www.darkreading.com