New Lines of Defense: EPPs and EDRs Put to the Test Against APT and Ransomware Attacks

One such APT attack simulation chosen in this test series begins with an email containing a spearphishing attachment (T1193); this attachment, infected with malware, is sent to a previously determined user account of the target system. By means of a prior social engineering campaign, it is ensured that the target person of interest will launch the attached file and activate the malware on the target system. The malicious e-mail attachment involves a standard archive format containing a file which, upon unzipping and opening, launches the actual attack code. In the attack simulation, the infected attachment, as is customary upon receipt of e-mails such as these, is saved by the user on their Windows 10 system and unzipped using standard file compression software. Afterwards, the user typically launches the file, thus executing the code contained in it (T1061 and T1204). 

The malicious code launched with the file establishes an Internet connection and begins secretly downloading an executable file from the system of the attacker. The file automatically launches immediately after download, creating a registry key in the infected target system, thus gaining persistence (T1060). In order to elude detection and to defeat network filters, connectivity is established via a standard user application layer to a C2 server controlled by the attacker (T1071). In order to obfuscate the network traffic and to prevent parsing, an unusual port is used for online communication (T1065). Upon establishing the connection, the C2 server begins exfiltration over the C2 channel (T1020 and T1041), searching through the databases of the local system (T1083) and connected networks (T1016 and T1135). Subsequently, the attackers encrypt identified data and any available system and network resources of the target system (T1490, T1486 as well as T1491).

Multilevel detection in the advanced EPP & EDR test

The evaluation of the detection and threat prevention of the tested EPP and EDR solutions occurs in a three-phase model:

1. In the "attack simulation", the testers evaluate how well the EPP solutions are able to detect and stop APT attacks deploying various ransomware samples, and how well EDR solutions detect and report these attacks.

2. In the "sanity check", the evaluated EPP solutions, put through false-positive checks, are required to prove whether they are capable of distinguishing between normal user behavior and detected attack patterns, or whether they wrongly block normal operations. This phase evaluates whether the operability of the system is impaired, whereby the testers test from both the user and admin perspective, according to the following schemas:

a) User perspective

• Use of browsers (Edge and Chrome)
• Installation of software
• Use of Office documents with and without macros
• Use of Outlook (sending and receiving e-mails, checking, saving, and opening e-mail attachments)
• Access to Windows and Samba shares, internal/external FTP, use of cloud storage such as Google Drive

b) Admin perspective

• Enabling scripting (evaluating various languages such as PowerShell or VBS)
• Enabling changes to user rights
• Enabling changes to network configurations

3. In the "noise check", it is examined which normal operations (techniques), that can be exploited by attackers, can be carried out and are logged by the EDR solutions. Above all, the test uses techniques that are also deployed within the scope of the tested attacks (e.g. unzipping an archive with the help of the GUI).

The tests record how well EDR products consider the context in classifying certain operations according to the following pattern:

• Harmless operations are simply logged if no potentially malicious context is detected.
• Harmless operations are flagged as suspicious if a potentially malicious context is detected.

This approach also helps in the evaluation of the attack detection:

• If the type of detection is identical in the attack test and the noise check (making detection more difficult from an admin perspective).
• If different reports or types of detection are used (improving detection from an admin perspective).

In order to successfully complete this test and to receive an AV-TEST certificate, EPP solutions were required to block 50% of the test attacks, and the EDR solutions had to detect and report 50 percent of the techniques used in the attacks. The evaluation of the test attacks was divided up into two segments according to the range of capabilities of the solutions evaluated.

1. EPP evaluation: In which phase was the attack detected and blocked? What kind of damage occurred up to the time the attack was blocked? The charts highlighting the results on the individual attacks indicate how far each attack progressed and at which point it was blocked by the EPP solution. In addition, the respective charts show which module of the EPP solution detected and blocked the test attack, as well as what kind of damage occurred up to that point on the test systems.

2. EDR evaluation: Which steps of the launched attack were detected and how detailed was the reporting? The charts highlighting the results of the EDR solutions document what type of detection occurred by the products in the individual phases of the test attacks:

• Technique reporting: Ideally, an EDR solution ought to detect and report on the respective technique for each attack phase according to the MITRE ATT&CK Matrix. This affords security analysts precise insights and helps them make the right decisions.  
• Alert reporting: One level below this are alert reports. They classify the operation as malicious and thus notify the user that intervention is necessary. However, the level of detail here is much lower than with the relevant technique reports.
• Telemetry reporting: Finally, there are telemetry reports, which are not to be read outright as warnings; instead, they only provide certain findings without any security evaluation. At the least, experienced analysts are able to identify attacks based on these reports and utilize the knowledge gained to later reconstruct the attacks.

EPP tests

• AhnLab V3 Endpoint Security
• Avast Premium Security
• Avira Antivirus Security
• Bitdefender Endpoint Security Tools
• G DATA Security Client
• McAfee Endpoint Security

EDR test

• Bitdefender Endpoint Security Tools
• McAfee MVision EDR + McAfee Endpoint Security
• VMware Carbon Black Cloud