Deutsch
English
Français
Español
- Share:
Endurance test: 32 security solutions for 6 months in the ATP test under Windows
The series of Advanced Threat Protection tests provides an impactful demonstration of how effectively security solutions – for both consumer users and corporate users – can withstand real attacks by ransomware or info stealers. The individual results of these ATP tests are already highly revealing. Yet, even greater insights can be gained when studying the results over a longer time period: the endurance test clearly shows which security solution stands out through consistently strong threat prevention. The endurance test conducted over six months under Windows 10, still the most widely used operating system, had a few surprises in store – and confirmed some things that experts already suspected.
In the advanced test, many products showcased their protection capability over an extended period
When it comes to purchasing a new security solution, those responsible for corporate procurement pay just as much attention to reliable test data as do consumer users. That is why the latest ATP endurance test of 32 products under Windows is a trove of incontrovertible test data. It provides a sound assessment of the many security solutions available on the market. The AV-TEST lab collected test findings from January through the end of June 2025. A large number of products took part in all 3 tests, others only in 2 or one. Nevertheless, it enabled a good side-by-side comparison of the products.
The ATP test clearly shows in up to 30 scenarios how well a protection package or solution for corporate users defends against a real attack by ransomware or a data stealer. The advanced test begins with the detection of malware. But that is only the first step. Because even after non-detection, the protection solutions have additional modules with which they can stop and liquidate an attacker in subsequent steps. The lab documents all these processes and lists them in its findings.
32 products confronted with real attack scenarios
As the ATP test checks how the products behave in real attacks, the testers use so-called attack scenarios in the laboratory. Because in a real attack, the malware does not simply land in the detection module of a protection software. Instead, the attackers lurk in a compressed e-mail attachment and then use various attack techniques to evade Windows protection. The malware often disguises itself as a legitimate Windows process or tries to exploit other loopholes.
In the lab tests, the following techniques were employed, either individually or also in combination.
UAC bypass: User Account Control (UAC) is a Windows security function designed to protect the operating system from unauthorized changes. Attackers can bypass the UAC mechanism to elevate their privileges and carry out tasks. In the tests, the lab experts implement a UAC bypass by exploiting the IFileOperation COM interface. In this way, it is possible to copy or move files having elevated privileges, without displaying a UAC prompt. The method works by launching a process that already has elevated privileges (a process with an auto-elevate function), such as mmc.exe, the Windows executable for the Microsoft Management Console. As a result, the malicious DLL was loaded into an elevated process. This allows the malware to either execute immediately or install its own Windows service with elevated privileges – without the user noticing.
MSBuild (Microsoft Build Engine): The Windows tool is part of the .NET Framework and Visual Studio. It is used to create (compile) executable applications from project files (.proj, .csproj etc.). Attackers, however, inject malicious code into a project file and offer it for compilation by the user. During this process, the malicious code is executed as fileless malware in the memory and can thus bypass protective measures, as MSBuild is considered a trusted application.
Code concealed in a LNK file: In a seemingly harmless shortcut file (.LNK), malware code is hidden, which is extracted via the PowerShell bundled within Windows. The code then loads ransomware and data stealers onto the Windows system and executes them.
A total of 13 protection packages for consumer users had to prove themselves in the 6-month endurance test
Over the scope of 3 tests, 3 company solutions achieved the maximum score of 105 points
32 security products in the advanced ATP test
All security products have participated in 1, 2 or 3 extended ATP tests. Each individual test is carried out within 2 months. Some of the products thus took part in the tests for a full 6 months. The table quickly indicates each product and the number of tests in which it was evaluated. The maximum protection score naturally changes as a result. Up to 35 points can be achieved in each test – i.e. a maximum of 105 points for 3 tests, otherwise 70 or 35 points.
The results are clearly documented by the testers in a matrix according to the MITRE ATT&CK standard. This is used internationally in professional assessments. It ensures that the tests are precisely reproducible in all steps.
The maximum possible score varies based on the number of tests completed. There are 10 scenarios in each test. In a series consisting of 5 scenarios, a ransomware or an info stealer launches an attack. For each ransomware sample blocked, up to 3 points are awarded, and for each info stealer, up to 4 points. If a product is successful in all scenarios, it receives 15 plus 20 points and thus racks up 35 points in its protection score. Minor errors or only partial defense can also lead to half points.
13 products for consumer users in the ATP endurance test
The table displays the results of the ATP endurance test for the 13 products geared to consumer users. The products with 3 tests in the evaluation are in the upper echelon. They are followed by the products with 2 tests and one test.
The first group with 3 tests consisted of the products from Avast, AVG, Bitdefender, Kaspersky, McAfee and Norton. McAfee Total Protection headed up the list of products with 3 tests with 105 out of 105 possible points. Following very closely behind were Kaspersky Premium with 103.5 points and Bitdefender Total Security with 102 points. They all detected the entire set of attackers in the 30 scenarios, but Kaspersky and Bitdefender committed minor errors and lost a few points.
Norton, Avast and AVG also delivered good results. They experienced detection problems in one or two scenarios and lost those points. There were also minor issues in subsequent steps that led to points being deducted. In the end, Norton scored 98.5 points, Avast and AVG each scored 98 points out of a maximum possible 105 points.
The second group with 2 tests consisted of the vendors Avira, ESET and F-Secure. None of the products detected all the attackers in the 20 scenarios. The packages already lost vital points at this juncture. F-Secure achieved 63 out of a possible 70 points – ESET still scored 62 points. Avira did not detect the attackers in 3 scenarios and thus only scored 59 out of a possible 70 points.
AhnLab, G DATA, K7 Computing and Microsoft made up the third group involving one test. AhnLab, K7 Computing and Microsoft recognized the attackers in all 10 scenarios and scored the full 35 points. Only the product from G Data triggered in merely 8 out of 10 scenarios and therefore only collected 28 of the 35 points for the protection score.
19 company products in the ATP endurance test
The ATP endurance testing of the products for corporate users involved 19 solutions, the majority in 2 or 3 tests. The maximum protection scores here were also 105, 70 or 35 points, depending on the number of tests.
The first group with 3 tests consisted of the 7 solutions from Avast, Bitdefender, Kaspersky (with 2 versions), Microworld, Trellix and WithSecure. Three solutions for corporate users earned the maximum 105 points: Kaspersky Endpoint Security, Kaspersky Small Office Security and Microworld eScan Enterprise EDR. The products from WithSecure and Bitdefender came in close behind with 30 detected attacks each, scoring 103 and 102 points respectively.
The Trellix solution detected 29 out of 30 attacks, but lost important points due to further minor errors and thus scored 98.5 out of 105 points. Avast only detected 28 attacks, but still collected 98 points.
The second group with 2 tests consisted of 3 products for corporate users. The solutions from Acronis and Qualys recognized the attackers in all 20 scenarios and committed no further errors. This performance was rewarded with the maximum 70 points on the protection score. ESET only detected 18 out of 20 attackers and lost many points as a result. This left the solution with only 59 out of 70 points in terms of the protection score.
The third group involving one test was the largest with 9 solutions for corporate users. This group included products from AhnLab, Bitdefender, Check Point, Crowdstrike, Huawei, Microsoft, Rapid7, Sophos and Symantec.
With 10 detected attacks and no other errors, 6 products earned the full 35 points for their protection score: AhnLab, Bitdefender, Crowdstrike, Microsoft, Sophos and Symantec. Huawei's solution completely missed an attacker, reaching a final tally of 31 out of 35 points. Check Point and Rapid 7 also detected only 9 attackers, yet they also committed minor errors on top. They only achieved scores of 29 and 28.5 out of a total of 35 points.
ATP endurance test: more than just a result
Even the regular ATP tests, occurring every two months, provide valuable test results on security packages and solutions for real attacks. The long-term test paints an even clearer illustration of what protection solutions really achieve in real attacks.
The group involving 3 tests was the most interesting. It showed that the products for consumer users from McAfee, Kaspersky and Bitdefender also do a really good job in an endurance test. McAfee deserves special mention with its 105 out of 105 possible points.
The group of solutions for corporate users with 3 tests reflected a similar picture. Three products achieved the maximum of 105 points: both solutions from Kaspersky and the product from Microworld. However, WithSecure and Bitdefender also made a strong showing in the tests with protection scores of 103 and 102 points respectively.
The test series with 2 tests was also interesting, but somewhat less conclusive. In the case of products for consumer users, no product was able to defend against all 20 attack scenarios. At least 2 of the 3 company solutions, however, were able to do so. The corporate packages from Acronis and Qualys fended off the attackers in all 20 scenarios and received the 70 points achievable for their protection score.
The ATP evaluations forming the basis of the endurance test include the following:
