Latest News
June 22, 2022 | Text: Markus Selinger | Antivirus for Windows
26 Security Solutions Undergo an Advanced Threat Protection Test Against Ransomware
In a test involving real attack scenarios, 26 protection solutions for consumer users and corporate users demonstrate their performance. In the series of Advanced Threat Protection tests, the lab investigates how successfully the products protect against ransomware. Each step of the malware attack is logged and evaluated, right through to an encryption. Many solutions do exactly what they promise: offer protection against ransomware. But not all solutions pass the test with flying colors.
Live attack test against ransomware
26 security solutions in the Advanced Threat Protection test
Ransomware is without a doubt the plague of the 21st century. Media reports on partially or even fully successful attacks are virtually endless, and the truth behind them is underlined by the interesting Sophos study “The State of Ransomware 2022”. One of the first core statements made in the summary of the study is that “ransom attacks are more frequent – 66% of organizations surveyed were hit with ransomware in 2021, up from 37% in 2020”.
26 products in the Advanced Threat Protection test
The Advanced Threat Protection tests provide vendors and users with substantial findings as to how securely a product can protect against ransomware in real-life scenarios. 12 products for consumer users and 14 protection solutions for business users are subjected to the current test. The manufacturers of the products for consumer users are: Avast, AVG, Bitdefender, F-Secure, G DATA, K7 Computing, Kaspersky, Microsoft, Microworld, NortonLifeLock, PC Matic and VIPRE Security.
The solutions tested for business users are products from the following vendors: Acronis, Avast, Bitdefender (two versions), Comodo, F-Secure, G DATA, Kaspersky (two versions), Microsoft, Seqrite, Symantec, Trellix and VMware.
All the products have to successfully defend against ransomware in 10 real-life scenarios under Windows 10. The test involves threats such as files containing hidden malware in archives, PowerPoint files with scripts or HTML files with malicious content. The 10 charts on the “test scenarios” list the type of attack and each step taken to fend it off. The lab even specifies the definitions in MITRE ATT&CK technique codes. Anyone interested in finding out more about the specific technical steps involved in an Advanced Threat Protection test can refer to the published article New Lines of Defense:EPPs and EDRs Put to the Test Against APT and Ransomware Attacks for more detailed information.
Defending against ransomware attacks
In the Advanced Threat Protection test, 12 protection packages for consumers prove how well they can protect again ransomware, with strong results
Corporate solutions against ransomware
10 of 14 solutions for companies pass all tests without any errors and protect the test system against ransomware in every step
Ransomware – the biggest threat
A ransomware attack is not simply a matter of black or white, successful or unsuccessful, and the Advanced Threat Protection test by AV-Test quickly shows what this means. When ransomware is detected by a protection product, this does not mean that its execution is completely prevented. In the same way, a failure to detect ransomware at the beginning does not mean that its execution may not be prevented further down the line. To make it easier to understand the techniques of an attack and how to defend against it, the Advanced Threat Protection test explains each step of an attack scenario with a malware sample. Based on the matrix of a MITRE ATT&CK Matrix chart, each step is visualized by means of a brief description and color-coding. If an attack is fended off at the beginning (during the initial access or execution), the field is highlighted in green to indicate that the attack has been successfully prevented. The sooner a green field can be seen, the better. If a field remains orange, the test item is considered undetected (no detection). A yellow field signalizes that the test item has only been partially detected or blocked.
If there is an orange field at the end of the row of fields in the chart, the attack is considered undetected, whereas a yellow field in that position indicates only partial detection of the attack. In case of ransomware, this means that some, but not all, of the files were encrypted (some files encrypted). If the last field is orange, everything has been encrypted (files encrypted).
If everything is detected and blocked, the product receives the maximum points total for the protection score. This maximum total can vary from test to test. In this test, it is up to four points. In the final overview, a product can therefore achieve up to 40 points in a total of 10 scenarios. That said, partial detections, which are highlighted in yellow, often occur. In these cases, 10 attacks have been detected but the product does not receive the full point score for 100 percent defense.
Test scenarios
All attack scenarios are documented according to the standard of the MITRE ATT&CK database. The individual sub-techniques, for example “T1059.001”, are listed in the MITRE database for “Techniques” under 1059.001 “Command and Scripting Interpreter: PowerShell”. Each test step is thus defined among the experts and can be logically understood.
Consumer users: live attack test with ransomware
In the current test, 12 consumer products from the following vendors are being subjected to the tests performed by the experts in the lab: Avast, AVG, Bitdefender, F-Secure, G DATA, K7 Computing, Kaspersky, Microsoft, Microworld, NortonLifeLock, PC Matic and VIPRE Security.
Each of the products has to prove its worth in the 10 scenarios with various modes of attack. All of the attacks involve the user receiving an e-mail with an attachment. This attachment is dangerous in each of the scenarios, for example infected PowerPoint files, scripts or packed archives containing malware. The test shows that all of the products already detect the attackers in the first steps (initial access or execution). 11 of the 12 protection packages also block any further execution of the attack at this stage and therefore receive the full total of 40 points. Only K7 Computing has a problem: although it detects the attack, it nonetheless still allows the attacker to create a file further down the line in scenario number 6. Although this file is harmless, 0.5 points are deducted from the overall score.
The end result of the test for home user products reveals that 11 products receive the full score of 40 points, while K7 Computing is awarded 39.5 points. Given that all of the products tested achieve at least 75 percent (30 points) of the maximum of 40 points, they all receive the “Advanced Certified” certificate.
Corporate users: live attack test with ransomware
The lab is testing 14 protection solutions for company networks in 10 real-life scenarios. Products from the following vendors are being put to the test: Acronis, Avast, Bitdefender (two versions), Comodo, F-Secure, G DATA, Kaspersky (two versions), Microsoft, Seqrite, Symantec, Trellix and VMware.
This test also involves the 10 defined scenarios. The primary mode of attack is an e-mail with an infected attachment. The attachment always contains dangerous attackers, for example in the form of Office files with scripts, which then execute further steps via tools such as PowerShell.
In the test, all of the products already detect the attackers in the first steps (initial access or execution). Only 10 of the 14 products, however, are able to detect the attacks and fully block them. The four products from Symantec, Seqrite, VMware and Trellix allow the attack to progress further.
The Symantec and Seqrite solutions fail to prevent the encryption of individual files in further steps, as does VMware, which additionally allows the background file of the desktop to be changed. It usually contains a reference to the attack by the ransomware group.
Trellix has the same problem as VMware – but the background image of the desktop is not changed once, but seven times. Although the file itself is harmless, points are deducted in each individual case.
In the final result, 10 products for corporate users achieve the full total of 40 points. These are followed by Symantec with 39.5 points, Seqrite and VMware with 39 points each and finally Trellix with 36.5 points.
All of the business products are awarded the “Advanced Approved Endpoint Protection” certificate because they achieve 75 percent (30 points) of the maximum protection score of 40 points. Only Acronis received no certificate. The product passed the test error-free, however AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfilment of all their criteria.
When even ransomware has no chance whatsoever
This test yet again shows that a protection solution needs to do more than just detect malware. The individual protection products for consumer and corporate users should, however, be credited for the fact that the errors made were at least not so severe that entire systems were encrypted.
Nonetheless, the security solutions should be able to detect hidden ransomware in real-life scenarios with zero errors. In this test, many products proved that this is indeed possible. Nearly all of the packages for consumer users were without error and received the maximum total of 40 points. Only K7 Computing made a small but fortunately harmless error.
The test on the solution for corporate users revealed that 10 of the 14 products tested were able to perform without error in the scenarios and thus guarantee companies a high level of protection. Although the errors made by Symantec, Seqrite, VMware and Trellix caused their manufacturers to lose valuable points, they were not so severe that the ransomware was able to fully execute its destructive mission.
