9 Security Packages for Consumer Users in an Advanced Threat Protection Test against Ransomware
The lab of AV-TEST examined protection packages in the Advanced Threat Protection test, a type of live-attack test. Each product in the test was required to withstand 10 sophisticated attacks with scripts, macros and ransomware, precisely resembling the threat scenario posed by hackers. In doing so, each defensive step was evaluated according to the pattern of the MITRE ATT&CK Matrix. The test shows that many products are true system defenders – whereas others are not.
Classical security tests demonstrate how well security packages offer protection against ten thousands of Trojans, viruses etc. But what happens when a new variant of malware attacks? Will it be detected, blocked and deleted? Can its execution be totally prevented, or are there perhaps individual files encrypted in the end, as is the case with ransomware, for example? Answers to these questions are found in the latest lab test by AV-TEST, in which each protection package was required to fend off 10 real attack scenarios with ransomware. In order for every reader to relate to the test, all the single steps of an attack per product are spelled out in special MITRE ATT&CK Matrix charts.
9 protection packages in the Advanced Threat Protection test
In the latest test, 9 well-known consumer security packages faced off to see how well they offered protection in 10 real-life scenarios against ransomware. The protection packages came from Bitdefender, BullGuard, G DATA, Malwarebytes, Microsoft, NortonLifeLock, PC Matic, Protected.net and VIPRE Security.
In another article, readers can also find out just how well corporate security solutions performed in the Advanced Threat Protection test. The article on protection solutions for corporate users can be found here: Protection Solutions vs. Ransomware in the Advanced Threat Protection Test.
Each of the 10 attack scenarios has in its attachment either an executable file, files that launch a hidden macro, or scripts. All of them have the singular objective: launching ransomware to encrypt data.
Each of the attacks in the test scenarios is launched via e-mail with an attachment. That is the classical attack, as carried out on a massive scale by most large hacker groups. Because even private users are also an easy mark for hackers – provided they pay the ransom for their data in bitcoins. And there are more private victims than companies worldwide, as statistics show. This reality is manifest in the vast ransomware campaigns from recent years up to this day. Many users in Germany still recall, for example, how in 2017 at the German railway, Deutsche Bahn, the ransom demand from WannaCry was blinking on nearly every display, along with a payment countdown. Some 250,000 users in 150 countries worldwide saw this demand on their personal computer screens at the same time.
Since 2018 to this day, there have been repeated outbreaks of Emotet malware, which nearly always reaches users via e-mail with attachments. This diabolical malware doesn't just launch an isolated attack on a system; once it lands, it immediately opens up a back door and calls for reinforcement: normally Ryuk for encryption and ransomware extortion or TrickBot, which enslaves a system to mine for cryptocurrency.
The attacks and the evaluation
As already mentioned in the beginning, each of the 10 attacks is evaluated step-by-step, and the steps are recorded in a matrix modeled according to MITRE ATT&CK. The test charts show all the individual steps of the attacks, also plotting how far the attack progressed. If the attack was detected and stopped at any point, the field highlighted in green indicates that the attack was over at that point. However: even if the attack was recorded as having been stopped, the attack may have been successful after all in a small area. Then the problem is logged at the "Impact" point (i.e. the hit sustained, along with its effect) in terms of "some files encrypted" or "files encrypted".
The evaluation by the lab depends on the respective scenario. A product can achieve 3 or 4 points per detected or prevented attack, even if the scenario involves up to 8 steps. This is because the lab may record several steps that are not security-relevant and thus are not counted. Overall, a product can achieve up to a total of 34 points in this test.
In order for a product to receive the certificate "Advanced Approved Endpoint Protection", it has to earn at last 75 percent of the protection score points in the test. In this test, it means scoring a minimum of 25.5 points. All 9 candidates received this certificate.
Here is how the evaluation was conducted in practice
If an attack is fully blocked in one of the first two steps, "Initial Access" or "Execution", the party is over for the ransomware, and the test candidate receives one point toward its protection score. In some scenarios, the next step involves "Persistence". In this scenario, the ransomware adds keys in the registry or entries in autostart, for example. Detection of the operation is essential, for instance, in order to clean a system. If this operation is prevented, one point is awarded towards the protection score here as well.
The steps "Defense Evasion, Discovery" or "Collection" can follow either separately or as a group. However: these are all documentation steps for which no evaluation is made.
The next step, "Command and Control", means the following: After a system has been hijacked by a script or macro, and subsequently by ransomware, this is followed by the step where the controlling server is contacted. Here, the extortion software tries to exploit internal Windows ports in order to establish a channel. If this is prevented by the security software, another point is added to the protection score total.
If the last step, "Impact", is not prevented, this means that the ransomware is able to reach into its bag of tricks, encrypting the data and making its ransom demand. In a test, this tends to be a rare occurrence. Nonetheless, it did occur once with NortonLifeLock; and twice each with BullGuard and Protected.net. In this case, the lab awarded 0 points for the entire scenario for the product in question.
6 out of 9 products achieved the top score
The overview table indicates how well many protection packages averaged in this very comprehensive live-attack test. After all, the objective is not only to detect but also to completely fend off the ransomware. This was achieved by the packages from G DATA, Malwarebytes, Microsoft, PC Matic and VIPRE Security. These 5 products detected 10 out of 10 attackers, thwarting them completely, thus earning 34 out of 34 points.
While the security package from Bitdefender detected all 10 attackers, it was unable to fully stop two of the attacks, suffering a point loss as a result: 31.5 out of 34 points. This was followed by the security program from NortonLifeLock, which failed to detect one attacker and thus received 31 points. With only 8 out of 10 detected attacks and a corresponding score of 28 points, BullGuard and Protected.net were at the end of the ranking.
As all 9 products achieved more than 75 percent of the protection score points, i.e. more than 25.5 points, they were recognized with the certificate "Advanced Approved Endpoint Protection".
G DATA 1/2
G DATA 2/2
Microsoft Defender 1/2
Microsoft Defender 2/2
PC Matic 1/2
PC Matic 2/2
VIPRE Security 1/2
VIPRE Security 2/2