March 21, 2022 | Antivirus for Windows
  • Share:

29 Protection Solutions against Data Stealers and Ransomware under Windows 10

In its series of Advanced Threat Protection tests, the lab at AV-TEST evaluated 29 protection solutions for consumer users and corporate users against data stealers and ransomware. The object of this live attack test was to fend off 10 real-life attack scenarios under Windows. In doing so, in addition to detection of malware, each individual defensive step of a protection solution is important. In the recent December test, fortunately only a few attackers were successful in the scenarios.

Advanced Threat Protection test

29 security solutions put to the test against ransomware and data stealers

zoom

The Advanced Threat Protection tests from AV-TEST evaluate protection software in the lab using very realistic and dynamic attack scenarios. As these tests involve a great deal of time and effort, the attacks per product are limited to 10 scenarios. In the December test, the lab carried out 5 special attacks with ransomware and 5 attacks with so-called data stealers. The findings are very interesting, as the solutions for corporate users were highly effective in withstanding all attacks, whereas two products for consumer users had problems.

29 products in the Advanced Threat Protection test

The test line-up involved 15 products for consumer users from Acronis, Avast, AVG, Bitdefender, BullGuard, F-Secure, G DATA, Malwarebytes, McAfee, Microsoft, Microworld, Norton, PC Matic, Protected.net and VIPRE Security. Moreover, the lab evaluated the 14 corporate solutions from Acronis, Avast, Bitdefender (two versions), Comodo, F-Secure, G DATA, Malwarebytes, McAfee, Microsoft, Sangfor, Sophos, Symantec and VMware.

Each product in this test was required to successfully withstand 5 scenarios with ransomware and 5 scenarios with data stealers. In the process, the lab examined each individual step of the attack. It starts out with an incoming e-mail, the detection of the attacker and the evaluation of the types of scripts or accessory tools that are being launched. Subsequently, each additional step of the attack has to be blocked. If a protection program detects and blocks an attacker, the attack is considered identified and resolved. In this case, the test candidate receives the full point score for protection. The maximum score in this test is 45 points.

Protection packages against ransomware and data stealers

In the Advanced Threat Protection test in December, the lab at AV-TEST evaluated protection packages for consumer users against ransomware und data stealers

zoom ico
Endpoint solutions against ransomware and data stealers

A perfect test for all 14 corporate solutions – every protection solution fends off all attacks, garnering the maximum number of points, for a protection score of 45 points

zoom ico

1

Protection packages against ransomware and data stealers

2

Endpoint solutions against ransomware and data stealers

Ransomware and data stealers

An attack with ransomware is quickly explained: The attackers try to penetrate the system, launch malware, encrypt the system’s data and demand ransom for decryption. This type of assault tends to be a mass attack, and is often carried out per botnet.

The attack with a so-called data stealer is far more intricate, as it is normally preceded by a spear phishing attack. This factor alone demonstrates that considerable time and effort are put into the attack, more is invested, and it is sometimes aimed at a particular victim.

Data stealers are a type of two-pronged malware. On the one hand, an attack is carried out that resembles a ransomware assault. However: if the attack is successful, as a deception or diversion tactic, only a small volume of data is encrypted or simply renamed, in order to fake an encryption. Afterwards, a ransom is demanded. The actual target, however, is the theft of data in the background, which is otherwise not typical. The data then ends up in the darknet or with a client, e.g. in the case of corporate espionage. The lab carried out 5 of these special attacks in December in its evaluation of the products.

Although the attacks with ransomware and data stealers are similar, the analysis diagrams according to the MITRE ATT&CK matrix charts reveal clear differences. In the diagrams, there is technical documentation of the individual steps according to MITRE. In other words, an attack step is defined in the MITRE ATT&CK database, e.g. "T1059.001" stands for "Command and Scripting Interpreter: PowerShell". All experts documenting an attack simply utilize these technical designations for the attack steps. The 10 illustrations of the scenarios used in the tests specify the sequence and documentation descriptions.

This technical documentation is vital in the forensic analysis of an attack. It can be more easily demonstrated, for example, that data is also typically siphoned off during an attack. If a well-known ransomware is used, then most of the time there is no data transfer. If a data stealer is identified or newly catalogued according to MITRE ATT&CK, then data theft does occur. This is important, e.g. for reporting an attack according to the EU's General Data Protection Regulation (GDPR).

The analysis matrix according to MITRE ATT&CK

The below charts show the actions and reactions of each protection software to each individual scenario. The configuration of the charts is essentially very basic: If an attack is already fended off in the beginning in one of first two steps "Initial Access" or "Execution", the attack is considered successfully thwarted. For this, a product then receives the maximum number of points for the protection score. In this test, it means scoring 3 to 5 points. The outcome of the attack is color-coded: as soon as an attack is fended off in one test item, the field is highlighted in green. The sooner a green field can be seen, the better. If a field remains orange, the test item is considered undetected (no detection). If there is an orange field at the end of the chart, the attack is considered undetected, whereas a yellow field in that position indicates only partial detection of the attack. In case of ransomware, this means that some, but not all, of the files were encrypted (some files encrypted). If the last field is orange, then everything has been encrypted (files encrypted).

Test scenarios

All attack scenarios are documented according to the standard of the MITRE ATT&CK database. The individual subtopics, e.g. "T1059.001" are listed in the MITRE database for "Technics" under 1059.001 "Command and Scripting Interpreter: PowerShell". Each test phase is defined among experts and can be logically understood.

01
zoom ico
02
zoom ico
03
zoom ico
04
zoom ico
05
zoom ico
06
zoom ico
07
zoom ico
08
zoom ico
09
zoom ico
10
zoom ico

1

01

2

02

3

03

4

04

5

05

6

06

7

07

8

08

9

09

10

10

Advanced Threat Protection tests for consumer users

In the lab test performed in December 2021, a total of 15 protection products for consumer users under Windows 10 faced off against 10 attack scenarios with ransomware and data stealers. A total of 13 out of the 15 products completed the test with a perfect result of a maximum 45 points: Acronis Cyber Protect Home, Avast Free Antivirus, AVG Internet Security, Bitdefender Internet Security, F-Secure SAFE, G Data Total Security, Malwarebytes Premium, McAfee Total Protection, Microsoft Defender, Microworld eScan Internet Security Suite, Norton 360, PC Matic and VIPRE AdvancedSecurity. The charts clearly show that the attackers were immediately identified and fended off.

Only in one instance each, the products BullGuard Internet Security and Protected.net Total AV had a problem identifying the attacker. The data stealer was able to complete its destructive work in the test. For both products, this resulted in 5 fewer points in the protection score, and thus an overall result of 40 points each.

As all products for consumer users achieved at least 75 percent (33.75 points) of the maximum 45 points possible for the protection score, they received the certificate of "Advanced Certified". Only Acronis received no certificate. The product passed the test error-free, however AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfilment of all their criteria.

Advanced Threat Protection tests for corporate users

The December test involving products for corporate users could not have turned out better. All 14 products in the test performed perfectly in each of the 10 scenarios, stopping all the attacks. Thus, all the endpoint products received the maximum 45 points: Acronis Cyber Protect, Avast Business Antivirus Pro Plus, Bitdefender Endpoint Security, Bitdefender Endpoint Security (Ultra), Comodo Client Security, F-Secure Elements Endpoint Protection, G DATA Endpoint Protection Business, Malwarebytes Endpoint Protection, McAfee Endpoint Security, Microsoft Defender Antivirus, Sangfor Endpoint Secure Protect, Sophos Intercept X Advanced, Symantec Endpoint Security Complete and VMware Carbon Black Cloud.

In order to receive the certificate "Advanced Approved Endpoint Protection", a product has to earn at least 75 percent of the protection score points in the test. In this test, it means scoring a minimum of 33.75 points. Thus, all test participants of the products for corporate users received this certificate. Only Acronis received no certificate. The product passed the test error-free, however AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfilment of all their criteria.

Capably fending off ransomware and data stealers

In the special December test, most of the protection programs for consumer users and corporate users showed that they offer good protection even in very realistic cases. Among the programs for consumer users, only 2 packages were defeated in one instance each. All other protection packages worked perfectly.

All endpoint candidates for corporate users worked to complete perfection. The protection solutions identified all attackers and thus reliably stopped the attacks – even among the dangerous candidates from the field of ransomware and data stealers.

In this Advanced Threat Protection test series, AV-TEST always conducts very realistic tests with special scenarios, which are partly aligned with what is currently happening in terms of cyberattacks. 

This article primarily spells out the findings of the Advanced Threat Protection test in December. More technical background and explanations are available from the previously published article APT: Strategic Attacks Require Strategic Tests.

Test results for consumer users: Acronis, Avast, AVG

Acronis 1/2
zoom ico
Acronis 2/2
zoom ico
Avast 1/2
zoom ico
Avast 2/2
zoom ico
AVG 1/2
zoom ico
AVG 2/2
zoom ico

1

Acronis 1/2

2

Acronis 2/2

3

Avast 1/2

4

Avast 2/2

5

AVG 1/2

6

AVG 2/2

Test results for consumer users: Bitdefender, BullGuard, F-Secure

Bitdefender 1/2
zoom ico
Bitdefender 2/2
zoom ico
BullGuard 1/2
zoom ico
BullGuard 2/2
zoom ico
F-Secure 1/2
zoom ico
F-Secure 2/2
zoom ico

1

Bitdefender 1/2

2

Bitdefender 2/2

3

BullGuard 1/2

4

BullGuard 2/2

5

F-Secure 1/2

6

F-Secure 2/2

Test results for consumer users: G DATA, Malwarebytes, McAfee

G DATA 1/2
zoom ico
G DATA 2/2
zoom ico
Malwarebytes 1/2
zoom ico
Malwarebytes 2/2
zoom ico
McAfee 1/2
zoom ico
McAfee 2/2
zoom ico

1

G DATA 1/2

2

G DATA 2/2

3

Malwarebytes 1/2

4

Malwarebytes 2/2

5

McAfee 1/2

6

McAfee 2/2

Test results for consumer users: Microsoft, Microworld, Norton

Microsoft 1/2
zoom ico
Microsoft 2/2
zoom ico
Microworld 1/2
zoom ico
Microworld 2/2
zoom ico
Norton 1/2
zoom ico
Norton 2/2
zoom ico

1

Microsoft 1/2

2

Microsoft 2/2

3

Microworld 1/2

4

Microworld 2/2

5

Norton 1/2

6

Norton 2/2

Test results for consumer users: PC Matic, Protected.net, VIPRE Security

PC Matic 1/2
zoom ico
PC Matic 2/2
zoom ico
Protected.net 1/2
zoom ico
Protected.net 2/2
zoom ico
VIPRE Security 1/2
zoom ico
VIPRE Security 2/2
zoom ico

1

PC Matic 1/2

2

PC Matic 2/2

3

Protected.net 1/2

4

Protected.net 2/2

5

VIPRE Security 1/2

6

VIPRE Security 2/2

Test results for corporate users: Acronis, Avast, Bitdefender

Acronis 1/2
zoom ico
Acronis 2/2
zoom ico
Avast 1/2
zoom ico
Avast 2/2
zoom ico
Bitdefender 1/2
zoom ico
Bitdefender 2/2
zoom ico

1

Acronis 1/2

2

Acronis 2/2

3

Avast 1/2

4

Avast 2/2

5

Bitdefender 1/2

6

Bitdefender 2/2

Test results for corporate users: Bitdefender (Ultra), Comodo, F-Secure

Bitdefender (Ultra) 1/2
zoom ico
Bitdefender (Ultra) 2/2
zoom ico
Comodo 1/2
zoom ico
Comodo 2/2
zoom ico
F-Secure 1/2
zoom ico
F-Secure 2/2
zoom ico

1

Bitdefender (Ultra) 1/2

2

Bitdefender (Ultra) 2/2

3

Comodo 1/2

4

Comodo 2/2

5

F-Secure 1/2

6

F-Secure 2/2

Test results for corporate users: G DATA, Malwarebytes, McAfee

G DATA 1/2
zoom ico
G DATA 2/2
zoom ico
Malwarebytes 1/2
zoom ico
Malwarebytes 2/2
zoom ico
McAfee 1/2
zoom ico
McAfee 2/2
zoom ico

1

G DATA 1/2

2

G DATA 2/2

3

Malwarebytes 1/2

4

Malwarebytes 2/2

5

McAfee 1/2

6

McAfee 2/2

Test results for corporate users: Microsoft, Sangfor, Sophos

Microsoft 1/2
zoom ico
Microsoft 2/2
zoom ico
Sangfor 1/2
zoom ico
Sangfor 2/2
zoom ico
Sophos 1/2
zoom ico
Sophos 2/2
zoom ico

1

Microsoft 1/2

2

Microsoft 2/2

3

Sangfor 1/2

4

Sangfor 2/2

5

Sophos 1/2

6

Sophos 2/2

Test results for corporate users: Symantec, VMware

Symantec 1/2
zoom ico
Symantec 2/2
zoom ico
VMware 1/2
zoom ico
VMware 2/2
zoom ico

1

Symantec 1/2

2

Symantec 2/2

3

VMware 1/2

4

VMware 2/2

Social Media

We want to stay in touch with you! Now there is an easy way to receive regular updates on the latest news and test releases.