Test and Study: Do Security Solutions stop Current Ransomware under Windows 11?

Real-life attack scenarios with the latest techniques

In this test, each product is confronted with 10 special scenarios, comprising three different attack techniques currently being used by cyber gangsters or APT groups.

SysWhispers: This attack technique is a technically sophisticated approach to escape detection by a security solution or EDR. Popular malware that has been around for some time using this technique is BluStealer Loader, for example.

Reflective DLL Injection: In this technique, a malicious code is executed as part of a trusted code, thereby manipulating a legitimate process in such a way that it loads and launches the DLL with malware. The remarkable feature: The attack is fileless, i.e. it is written directly into the memory of the legitimate process. Which means there is no file to be detected. This technique has been used by Netwalker Fileless Ransomware, among others. There is a number of current attackers that continue to emulate this attack technique.

Reflective DLL Loading: This attack technique requires no additional legitimate process, rather it loads the malware and an encrypted payload, only unpacking them in the memory, which makes detection of the attack extremely difficult. Among the malware making use of this attack mode is LokiBot, which was used by the SilverTerrier group.

All of these techniques can harbor many types of malware or be used directly as a means of attack. For those interested in being able to trace the steps more effectively, the descriptions to the 10 scenarios (see "Scenarios" box below) have been expanded accordingly. Consistently featured there are the "Techniques" codes from MITRE ATT&CK, in order to trace the individual steps of the attack. This also helps to identify the step where the security solution did or did not respond. Now additionally included is the technical description of the actual attack, along with the attack technique deployed. In this test, it is a very detailed description of the three techniques briefly mentioned above, SysWhispers, Reflective DLL Injection and Reflective DLL Loading.

22 security solutions and packages in the lab

The final evaluation charts very clearly indicate when a security product detected, failed to detect, fended off or failed to fend off the attack, or thwarted it in subsequent steps or not. For successful defensive steps, the solutions received up to 3 points per scenario from the lab in this test. For all 10 scenarios, the lab awarded up to 30 points for the final protection score.

As explained above, the procedure is clearly defined in the 10 test scenarios and the result is documented. The charts of results with the 10 evaluations are color-coded for an easier overview. If a security solution detects ransomware in one of the first two steps (initial access or execution), the attack is considered thwarted. If this is the case, it is color-coded in green: attack stopped. Yellow means: only partially stopped. Orange indicates: attack not stopped (no detection). The yellow field at the end can indicate two results: if the attack is only partially detected, then there is either encryption of individual files (some files encrypted) or the ransomware was indeed prevented from encrypting files but it is able to remain on the system (malware remains on system). If there is an orange field at the end of the row of fields in the chart, the attack is considered undetected and the ransomware is able to launch completely (files encrypted).

Test scenarios

All attack scenarios are documented according to the standard of the MITRE ATT&CK database. The individual sub-techniques, for example “T1059.001”, are listed in the MITRE database for “Techniques” under 1059.001 “Command and Scripting Interpreter: PowerShell“. Each test step is thus defined among the experts and can be logically understood.

In addition, all attack techniques are explained, along with how successful the malware is.

Ransomware protection for consumer users

In the evaluation of the 9 security packages for consumer users under Windows 11, there were indeed many good results, but also some disappointments. Flawlessly detecting and blocking all further attack steps in all 10 scenarios were the packages from Kaspersky, Microsoft and PC Matic. For this they were awarded the full 30 points.

While Bitdefender and NortonLifeLock also detected all the attackers, in one scenario they were not able to fend them off completely, and the encryption of individual files did occur. As a result, the two packages still earned 29 out of 30 points.

Microworld and G Data also had their problems in one scenario. Both detected the attacker but could not stop it completely. Microworld was able to prevent the malware from creating a registry key in the system – G Data was not. In the end, the data was encrypted, and both products had important points taken off as a result. Thus Microworld scored 28.5 and G Data 27.5 points.

VIPRE Security responded perfectly in 9 scenarios, but did not detect the attacker at all in one instance. In one case, 0 points were awarded for the encrypted system – thus the tally was 27 points in the end.

While the security software from AhnLab was able to detect the attackers in 10 out of 10 scenarios, in 4 instances, however, it did not block the attack completely. In one case, a partial encryption occurred, and three times the entire Windows 11 system was encrypted, followed by the classical ransom demand via text display. This amounted to only 24.5 points.

As soon as a product in the test achieves at least 75 percent of the maximum 30 points, it receives the certificate "Advanced Certified". As all the products reached a protection score of more than 22.5 points, all of them earned this certificate.

Corporate user solutions against ransomware

Among the security products for companies, the final result was considerably more favorable. A total of 9 out of 13 security solutions detected all the attackers error-free, successfully protecting the Windows 11 test systems. Thus, all attacks, as they are carried out daily on the systems from corporate users, were fended off.

Garnering the full 30 points were the solutions from Acronis, Check Point, Xcitium, Kaspersky (both versions), Microsoft, Symantec, Trellix and VMware.

Bitdefender with its Endpoint Security and Endpoint Security Ultra each had a problem in the same scenario. The solutions were in fact able to detect, but not completely block the attacker. As a result, there was partial encryption of individual files. Both product versions thus received 29 out of a possible 30 points.

The corporate solution from G Data had severe problems in one instance, although it detected the attacker. However, the security solution could not block the malware which was able to execute, write registry keys and encrypt the data of the test system. For this instance it only received 0.5 out of 3 points – the other 9 test instances were without a glitch. As a result, G Data racked up a total of 27.5 points.

The solution from AhnLab finished in last place in the point standings with 27 out of 30 possible points. The product was in fact able to detect the attackers in 2 instances, but it did not block them completely. Although the creation of registry keys was thwarted, the ransomware continued to execute, encrypting the entire system in each case.

When a product receives 75 percent of the maximum achievable 30 points, the lab awards the certificate "Advanced Approved Endpoint Protection". In this test, all the products came in above the necessary 22.5 points and received the certificate. Only Acronis received no certificate. The product passed the test error-free, however AV-TEST only certifies products that achieve certification in the regular monthly tests and fulfilment of all their criteria.

Sophisticated attacks demand flexible and secure solutions

The latest evaluation demonstrates that many security products for consumer users and corporate users are also able to stand up against actively used attack techniques along with the latest ransomware under Windows 11.

The situation is somewhat more favorable involving solutions for companies than with products for home users. In the latter category, only the three products from Kaspersky, Microsoft and PC Matic earned the full 30 points in the protection score.

Among products for corporate users, a total of 9 out of the 13 security solutions evaluated were able to reach the full 30 points in the protection score: Acronis, Check Point, Xcitium, Kaspersky (both versions), Microsoft, Symantec, Trellix and VMware.

The descriptions of the 10 scenarios, along with attack techniques, indicate the sophisticated ways in which the attacks unfold and what types of tricks the attackers use to successfully encrypt a system. The security solutions are required to monitor and evaluate many operations and processes in the Windows 11 systems. At the same time, they have to decide which operations are harmless and when to block dangerous processes. Not an easy task, especially in the case of fileless attack techniques that occur directly via processes in the memory, thus seeking to circumvent defense mechanisms. But the security solutions for Windows 11 are able to detect and stop these attacks as well – at least the majority of the products.