When malware doesn't have a chance

The key element of the endurance test is the evaluation of a product's protection capability. Here is how the lab goes about it: In the real-world test, the experts use freshly-caught zero-day malware from emails and downloads. These brand-new malware samples are particularly dangerous and also cannot be easily detected by some programs. In the second test, involving the so-called reference set, the lab examines whether the products can also detect already registered malware. All the packages were required to successfully complete a round of testing each month and to detect, quarantine and delete nearly 40,000 malware samples.

More than half the participants, 11 to be exact, passed both test phases of protection evaluation absolutely error-free: Avast, AVG, Bitdefender, ESET, F-Secure, G Data, Kaspersky, McAfee, Microsoft, NortonLifeLock and Trend Micro detected all attackers 100 percent.

Whereas the remainder of the test field also managed the reference set error-free, there were issues in the real-world test. BullGuard, for example, detected 99.9 percent. AhnLab, Avira and VIPRE Security achieved high levels of 99.7 and 99.6 percent respectively. From Protected.net on down, the detection rate dropped considerably, however. Here, the results were only 98.4 percent. For K7 Computing, Malwarebytes, Microworld, and PC Matic, the detection rate dropped further from 98 all the way to 93.6 percent.

Detection, system load, slow down

While several of the production packages are consistently effective in detecting the malware samples, some of them use a good amount of PC resources in the process. The lab meticulously measured the extent to which this is the case for some security suites. In doing so, many operations are carried out on a high-end PC and a standard PC, clocking the time required for each. This includes opening websites, installing, and launching programs, as well as performing downloads and copying data. In the test, all these operations are repeated, but this time a protection product is installed on each system. The result shows that many programs are able to use the system resources of the PC very sparingly, thus also earning the full 6 points. The list includes these 12 products:  AhnLab, Avira, Bitdefender, BullGuard, F-Secure, K7 Computing, McAfee, Microsoft, NortonLifeLock, PC Matic, Protected.net and VIPRE Security.

A minor load on the PC was caused by G Data, Kaspersky, Microworld and Trend Micro. Yet for this they still received good scores of 5.8; ESET earned 5.7 points. With Avast, AVG and Malwarebytes, the load is somewhat higher than with all other products: they all received 5.5 points. In overall terms, the load placed on the system by using protection software really tends to be small.

A dangerous file – or harmless after all?

Every Windows PC handles significantly more harmless files than dangerous ones. That is why an innocuous file may occasionally be falsely detected and classified as a threat. Typical users find this not only vexing but downright annoying. Because then they normally have no way of telling whether they are actually in peril.

Which is why the lab evaluates each product in terms of false alarms. When the security application is active, the lab tests thousands of websites, performs program installations, then launches the applications. The lab continues to copy millions of harmless files onto the system, which the protection packages are also required to recognize as harmless.

More than half of the packages examined successfully completed the tests without any false alarms, receiving 6 points for this: Avast, AVG, Avira, ESET, F-Secure, G Data, Kaspersky, McAfee, Microsoft, NortonLifeLock, Protected.net, Trend Micro and VIPRE Security.

BullGuard, K7 Computing and AhnLab are somewhat overprotective, reporting a few harmless files as threats: for this, they received 5.8 and 5.7 points respectively. Bitdefender and Microworld earned 5.3 points; 5 points were awarded to Malwarebytes. PC Matic filters out far too many harmless files: only 4.3 points.

Good long-term protection for consumers

The endurance test over a 6-month period quickly showed which security product is very reliable. Finishing the test with maximum 18 points were 4 packages from F-Secure, McAfee, Microsoft and NortonLifeLock. Coming in close behind with 17.8 points were Avira, BullGuard, G Data, Kaspersky, Trend Micro and VIPRE Security. Which means that already half the test field can be recommended for all consumer users. Users can thus focus more on the standard features and the enhanced functions of the individual packages.

All in all, 17 out of the 20 protection packages evaluated in the endurance test over a 6-month period achieved between 17 and 18 points. Only 3 candidates were in a range from 15.9 to 14.3 points.