Endurance Test with 18 Internet Security Suites

Achieving the highest security level 9 times

When considering the test only in terms of protection, the result looks somewhat different. In this respect, the lab examined all suites over a 6-month period in the real-world test, first with over 1,000 samples in the category of zero-day malware. In the second test segment, also over a period of 6 months, all products were required to detect the reference set with just under 60,000 attackers.

The products from Kaspersky, NortonLifeLock and Trend Micro completed both test segments with 100-percent detection, achieving this throughout the 6-month test period. While the solutions from Bitdefender and Check Point seamlessly detect the reference set, they do commit minor errors in the real-world test, however: 99.9 percent.

The following security packages in the list all make a very strong showing in the test with the reference set, mostly achieving 100 percent. The poorest detection rate here is seen in Protected.net with 99.8 percent. By contrast, the real-world test separates the chaff from the wheat. Whereas the poorest detection rate is 99.9 percent for the first 7 products, Windows Defender is the first here to drop down to 99.7 percent. From there on in, the detection rate in the real-world test becomes continuously worse. The product coming in last, Protected.net, can detect a mere 85.9 percent of the malware files.

In total, 9 products received the full 6 points for their detection capability: Avast, AVG, Avira, Bitdefender, Check Point, K7 Computing, Kaspersky, NortonLifeLock and Trend Micro.

System protection or system slowdown?

Users are rightly annoyed if security software, at the height of providing good system protection, requires so many system resources that it nearly paralyzes the system. Luckily, these drastic scenarios have been a thing of the past for quite some time now. However, impact on the system is considerably higher for some products than for others. The lab uses a high-end PC for this test and a standard PC, on which many defined operations are carried out: launching websites, installing and starting up programs, performing downloads and copying data. The times required for this are then used as reference values for the test. The lab then repeats all the operations – with installed protection software, however. Later, the experts compare the times required with the reference time and thus award the points for each solution. In the endurance test, the security solutions were put through the works, and after 6 months, the average performance per package was established.

In all routines tested, the packages from Kaspersky, McAfee, Microworld, NortonLifeLock, PC Matic and VIPRE Security create hardly any load on the system. For this, they received the full 6 points. Somewhat more measurable are the load values of Bitdefender, K7 Computing and Microsoft. For this they still received good scores of 5.8 and 5.7 points respectively.

A large share of the Internet security suites exhibited an increased, yet not especially high, system load on Windows: Avast, AVG, Avira, BullGuard, G Data and Trend Micro. As a result, these packages earned 5.5 out of 6 points. AhnLab still received 5.3 points.  The packages from Check Point and Protected.net have been shown to use too many system resources and thus received only 4.7 and 4.3 points respectively.

User files on the black list

So what happens if security software classifies a harmless program file as a malware file? Normally the application crashes, cannot be re-launched or has to be re-installed. In the worst case, the innocuous file is part of Windows, is blocked and the system crashes. While that doesn't happen too often, it has indeed happened in the past.

The lab at AV-TEST has been collecting harmless application files continuously for years. All security tools are required to examine and identify as harmless this storehouse of millions of files. Only then can users be assured that the security suites will not trigger false positives. But the lab drills down much deeper in its analysis: it also visits thousands of websites and has the tools monitor the installation of nearly 200 clean applications.

Here, the best ratings were achieved by the packages from Avast, AVG, Avira, Check Point, Kaspersky, McAfee, Microsoft, NortonLifeLock and Protected.net. They either commit no errors at all or only one error, thus deserving the full 6 points.

The large midfield frequently classifies individual harmless application files or websites as risky and triggers false alarms. This led to a deduction of points. The following security packages completed the test with 5.8 to 5.3 points, which is still acceptable: VIPRE Security, Trend Micro, BullGuard, Microworld, Bitdefender, K7 Computing, AhnLab and G Data.

Only PC Matic doesn't know its way around the world of harmless files, falsely quarantining too many files. As a result, it earned only 3.5 out of 6 possible points.

9 out of 18 packages with 17.5 to 18 points

The endurance test shows all users which security package they ought to invest in, or whether they are relying on the right solution already. While standard tests always provide only a snapshot of a security package's quality, the endurance test enables far greater insights in terms of consistency. The Internet security suites from Kaspersky and NortonLifeLock proved absolutely consistent. Both completed the endurance test from July to December 2019 with the top score of 18 points.

Although 7 packages followed with 17.5 points, only Avast, AVG, Avira and Bitdefender were also able to achieve the full 6 points in terms of protection. Even Windows Defender is in the mix with 17.5 points. But in terms of security, it can only boast 5.8 out of 6 points. K7 Computing, Trend Micro and Check Point offer more security with 6 points, but significantly less convenience in terms of performance and usability.