The Best Internet Security Suites for Windows Complete an Endurance Test Lasting 6 Months
A large corpus of individual data was collected in three tests that took place between January and June 2013. This corpus now forms the basis of the overall result of the AV-TEST endurance test on 28 Internet security suites. A total of 24 suites underwent all three tests, while two packages only completed two tests and two other suites were only tested once. These programs are marked accordingly in the results table.
Can protection packages improve the basic protection provided by Windows?
The tests were each carried out once using Windows XP, 7 and 8 operating systems. The products tested included the internal security solutions provided by Microsoft, namely Windows Defender in the test using Windows 8 and Security Essentials in the tests using Windows XP and 7. Defender and Security Essentials were both tested together with the Windows Firewall. The results of the tests involving these Microsoft solutions are used as base values for the other packages involved in the endurance test. If the other packages achieved higher values, this indicates that using an external Internet security suite is a worthwhile decision. The endurance test delivered a clear outcome, with the overall results after three tests showing that all but one of the suites examined achieved better scores than the basic value of the Microsoft solution.
Well-Known Names in the Top Ten
On the whole, the test results show that the development and average performance of the protection packages have remained on the same consistent level. The average detection rates of the programs, which are used to evaluate the protection that they provide, lie between 93 and 98 percent. These values were, however, outperformed by well-known names such as Bitdefender, F-Secure and Kaspersky Lab, which achieved detection rates of 100% in both areas after three tests, while the packages from GData and Symantec are very close behind.
The results achieved by the basic Windows protection programs, namely 79 and 97 percent, are below the average value of all test candidates.
High Scores in the Overall Results
The maximum number of points available to the candidates in the individual categories of "Protection”, “System Load” and “Usability” was six points, resulting in a maximum overall total of 18 points. The package from Bitdefender made it to the top of the table with an overall score of 17.2 points, followed by the protection suites from Kaspersky Lab and Symantec with 16.0 and 15.5 points respectively.
Fourth place in the results table is shared by F-Secure, GData and Avast, with the Avast suite achieving the honour of being the best free protection package tested. Nevertheless, the product from Avast only made it to eighth place when assessed according to its protection alone.
How We Measured the Protection Provided
The first major hurdle facing all of the packages involved our protection measurements. This stage of the examination required all programs to pass two detection tests, the first of which was the real-world test, which used approximately 400 brand-new pieces of malware, so-called zero-day malware.
This complex real-world test required the protection packages to fend off attacks from the latest malware found on websites and in e-mails, downloads or drive-by downloads, just like they may be expected to do during the day-to-day use of e-mail and the Internet.
The second test, the AV-TEST reference set test, required the packages to detect and remove approximately 60,000 pieces of malware that had been collected in the four weeks leading up to the start of the test.
A More Complex Procedure for More Precise Results
The test using the AV-TEST reference set involved more than just simply scanning the files and making a record of detected malware. After all, most of the protection packages not only feature basic detection functions, but also a number of other important tools that they use to identify malware.
Example: A protection software may be confronted with 20,000 infected files and successfully identify 17,000 of these malicious files in the first scan. The 3000 files that were not detected by the software are then individually copied onto and run on the PC in the test laboratory. The protection packages are therefore given another chance to identify malicious files using their on-demand scanner. When a file is executed, the packages can examine it with another scanner, namely their on-access scanner, as well as ultimately being able to activate their behavioural detection function. A protection solution is only recorded as having failed to detect a piece of malware when all of its security measures have been exhausted.
A Detection Test Using 45 PCs that Work around the Clock
The levels of complexity and uniqueness of this test alone in comparison with those carried out by other test laboratories are so high that AV-TEST uses a total of 45 of its own computers to obtain results around the clock. After all, the test platform using Windows has to be re-installed after every single infection.
The packages involved in the endurance test have to complete the real-world test and the test using the AV-TEST reference set once every month. The endurance test therefore examines the scanning performance recorded in six consecutive tests!
High Security at the Expense of System Speed
The worst scenario for a PC user is security software that paralyses or significantly slows down Windows when used. The tests carried out in the AV-TEST laboratory therefore also measured how long websites and downloads took to load, the duration of the installation or execution of programs and the time required to copy files. These tasks were carried out twice, once with the security solution in question and once without it, in order to obtain results for comparison.
Although the best programs in the “Protection” category also achieved excellent results in this “System Load” category, none of them were able to score the maximum total of six points. This test category is proof that high security comes at the expense of a certain amount of system performance. In fact, the top ten products in the endurance test were only able to achieve an average of 4 of a maximum possible total of 6 points. The leading package from Bitdefender was the only one of these suites to put less of a load on the system, achieving an average of 5.2 of the 6.0 points available.
Usability on a Day-to-Day Basis
How do the protection packages behave when used on a day-to-day basis? Do they suddenly block safe websites or disable normal software after incorrectly identifying it as malware? In order to obtain the answers to these questions, each of the protection packages tested was used to visit 500 websites in the background, as well as having to identify over 600,000 safe files without making any errors.
On top of all this, all of the security suites also had to prove that they do not delay or even block the execution of standard programs or their actions.
25 of the 28 suites tested achieved 5 to 6 points of a maximum possible total of 6.0 points in the test category of “Usability”. The Microsoft solution also scored extremely well in this category in comparison to all of the other products, achieving a total of 6.0 points. The only other packages to achieve this maximum value were the suites from Bitdefender, Symantec, Avast, AVG (purchasable version) and Check Point.
Summary: Windows Is Easy to Protect
This test summary only covers the protection suites that completed all three tests between January and June. Suites that only completed one or two tests cannot be compared to the other products.
The endurance test is able to quickly expose protection packages that only achieve top results as a one-off. In the “Protection” test category, for example, the real-world test shows the strength of suites when used to detect pieces of malware that are still considered to be brand new. The suites from Bitdefender, F-Secure and Kaspersky all did the best job in this category, achieving detection rates of 100%, while the best free programs, namely those from Avast and AVG, were only able to make it to eighth and twelfth place respectively. The Windows Defender provided by Microsoft in its operating system set a very low benchmark value with a detection rate of just 79 percent.
A similar product ranking can also be seen in the overall results table, in which Bitdefender, Kaspersky Lab and Symantec lead the field while the protection packages from Avast, F-Secure and GData share fourth place.
The test results of these seven products alone are all nearly 5 points higher than the overall result obtained by Microsoft's Windows Defender or Security Essentials when used together with the Windows Firewall. This proves that the use of external security solutions can lead to a massive improvement when it comes to system protection.