18 Corporate Solutions Put to the Test
The market of security solutions for companies is vast and the range is partly very comprehensive. But even the most comprehensive solution doesn't do an enterprise much good if it provides poor protection or slows down the office PCs through excessive use of system resources. In its two-month test, the lab evaluated 18 solutions for corporate users in terms of protection, performance, and usability on a Windows 10 PC. The top score in the test is 18 points, as up to 6 points can be earned in each of the three test categories. The test period ran from May through June 2020.
9 out of 18 solutions earned the TOP PRODUCT rating
Half the products evaluated by the lab also received the label of TOP PRODUCT in addition to the traditional certificate for tested security. This is a label given only to products achieving 17.5 or the top score of 18 points.
With 18 points, the solutions from F-Secure, Kaspersky, Microsoft, and Symantec shared first place. In hot pursuit at 17.5 points were the corporate products from Avast, Bitdefender (Endpoint Security), G Data, McAfee and Seqrite.
Coming in close behind the TOP PRODUCT group, with an excellent 17 points, were the solutions from Bitdefender (EP Ultra), Check Point, ESET, FireEye, Sophos, and Trend Micro.
The product from VMware Carbon Black still achieved 16 points, Cylance 14.5, and Webroot only 12 points, as protection in particular was too weak.
Security solutions for Windows 10 clients in companies
Malware detection for corporate solutions
2-month test period with just under 45,000 malware samples
In the first test phase, the lab tests all solutions for their protection. The procedure requires the products first to pass the real-world test over a period of 2 months, and afterwards to successfully handle the tests involving the reference set. Whereas the known malware is tested for up to 2 weeks in the set, in the real-world test, the lab uses brand-new malware samples. This so-called 0-day malware is collected and then used from websites and e-mails from a few minutes up to hours prior to the beginning of the test.
While many products successfully completed this admittedly grueling test, some did exhibit weaknesses. This can be seen very clearly by the table with the detection rates.
The solutions from Avast, Bitdefender (with 2 versions), Check Point, F-Secure, Kaspersky, Microsoft and Symantec, showed what a perfect result looks like. In both months and in both test phases, the products for corporate users always found, blocked, and deleted 100 percent of all malware samples. The packages from ESET, G Data and Seqrite each overlooked a few malware samples once in the real-world tests. They had a detection rate of 99.4 percent, otherwise 100 percent. McAfee and VMware also struggled with a few minor problems in the real-world tests. But they still were among the 13 products that achieved the full 6 points in the category of protection.
All other products overlooked several times 0-day malware samples. Overall, most of them, however, exhibited high detection rates. Only Webroot had major difficulties in the real-world tests, detecting only 74.4 and 72.6 percent, respectively.
One remarkably interesting point: in the tests with the reference set, all products were totally error-free in always reporting the attackers 100 percent.
Fast or slow office PC?
The workstation PCs existing in companies usually vary greatly in quality. Some are newer and quicker, while those in other departments may tend to be getting up there in age and are slower. The protection client of a solution is naturally required to work on all PCs in such a way that it does not impair the workflow. In order to evaluate this, the lab uses reference devices of a high-end PC and an office PC. A series of routine tasks are performed on these PCs, such as launching websites, copying files, or installing software. The time required for this then counts as the reference time for the comparison. In a later step, the client of each solution is installed, the identical routine tasks are executed, the times are measured, and the load behavior is documented.
Surprisingly, there were mixed results. Garnering the full 6 points for a hardly discernible system load were the solutions from Cylance, FireEye, F-Secure, Kaspersky, Microsoft, Seqrite, Symantec and Trend Micro.
A very minor load was caused by the clients from Avast, Bitdefender (Endpoint Security), G Data, McAfee, and Sophos. They received only a small point deduction: 5.5 points. Due to a somewhat greater load on the client, only 5 points were awarded to Bitdefender (EP Ultra), Check Point, ESET and Webroot. The solution from VMware created a noticeable load on the client: only 4 points.
F-Secure PSB Computer Protection
Kaspersky Endpoint Security
Symantec Endpoint Security
G Data Antivirus Business
Avast Business Antivirus Pro Plus
Bitdefender Endpoint Security
The test criterion of usability examines the false detections through a protection solution and the related false alarms. In this test phase, all products scan only innocuous files and programs and in doing so, should not sound any alarm or block the workflow. In the test: over 1 million files, 500 websites and over 80 popular programs that are downloaded, installed, and used.
Committing no errors or only one error in all test phases earns the coveted 6 points. These results were achieved by the security solutions from Avast, Bitdefender (both versions), Check Point, ESET, F-Secure, G Data, Kaspersky, McAfee, Microsoft, Sophos, Symantec, Trend Micro and VMware. Out of a mass of over 1 million files, the products from FireEye and Seqrite falsely detected up to 20 files and thus only achieved 5.5 points.
The products from Webroot and Cylance only received 4 points, as they produced too many false alarms. The rate was from roughly 50 to over 400 files.
Conclusion: small and subtle differences
In the test at hand, the lab was also able to recognize 9 out of 18 solutions as TOP PRODUCTs, as they earned 17.5 or 18 points for their performance. The solutions from F-Secure, Kaspersky, Microsoft and Symantec reached 18 points. This was followed by the products from Avast, Bitdefender (Endpoint Security), G Data, McAfee and Seqrite at 17.5 points. An additional 6 products still achieved an excellent 17 points. The field thinned out somewhat after that.
The interesting factor: 12 out of 18 solutions reached the full 6 points in the crucial test area of protection. The majority of the tested solutions did not even require many resources on office PCs for good detection.
The test, however, exhibited minor differences, manifested by somewhat more system load or an occasional false positive. But most products still operate on such a high level that ultimately the performance differences are only exceedingly small.