32 products put to the test: How good is antivirus software at protecting itself?
The laboratory of AV-TEST is a stickler when it comes to how well security solutions for consumers or corporate users are able to protect themselves. That is why the experts examined 32 security packages in terms of how well they deploy protection technologies such as ASLR & DEP. The testers also looked at distribution channels and additional means of protection.
For the third time now, the experts at AV-TEST examined a large number of the security software applications on the market in terms of their self-protection. The focus was on file protection with ASLR & DEP, which programmers can easily use – or foolhardily forget. Moreover, it was examined whether the distribution of test versions use secure channels so that attackers cannot distribute any manipulated software packages. The technology of forced integrity checking can protect executable files through the use of certificates. This criterion was evaluated as well.
Self-protection with ASLR & DEP in business solutions
Individual and comparative figures of DEP & ASLR in consumer packages
Individual and comparative figures of DEP & ASLR in enterprise solutions
Signed files in consumer software
Signed files in business solutions
Who offers secure downloads
Many manufacturers are quick learners
As early as November 2014, it came as a severe shock to several manufacturers: at that time the initial findings were released on self-protection with ASLR & DEP for antivirus software. Back then, the results for some manufacturers were not pretty. After all, the test only examined whether the freely available programming technologies of ASLR and DEP were utilized for protection of 32- and 64-bit user-mode PE (portable executable) files. This technology is easy and economical for manufacturers, and has no influence on software performance. The ASLR and DEP technology is precisely explained in the article from October 2015. In 2014, only 2 manufacturers deployed the technology 100%. The low mark at that time was under 20%.
The test shock had an effect, as by the time the repeat test rolled around in 2015, the statistics were significantly better. Whereas the lowest figure was 18.7% back then for the software that brought up the rear, its numbers were suddenly above 95%.
The current test includes 19 security packages for private users and 13 solutions for corporate users. A total of 16 solutions protect themselves 100% with ASLR & DEP! Manufacturers are also quick learners. For individual products, the experts anticipate even more improvements. The low point in terms of ASLR & DEP use this time was 36.3% for the product at the bottom of the table.
Some suppliers informed the lab experts after the last tests that their product files in the test would never reach 100 percent. Because they used protection technologies that were not compatible with ASLR and DEP. Yet the manufacturers do not wish to disclose which technologies are involved.
1. Security test: 16 out of 32 solutions use ASLR & DEP 100%
As described, many of the products use the ASLR and DEP technology for their program files, in the respective 32-bit and 64-bit versions. Here is the exact breakdown of the products for private users and corporate users.
19 solutions for private users
The packages from Avira, Bitdefender, ESET, F-Secure, Kaspersky Lab, MicroWorld, Symantec and Trend Micro use the ASLR and DEP security technology without exception. Also achieving an almost perfect score are BullGuard with 99.6 percent, G Data with 98.8 and AVG with 97.2 percent. Additional manufacturers such as Comodo, Emsisoft, Avast, McAfee, ThreatTrack, Quick Heal and K7 do not use the technology consistently enough. Their rates range between 92.2 and 58.5 percent. Ahnlab, which achieved the lowest percentage, deploys ASLR & DEP even as low as 36.3 percent. The manufacturer needs to act swiftly to remedy the situation.
13 corporate security solutions
In the products for corporate users, the manufacturers were considerably more consistent in applying ASLR & DEP security technologies. Thus the products from AVG, Bitdefender, ESET, F-Secure, Kaspersky Lab and Symantec use the technology 100 percent. The packages from Trend Micro, McAfee, G Data and Sophos still need to do some fine tuning: the figures range between 98.1 and 99 percent. The developers of Seqrite really need to come up with a remedy. The deployment percentage of 77.1 is definitely too low.
2. Security test: Are the program files signed?
The signing of files with certificates may be old hat for programmers, but it is safe. Signatures and certificates for files are important, as they help identify the manufacturer from which they originate. Unsigned files are always a potential security gap. Because for self-protection, a software application has to be able to verify the authenticity and integrity of its own files. Digital signatures with valid certificates along with hash values are helpful in this. That is why on all packages, for user-mode 32- and 64-bit PE (portable executable) files, the lab also checked the use of a certificate and its validity. And here, it was also evident that some manufacturers work very precisely – others in turn need to quickly have their file certificates examined.
Certificates for consumer software: The manufacturers Bitdefender, Comodo, Emsisoft, ESET, G Data, Kaspersky Lab, McAfee and Symantec use valid certificates for all their 32-and 64-bit PE files without exception. Many of these manufacturers are also among those that use ASLR & DEP 100%. For Avira, F-Secure, BullGuard, MicroWorld, Ahnlab and AVG, 1 to 5 files for are not signed or use an invalid certificate. For Trend Micro and Avast, there are already 6 and 8 files respectively, which are not correctly signed. The products from ThreatTrack and Quick Heal even have 13 and 40 vulnerable files respectively in the package.
Certificates for enterprise software: The level of due diligence when it comes to signed files is much higher for corporate solutions. Of the 13 products evaluated, 8 had signed their files perfectly: Bitdefender, ESET, F-Secure, G Data, Kaspersky Lab (both versions), Symantec (cloud version) and Trend Micro. With AVG, Sophos and Symantec Endpoint Security, one file still remains unsigned. With McAfee, 4 files are not signed, with Seqrite it is even 42 out of 256 existing PE files, i.e. over 16 percent.
3. Security test: Software distribution via secure channels
When distributing software via manufacture websites, everyone really ought to be using the secure web protocol HTTPS. Even browsers such as Chrome warn the user with red fonts in the URL line if no secure protocol is being used. Because this is the only way to ensure secure communication with the server. HTTP allows the possibility of a man-in-the-middle attack. In this way, a download can be manipulated and a counterfeit installation file can be pawned off on a website visitor. Users usually doesn't notice it. If they launch the manipulated file, it will be unsigned, and Windows will indicate this. But in this case, who would distrust the installation of a security application?
While there are hardly any direct downloads for enterprise solutions, almost every manufacturer offers test versions for private customers. The result of this brief evaluation of the downloads is sobering: 13 out of 19 manufacturers offer the test versions via unsecured HTTP. Only Avira, Bitdefender, ESET, F-Secure, G Data and Kaspersky Lab use the secure HTTPS protocol.
It’s up to role models to set a better example
Of all companies, security software manufacturers must utilize all opportunities to make their products secure. After all, they have a role model function in the industry. Some manufacturers do indeed live up to this: Bitdefender, ESET and Kaspersky Lab use 100 percent ASLR & DEP, sign all PE files with valid certificates and offer secure downloads.
For many suppliers, all that is missing is a bit of extra effort and they would be in perfect shape as well. This should not and most certainly will not be any problem for these manufacturers. AV-TEST will evaluate this again in a future test soon to follow.
Does security software make PCs unsecure?
Maik Morgenstern, CTO AV-TEST GmbH
The test indicates that some manufacturers, through complacency, weaken their own security software due to potential security gaps. But using this as a reason to forgo all protection would result in even graver consequences.
The findings confirm statements by some experts who maintain that antivirus software itself has potential security gaps. However, these experts often go on to say that one might as well forgo the use of any security software. That, of course, is pure nonsense.
Naturally, some manufacturers are open to criticism as to why they are not capable of using the most basic protection mechanism such as ASLR & DEP on all their product files. Above all, companies such as Comodo, Emsisoft, Avast, McAfee, ThreatTrack Quick Heal, K7, Ahnlab and Seqrite. Using these protections is really simple, and as one can see in the test, competitors can indeed use them as well. Many of the above-mentioned companies also partly go down in flames when it comes to the second basic protection – the signing of files. It is all the more baffling that only certain files are not signed or are signed with an invalid certificate; the majority of them are in fact signed. One might also simply call it complacency. The fact that many manufacturers also offer their test versions via unsecured HTTP protocol is the icing on the cake, handing critics even more arguments against security software.
As constant tests in terms of protection effectiveness prove, the manufacturers have a mastery of their actual job. The security packages work well, and most of them have a high detection rate, only missing an extremely low percentage of threats. But that's not enough. The term ‘security package‘ does not just stand for offering many modules within the product. Some manufacturers still do not comprehend that a package has to be well-rounded overall: the best protection for the user on top of a flawlessly-protected base. And this already begins with the download of the test version from a secure server. Only Bitdefender, ESET and Kaspersky Lab appear to have understood the concept with the package.