Self-Protection in Windows Applications: How Secure are PDF Readers, Java and Browsers?

Just under 20 popular applications in the lab check

AV-TEST recently examined the latest security suites repeatedly in terms of their self-protection and once again revealed many vulnerabilities. In the second major test, the lab examined whether the Windows applications deploy the technologies for self-protection. This involved examining popular PDF readers, browsers, office programs, file archivers, graphics software and Java versions. Wherever available, each application was examined in the 32-bit and 64-bit version under a 32-bit and 64-bit Windows system:

- PDF readers: Adobe Reader, Foxit Reader, PDF-X-Change Editor
- Browsers: Chrome, Firefox, Opera
- Office packages: Free Office, Libre Office, Microsoft Office, Open Office, WPS Office
- File archivers: 7Zip, WinRAR
- Java: the latest updates of Version 5 to 8
- Graphics software: ACDSee, GIMP, IrfanView, Paint.Net

In the test, only the user-mode PE (portable executable) files for 32- & 64-bit versions were placed under scrutiny. All the other files were irrelevant for the test. The PE files include, for example:

.exe or any "executable" program or module
.dll or "dynamic link library", a program library
.sys or "system" software
.drv or "driver", a driver file for a device

In addition, all versions were examined in terms of whether the PE files are signed with a valid certificate.

Important: On the particular 32-and 64-bit applications, the lab always examined all PE-files, regardless of whether they work with 32 or 64 bits. Because even in the 64-bit version, files also work in the background with 32-bits. The converse naturally applies as well.

Also important is the fact that several software providers equip individual files with special protection that is not compatible with DEP & ASLR. That is why they cannot reach 100% in the tests.

The self-protection could be better

Among the most well-known applications with negative headlines are the Adobe Reader and Java. If we look at the evaluation table, however, these applications are almost model citizens in the use of DEP & ASLR. This trend is clearly visible especially in the Java versions. Whereas up to the old Version 6, the technology was not even used, now already as of Version 7, it is used seamlessly up to the Version 8 Update 60 available in the test.

For the major browsers, there is also widespread, but certainly not full-range, deployment of DEP & ASLR. While Firefox achieves a 100% deployment rate in Version 40.0.3, in Version 41.0.2, the average drops below 90%. The Opera browser perfectly utilizes DEP & ASLR in connection with the 32-bit files. The one existing 64-bit file is in turn ignored. Unfortunately, the Internet Explorer is not on the list. This is because it is so deeply embedded into the Windows operating system, the files used cannot be clearly isolated. An assessment would thus only be very vague and is therefore rejected by the laboratory.

For the office applications, the use could not be more heterogeneous. Microsoft Office 2016 relies almost entirely on the protection, WPS Office uses the additional protection above 80%, Libre Office uses it partially, but Free Office doesn't use it at all: 0 percent. Open Office only protects its 32-bit files. For the 64-bit files, the deployment is zero.

For the frequently-used open source software packages 7Zip and GIMP, disastrously, not a single file is additionally protected. But other software packages like ACDSee Ultimate or IrfanView do not use DEP & ASLR consistently either.

Files without a signature can be a risk

The next phase of the test examined whether all PE files have a signature along with a valid certificate. Along with an additional hash value, these help towards securely identifying a file. If a file is not equipped with this information, a security suite cannot precisely determine whether the file exists as an original or whether it is a manipulated file. This often results in error messages to the user, or even blocked applications for security reasons. Security solutions do have additional tools for checking the file, such as a sandbox, but that's additional time and effort and wasted performance, which does not have to be.

After all, applications such as Adobe Reader have often been harnessed to break into PCs. For this purpose, cyber-criminals will seek out any vulnerability and exploit it. Unsigned files are also considered by experts to be potential vulnerabilities. In this context, especially in the case of the Adobe Reader, it is baffling as to why only the 64-bit files are cleanly signed, whereas up to 50% of the 32-bit files are not. The files of PDF-Exchange are only completely signed in the 32-bit version. For the Foxit Reader, on the other hand, everything is in perfect order.

For the browsers, only Chrome and Opera had no unsigned files whatsoever. For Firefox, there are still individual unsigned files. The Internet Explorer was not included in the testing for reasons already mentioned under DEP & ASLR.

Microsoft Office with an expired certificate

In the segment involving Office applications, Free Office does not sign any files and Open Office hardly any. WPS Office and Libre Office have 5 to 10 percent unsigned files on board. For Microsoft Office 2016, while there were no unsigned files, one of the files in fact had an invalid certificate. This circumstance is all the more reprehensible, because after all, the Authenticode Code Signing technology comes from Microsoft, and the manufacturer should set an example with all its products. An inquiry submitted to Microsoft remained unanswered at the time this article was published.

For the file archiving applications, the situation in terms of signatures is almost identical to the use of DEP & ASLR: For 7Zip, not one single file is signed, for WinRAR only a few files.

Java is also frequently the target of attackers, as it is deployed on many devices. Only in Version 7 were all of the files cleanly signed. In Version 6 and 8, there continue to be individual unsigned 32- and 64-bit files.

Conspicuous open source software

The frequently-recommended graphics application GIMP is an open source software. In all its versions, roughly 50 to 70 percent of its files are unsigned. Yet other graphics programs are not exactly paragons of reliability either, although the number of unsigned files is significantly lower. Thus, there were unsigned files in all the other applications examined, such as IrfanView, Paint.Net or ACDSee. In ACDSee, even 2 files in the 64-bit version work with expired certificates.

Conclusion: Better than expected

To date, this type of testing has not been undertaken on this scale by any other laboratory. Compared to the findings with the current test of self-protection for antivirus software, and disregarding open source software, the following picture emerges: DEP & ASLR are relatively well implemented in most Windows applications. On average, even better than in the latest security software! But some manufacturers could do a better job of implementing the protection and simply sign their files. The laboratory will be repeating the test after a certain interval. Then we will see whether the manufacturers heed the call for more security.

It is somewhat conspicuous that for open source software, such as 7Zip, GIMP or Open Office, the final files are often not equipped with DEP & ASLR or file signatures. This is perhaps due to the open group of programmers and the resulting frequent files revisions. The leaders of these projects urgently need to come up with a solution, so that otherwise good software will not get a bad reputation. It ought to be easy to implement DEP & ASLR, and after all, the signatures don't cost a fortune from certificate providers.