November 11, 2013 | More tests
  • Share:

Four Patch Management Solutions Are Put to the Test

Patch management plays a major role in ensuring security on corporate computers, on which software that has been badly updated or not updated at all provides an immediate gateway for attackers and malware. AV-TEST tested four patch management solutions between May and July 2013 in order to ascertain how successful these solutions are in increasing system security.

Patch management solutions put to the test

The Kaspersky Lab solution beat the other test participants in a number of areas, one of which was the number of applications supported. The Kaspersky product also achieved the best values in the overall results compiled from the scores recorded in all stages of the test.

zoom

Patch management solutions for enterprise environments ensure that all managed systems are analysed, checked for weaknesses and kept up to date in terms of updates. They synchronise existing operating systems and software with the databases of the suppliers of the patch management software in question and activate the required updates. At least, that's what they do in theory. In practice, however, the success of these solutions depends on how quickly they install updates, whether they are able to recognise running applications and whether the updates installed can be run without any errors. After all, most malware uses the weaknesses that can be found in software as a means of invading systems or networks.

The number of applications supported by the patch management solution

The Kaspersky Lab solution has the largest program portfolio with a total of nearly 230 applications. The number of programs recognised by Lumension, on the other hand, is less than 90.

zoom ico
The average duration recorded for updates in days

While the solutions from Kaspersky Lab and VMware take four days to offer updates, the Symantec solution takes five days and the Lumension solution requires nearly 12 days.

zoom ico
Live updates on clients in cases involving interfering factors

A lot of solutions experience problems when urgent updates need to be installed on clients during normal operation. Running installations, open browsers, a lack of Internet connection or applications that need to be patched but are currently running all prevent the successful installation of patches by the Lumension, VMware and Symantec solutions.

zoom ico
Updating software with preset foreign languages

A patch solution also needs to recognise an application's preset language in order to install updates. The solution from Lumension barely recognises any foreign languages. Although both Symantec and VMware are able to successfully install patches, a lot of programs tested displayed different language settings after the patches had been installed.

zoom ico

1

The number of applications supported by the patch management solution

2

The average duration recorded for updates in days

3

Live updates on clients in cases involving interfering factors

4

Updating software with preset foreign languages

The Test Environment

Although the company Kaspersky Lab commissioned AV-TEST to carry out the patch management solution test, it only stipulated the scope of the test, leaving AV-TEST free to define the final test methods used and to carry out the test in its own laboratory. The test results of the products from Kaspersky Lab, Lumension, VMware and Symantec have now been published without any influence from Kaspersky Lab.
The patch management solution test was carried out on the following products:

• Kaspersky Security Center 10.1
• Lumension Endpoint Management & Security Suite 7.3
• Symantec Altiris Patch Management Solution 7.1
• VMware vCenter Protect 8.0 

You can access the English version of the technical test report, which specifies all of the individual test parameters used in the patch management solution test and contains a list of the applications tested, at: avtest_2013-07_patch_management_english.pdf.

The Test Hurdles

In Microsoft systems, weaknesses in operating systems can be eliminated relatively quickly by using the Microsoft database WSUS, provided that the patch solution is aware of this tool and makes use of it. In the case of other operating systems such as Mac-OS X, Red Hat or SUSE Linux, the databases provided by the patch solution suppliers are needed in order to eliminate weaknesses.

The number of applications used by companies continues to be virtually unlimited. The patch management solutions tested by AV-TEST were required to recognise and support a total of over 290 programs and operating systems based on the overall number of applications supported by the four solutions.

The best solution in this stage of the test was the Kaspersky solution, which recognised and supported over 78 percent of the programs. The solution from VMware came in second with a total of 65 percent followed by Symantec with 48 percent and Lumension with close to 27 percent.

Only the solutions from Kaspersky and Lumension were able to fully recognise all of the operating systems.

Recognising Is Good, Identifying Is Better

The test then went on to check whether the patch solutions were able to recognise and update the 60 most popular applications, which included browsers such as Firefox, Chrome, Safari and Opera, e-mail clients such as Thunderbird and archivers such as WinRAR and 7-Zip.

The results of the study are sobering: The results of this stage of the test were rather mixed, with the solutions from Lumension, VMware and Symantec proving that they recognised most of the programs but misreading or completely failing to read their version numbers.

Lumension does not even support browsers such as Chrome, Opera or Safari, does not recognise Thunderbird and is unable to make head or tail of WinRAR.

The solution from Kaspersky, on the other hand, also impressed in this stage of the test, in which solutions that could correctly identify both an application and its version number were also able to ensure that the correct update was installed.

Changing the Language to Enable Updates

Some solutions refuse to fully support applications when they are not installed in the system language. The solution from Lumension was particularly problematic where this issue was concerned and was only able to recognise two additional languages. In comparison, the solutions from Symantec and Kaspersky Lab were able to recognise 12 of the 14 languages tested, while VMware even managed to recognise 13. Nevertheless, although VMware and Symantec were indeed able to bring six and seven programs respectively fully up to date, they both needed to change the preset language in order to do so.

How long does the patching take?

Once an application has reported a weakness, a patch is published just a short time later. In the test carried out by AV-TEST, the solutions from Kaspersky Lab and VMware took an average of four days to respond to such weakness reports, while the solution from Symantec took five days. Lumension, on the other hand, needed nearly 12 days to respond.

Which solutions provide reliable patching and which solutions fall short?

The most important aspect of a patch management solution is the patching process itself. After all, every update installation that fails, leads to errors or, in the worst-case scenario, ends up in an infinite loop, has to be manually dealt with by the administrator.

In the AV-TEST patch management solution test, the solutions only had to update the applications from the test pool that they actually recognised. The results gathered in this stage of the test show how successful the solutions were in updating these applications. The patch solution from Kaspersky Lab, for example, was able to update all of a total of 51 recognised programs, achieving a rate of 100%. Lumension achieved a total of nearly 97 percent but only recognised 27 of the applications in the test pool, while VMware and Symantec supported an impressive total of 63 programs each, but only patched 97 and 87 percent of these programs respectively. Eight of the programs patched by the Symantec solution froze after the update was installed. The same problem also occurred with one program when using the patch management solution from VMware.

Stumbling Blocks when Installing Updates

If an update has to be made immediately on clients, it is often the case that some solutions conflict with certain system situations such as running installations, open browsers, a lack of Internet connection or the case that the application that needs to be patched is currently running. Such situations were re-enacted in this stage of the test.

Only Kaspersky Lab and Lumension were able to achieve positive results when installing updates for the applications supported by the patch management solutions. The solution from Lumension only became problematic in three cases because the software that it wanted to patch was running at the same time.

The solutions from VMware and Symantec, on the other hand, experienced a number of problems in all of the conflict scenarios, particularly where open browsers and active software that needed to be patched were concerned.

Security Vulnerabilities Caused by Add-ons

A large number of applications attempt to add features such as further toolbars or optimisation tools to systems during installations. If users do not pay careful attention, these potential security vulnerabilities can also be installed together with updates. In the test carried out by AV-TEST, both the Symantec and VMware solutions were not able to avoid the installation of individual add-ons on the system together with updates. Only the solutions from Lumension and Kaspersky Lab remained free of errors.

Automatic Updates Can Throw a Spanner in the Works

A large number of applications are well aware that they are constantly susceptible to security vulnerabilities. Adobe Reader leads the field where this matter is concerned and therefore also comes with an automatic update function. Patch management solutions should be able to deactivate applications' auto-update function, but in the test, only the solutions from Kaspersky Lab, Lumension and VMware came with this option as standard. Users of the Symantec solution, on the other hand, have to deactivate the function themselves using script control. The solutions from Lumension and VMware however only supported a small number of applications directly, whereas the solution from VMware at least also offers users the additional option of deactivating the function via script.

User Licences Should Be Accessible

Every time an administrator installs software, they should at least have the option of reading the software's terms of use and rejecting them if necessary. Only the solutions from Lumension and Kaspersky Lab, however, offer administrators this option, with the solutions from Symantec and VMware choosing not to display terms of use.

Update Support from Microsoft

When installing updates, users of Microsoft systems can use the Windows Software Update Service, known as WSUS for short. While Kaspersky Lab has chosen to incorporate this system into and have it managed by its solution, all of the other solutions tested contain no links to WSUS.

Summary: Patch Management Secures Systems and Makes Life Easier for Administrators

Most of the malware currently threatening computer systems mercilessly exploits weaknesses found in software and operating systems. Where patch management is concerned, the test carried out by AV-TEST proves that the patch management solution from Kaspersky Lab is currently leading this field. When compared with the other solutions tested, the patch solution from Kaspersky Lab supported the largest number of applications and achieved excellent, and sometimes even the best, results in all stages of the test.

Although the patch management solution from VMware performed well in the test, all of its results were a lot lower than those achieved by the Kaspersky Lab solution. In the overall test results, the solution from VMware is closely followed by the solutions from Symantec and Lumension, both of which are almost on the same level. Lumension recognises a conspicuously small number of applications and is too slow in terms of its update reaction. Although the solution from Symantec boasts a quicker update reaction, it is prone to change language when installing updates and also has problems when updating active systems.

You can find an overview of the test methods used by AV-TEST and a list of the applications with weaknesses involved in the test in the technical test report, which can be accessed at: avtest_2013-07_patch_management_english.pdf.

Social Media

We want to stay in touch with you! Now there is an easy way to receive regular updates on the latest news and test releases.