17 software packages in a repair performance test after malware attacks

The Test Procedure

In this endurance test, the software solutions were tested multiple times over a period of 10 months, from September 2013 to June 2014 in terms of their clean-up and repair performance. Throughout the entire period, various malware families were repeatedly tested, as they also undergo constant evolution.

The malware samples used were already known to all the solutions and thus should have been clearly detected by all of them. In the test tables, the clean-up and repair quality was recorded according to the following priority:

1. Was the malware detected or not?
2. Were the active components completely removed?
3. Did any harmless file remnants remain, and were all the changes to the system reversed?
4. Did the security and clean-up software perfectly remove and restore everything?

As a result, the table also includes a percentage rating of the overall clean-up and repair performance.

Test of the Antivirus Software

For antivirus solutions, the test was divided up into two typical infection scenarios.

1. On a system already infected with malware, the protection software was installed and the subsequent detection, clean-up and repair of the damage was logged.

2. The protection packages were briefly deactivated, the malware was loaded, and protection was reactivated. Here the detection, clean-up and repair were logged again.

This two-part scenario was intended to simulate what happens if a user installs the software retroactively and if an installed security package only detects malware after a certain period of time and then attempts to remedy the damage. In each step, the system watchdogs had to deal with 30 malware samples.

Test of the Recovery and Repair Tools

The recovery tools are generally only deployed after a malware infection has occurred. Which was exactly the scenario examined in the test. Each of the 7 tools had to remove 55 intrusions of malware samples and their damage.

These threats were known to the tools as well; after all, the object was to examine the repair performance and not the detection ability.

The Test Platform and Environment

This endurance test – as with all others conducted by AV-TEST – was performed exclusively on real hardware, this time under Windows 7, however. The rationale: some malware samples can detect whether they are in a virtual environment. In this case, there was a possibility that some of the malware code would not fully execute. With the use of real hardware, the scenario is as realistic as the user's everyday environment.

The lab test was extremely comprehensive, as each platform had to be infected individually with a malware sample. Afterwards, the protection packages were installed or reactivated, and clean-up tools were launched respectively. Finally, the cleaned system always had to be compared to the original system. For a total of 60 and 55 malware samples respectively and 10 security packages, plus 7 tools, this added up to 985 individual tests over a period of 10 months.

Evaluating Results

For both test groups, security packages and recovery tools, the individual test results were entered into the table according to the following descending risk schema:

1. Malware not detected
2. Active malware component not removed
3. Only harmless file remnants left behind
4. Complete removal, clean system

Result 1: Malware not detected

The result in this category actually should have been 0 for all the test candidates, because it was established prior to the test that each malware sample was known to the solution.

Among the security packages, known malware was not detected as such by Microsoft Security Essentials and Avira Free Antivirus. That actually should not have happened.

The seven clean-up tools revealed no errors in basic detection.

Result 2: Active malware components not removed

In this case, the malware was detected and some files were removed, but the core of the malware was not rendered harmless. Thus, the system remains infected.

Among the 10 security packages, the freeware solutions from Avira, Avast and AVG, as well as Microsoft's Security Essentials, had several problems. They failed to remove active components in the system between 1 and 7 times. The additional paid software products and the freeware Malwarebytes Anti-Malware Free cleaned error-free in this case.

Among the 7 clean-up tools, more than half the participants also failed on this key task: Hitman Pro, Panda Cloud Cleaner, Avira Cleaner and the F-Secure Removal Tool were unable to remove 1 to 11 active components.

Only the Kaspersky Removal Tool, the Norton Power Eraser and Disinfect2013 from Heise performed reliably in this category.

Result 3: Harmless file remnants left behind

This segment documented the harmless code remnants left behind in the aftermath of malware attack clean-up. These remnants mainly included additional ineffective files or orphaned entries in the Windows registry.

The security packages from Bitdefender, F-Secure, Kaspersky, Norton and ESET only left small file remnants and insignificant entries in the registry between 1 and 9 times. In terms of quality, this put them just slightly below the perfect score attained by Malwarebytes Anti-Malware Free.

Among the clean-up tools, the Kaspersky Removal Tool only missed removal of a harmless file in one instance. It performed all other tasks with flying colors.

Norton Power Eraser fell short of a perfect clean-up 11 times, and Disinfect2013 from Heise even missed it 48 times. All it left behind, however, was digital garbage without any dangerous components.

Result 4: Complete removal, clean system

Of all the security packages, only the Malwarebytes Anti-Malware Free utility left the system completely cleaned and repaired after 60 tests.

The package from Bitdefender was able to do this 59 times, whereas the packages from F-Secure and Kaspersky were each successful 56 times. The two security solutions merely overlooked harmless registry entries. Otherwise they achieved an almost perfect performance.

Among the clean-up tools, none of the tools was able to render the infected test systems completely clean. Only the Kaspersky Removal Tool achieved top performance. It would have left the system totally error-free except for one overlooked, yet harmless, file remnant.

Summary: There is software for the morning after

The test results disproved a statement frequently posted in forums, that all you can do is delete your Windows system if it becomes infected with malware.

Among the security suites, the solutions from Malwarebytes, Bitdefender and Kaspersky demonstrated the best performance among all the packages tested. All in all, however, the rest of the field still showed quite solid performance, even though a few active components were left behind.

Among the freeware clean-up tools, the Kaspersky Removal Tool is worth recommending. While the Norton tool, as well as Disinfect2013 from Heise, left behind quite a lot of data garbage, neither failed to detect a single active malware component.

In final analysis: In case an existing security suite ever fails, there are reliable rescue options for having a Windows system cleaned and repaired. The best part of all: in case of emergency, most of the tools can even be used free of Charge.