Since there is no such thing as the perfect protective program nor the error-free user, infections of computer systems will occur time after time. This scenario is tested in the field of repair performance. Until the end of 2012 AV-TEST performed these reviews every two months.
'Repair' will be covered in dedicated reviews from now on. These special tests will be performed over a longer period of time and also focus on stand-alone cleaning utilities and rescue media.
This involves selecting typical examples from all of the malware families that are relevant at the time of the test in order to use them to infect the test systems. These infections must then be removed to the largest possible extent by the protection product. Malware with so-called rootkit functions are also used in the test environment. These are programs that are able to hide their own components on the system in such a manner that, for example, files or entries in the Windows Registry can not be seen by the user. This also presents a particular challenge for protection programs and is therefore explicitly factored into our tests.
The test procedure:
The products are installed, updated and started up using standard/default settings. The protection program has complete Internet access at all times.
The protection product is temporarily deactivated so that the malware can infect the system.
AV-TEST then uses the analysis program Sunshine, which it developed itself, to check whether the infection was successful and all malicious system changes also took place.
Once the system has been restarted, the protection product is reactivated and an on-demand scan is carried out.
- Any findings are then documented and dealt with in accordance with the suggestions made by the respective protection program. The system is restarted in all cases in order to conclude the cleaning process.
Given that the detection and repair of malicious components is not always complete and reliable, Sunshine is subsequently used to produce a map of the potentially cleaned system.
- A result for the test case is then determined based on the documented detection and cleaning according to the protection program and the system status recorded by Sunshine. This involves differentiating between active malicious elements (executable program files), harmful system changes (changes to authorisations in the registry or the redirection of websites in the hosts file) and harmless system changes (empty files, created folders).
This procedure is carried out on all tested programs and all test cases at the same time in order to ensure that all protection programs have the exact same test conditions. If a test case is no longer available or can no longer be run during the test procedure or its response varies in different protection programs (which can be clearly determined using the Sunshine analyses), the test case is deleted. This ensures that all products were tested in the exact same test scenarios.
All test cases only derive from internal AV-TEST sources and are always fully analysed by AV-TEST. We never resort to using test cases or analyses provided by manufacturers or other external sources.