Test Modules under Windows

Protection

How well does a security product actually protect against real threats? Instead of being theoretically evaluated in simulated test conditions, at AV-TEST, products are required to fend off the latest attacks. All the weapons also launched by cyber criminals are used in the meticulously designed tests: e.g. zero day malware, drive-by attacks, downloads from websites, attacks via infected emails and many more.

Stage 1 – Test of the protection function: protection against 0-day malware attacks from the Internet, inclusive of web and e-mail threats (real-world testing)

This test module reflects the real threat status confronting security programs in the Internet. Accordingly, the products are required to defend against online attacks by the latest malware and the pathways of infection used most often. The sophisticated test system encompasses protection effectiveness in a real-world test involving several stages. The test exclusively involves malware, which AV-TEST has discovered no later than within the past 24 hours.

Real-world test routine

  1. Whereas the products for home users are installed with default settings, the manufacturer is able to specify the configuration of corporate solutions. The products are updated and have complete Internet access at all times.

  2. An AV-TEST analysis program produces a map of the non-infected system.

  3. Then a website or email infected with malware code is called up.

  4. If a protection program indicates and blocks dangerous access attempts, this is documented. In the process, every possible security function is covered, regardless of where the function blocks the attack or the technology with which an attack is blocked. In defending against malware via launched websites, this is done as follows, for example:

    • Access to the URL is blocked.
    • The exploit on the website is identified and blocked.
    • Download of malicious components is blocked.
    • Use of malicious components is blocked.
  5. As the detection of malicious components or actions is not necessarily synonymous with the successful blocking of malware, all operations are monitored at all times on the test system. This enables us to determine whether attacks are blocked completely, partly, or not at all.

AV-TEST creates identical and reproducible conditions for all the products within a test. For this purpose, the test routine for all products and for each test case is carried out simultaneously, and identically configured test systems are deployed.

Stage 2 – Test of the detection function: detection of widespread and prevalent malware discovered in the last 4 weeks (the AV-TEST reference set)

The AV-TEST reference set consists of extremely widespread malicious Windows programs discovered by AV-TEST during the tests and in the last 2 weeks prior to the beginning of the test.

The reason for this test approach is that infections frequently occur because products do not receive updates, are not properly configured or are not yet capable of protecting against attacks. Against this background, tests are made in terms of how well products are capable of hunting down malware that is only a few days old and thus detecting potential infections in the network or on the home PC.

Test routine for the AV-TEST reference set

  1. Whereas the products for home users are installed with default settings, the manufacturer is able to specify the configuration of corporate solutions. The products are updated and have complete Internet access at all times.

  2. An on-demand scan occurs via the AV-TEST reference set.

  3. All files not detected in the on-demand scan are executed on the test system in order to test the dynamic detection.

AV-TEST creates identical and reproducible conditions for all the products within a test. For this purpose, the test routine for all products and for each test case is carried out simultaneously, and identically configured test systems are deployed.

Subscribe to the AV-TEST Newsletter

Well-informed
on security

More ›