A peek inside antivirus’ cloud features

As an information security firm, we at AMOSSYS, are interested in understanding how antivirus software work. In recent studies, we’ve noticed that “cloud-related” viral scans become a feature more and more publicized. However, if you take few minutes to browse vendors’ websites you can see that antivirus solutions seem to use the cloud, but you don’t really know how and why. This article exposes some of the observations we made while digging into this trend.

Objectives of this analysis

Cloud related features are not well documented by antivirus vendors. If you want to find a beginning of useful information, you have to dig further into the web. As described in a Kaspersky whitepaper, the cloud seems to be used to provide a better detection rate and a faster execution. Similarly, Norton website refers to the cloud as a way to detect new emerging threats. In that context, we decided to focus this study on these three specific questions:

  • Do antiviruses have better detection rate and faster execution when they are connected?
  • Are the antiviruses’ behaviors changing when you connect your system to the internet?
  • What kind of information your antivirus software is sending to the internet?

In this article we try to answer these specific questions. As you’ll read, we did some pretty simple tests, but we do have several results which are already quite surprising.

Environment description

To conduct this study, we took the following antiviruses in their last version (i.e. 2015):

  • Avast Internet Security,
  • BitDefender Internet Security,
  • Kaspersky Internet Security,
  • Norton Internet Security.

All tests were done on fully patched Microsoft Windows 8.1 64 bits and with 2 GB of RAM. Considering the network, an internet connection with a download speed of 11 MB per seconds and an upload speed of one Mbps was used.

In order to study differences between a connected and a disconnected solution, we’ve setup the following environment: 2 virtualized systems are connected to a gateway that filters accesses to the internet. This architecture lets both machines access the internal network, and lets us restrict the internet access to the disconnected VM. We need this kind of architecture regarding some tests (for example, we want to detect if an internal connect back is really happening).

Platform description

Once our virtual machines were setup, we installed our four antivirus solutions. For each of these, we selected all possible options to tell the software we want to send information to the cloud. And the first observation here was that some solutions give more details about what they send to the cloud during the installation phase than in their online documentation… It is the case for example of Bitdefender, which explains at the installation that they use the cloud to give you a “behavioral protection against e-threats” (as stated in French on the figure below). We tried to find such information on the vendor’s website (because we really would like to know more about such a feature) but with no result.

Bitdefender against e-threats

From the different installation phases, we at least learned that all antiviruses send what they call “analysis reports” to their servers. What these reports contain is however a bit vague. Once the installation phases were finished, we left our four antiviruses with their default configuration settings regarding their protection level.

Tests descriptions

Comparing antiviruses’ efficiency is a quite difficult task. What we did in this work is easier: we wanted to find some antiviruses that have weird behavior when connected / disconnected. In order to do this, we simply ran 2 tests that are described below.

Speed and detection test

In this first test, both VMs make a static scan of a directory that contains a collection of virus samples (around 2000 samples) that we downloaded from the VxHeaven site. The fact that viruses are well known and/or outdated should not be a big matter since the goal of this test is to answer these two specific questions:

  • Do the two VMs detect the same number of threats?
  • Do the two VMs run their scans at a different speed?

The results that are summed up in Table 1 below, are quite surprising:

  • For Bitdefender, both VM did not detect the same number of threats, with the connected VM executed this task a bit faster than the non-connected one. What is a bit controversial here is that this speed up is at the expense of the detection ratio since the connected VM detected less threats than the non-connected one. This decrease is however quite limited since there is “only” 30 threats of difference (on a total of 2000).
  • Considering Norton, it is surprising to see that a connected Norton does not find the same number of threats on successive experiments. In addition, what’s even more surprising is that Norton loses speed when connected to the internet… Some deep reverse engineering would be necessary to understand this specific behavior. However we didn’t focused in this study on the reasons and decided to stick with these results.
  • For Kaspersky, both VM detected the same number of threats. However the connected VM did it in 20 minutes whereas the non-connected one did it in 10 minutes. From that observation, it is important to keep in mind that Kaspersky’s detection rates cannot really be compared when scanning a VxHeaven database. Indeed, as VxHeaven guys explained here, Kaspersky is doing a nearby 100% ratio when scanning such collections, and we can all understand that they apparently use the VxHeaven database to make some tests. But in this case, Kaspersky is not that much of a winner: a connected solution takes 2 times more than a disconnected one without finding any new threat (that might be a problem when you detect 100% of threats).
  • Finally, Avast results are exactly the same with a connected and a disconnected solution. When listening the network, Avast doesn’t seem to send any information to their servers when running a local scan. This would explain that the connected VM has the same results than the non-connected one.

Table 1: Speed and detection results
Antivirus Online Virtual Machine Offline Virtual Machine
Bitdefender Speed: around 15 min
Nb threats: 2034
Speed: around 20 min
Nb threats: 2062
Norton Speed: around 20 min
Nb threats: between 850 and 870
Speed: around 10 min
Nb threats: 862
Kaspersky Speed: around 20 min
Nb threats: 2083
Speed: around 10 min
Nb threats: 2083
Avast Speed: 2 min
Nb threats: 1120
Speed: 2 min
Nb threats: 1120

Important remark regarding this table: these results are not here to say which antivirus is better than the other. If we wanted to do this, we would certainly not have taken a public database known by every security companies. The interesting facts we wanted to highlight here, are the differences between connected and non-connected solutions.

In conclusion to this test, we can see that three out of four antiviruses use the cloud when running static scans. On these three antiviruses, only one is faster when connected to the internet, but this is at the expense of the detection ratio which is a bit lower. The other two antiviruses are running their scans slower when online. However, it’s important to understand that this test cannot be considered as an exhaustive one: we rely for this test on a public and well-known virus database.

Unknown threat test

In this second test, the connected VM scans at several moments a custom reverse TCP meterpreter we’ve built, to see if the threat is detected at some time. Every antivirus solutions should detect a reverse TCP meterpreter, but it’s not difficult to find techniques or tools to circumvent these kind of detection. In order to have two different samples with a small behavioral difference, we have built 2 versions of this binary: one is sleeping 15 seconds before executing (called Sample 2) and the other not (called Sample 1). Once binaries have been compiled and we’ve checked they aren’t detected by our 4 antiviruses at a time T, we scan it and execute it once every day. If one antivirus solution detects the threat at a time T+1, we can make the reasonable assumption that the solution has sent the binary to the cloud and that this latter analyzed it in order to detect it as a new threat. The results of these tests are summed up in Table 2.

Table 2: Unknown threat test results
Antivirus Initial time T Time T+3 (3 months later)
Bitdefender Sample 1 not detected
Sample 2 not detected
Sample 1 not detected
Sample 2 not detected
Norton Sample 1 not detected
Sample 2 not detected
Sample 1 not detected
Sample 2 not detected
Kaspersky Sample 1 not detected
Sample 2 not detected
Sample 1 not detected
Sample 2 not detected
Avast Sample 1 not detected
Sample 2 not detected
Sample 1 detected
Sample 2 detected

We’ll explain in next paragraph why Norton is differentiated (in grey) from the others. Let’s finish first with the results from Table 2. What we see is that only Avast has apparently learned that Sample 1 and 2 are malicious. We iterated this test on a period of three months (for real, Avast has taken less than a week to detect this). On four antiviruses, only one has apparently analyzed the unknown binary and added it to their knowledge database.

Now let’s go back to our Norton results: Norton is kind of different from the other antiviruses because it is indeed detecting an unusual network connection when both binaries are executed. The following popup (sorry, still in French) appears when we execute samples 1 and 2. It says: “A suspicious activity has been detected on the network”.

Suspected activity detected by Norton

A real time detection is therefore done by Norton: the user is asked if he really wants to execute this binary. However, even after this “incident”, a static scan is comforting the user with a message saying that this binary is safe…

But what’s even more interesting is that when analyzing the traffic during this test, we noticed that there were specific TLS connections made just before the popup appear. The TLS traffic seems to be initiated by the antivirus software and connects to a server within the network, which is owned by Symantec. Rehearsing the test with a disconnected VM lets us confirmed our hypothesis: our meterpreter successfully connected back, meaning that Norton apparently asks the cloud if it has to block the binary. Several attacks can be deduced from this remark:

  • If you successfully man-in-the-middle the antivirus, you may be allowed to change the response from the cloud. However, this doesn’t seem to be easily done since Norton checks the SSL certificate it receives.
  • If you have the opportunity to disable the connectivity (either by DoS, spoofing, etc.) of the solution, you may be able to bypass the decision from the cloud.

So what we see here is that a connected Norton has a different behavior than a disconnected one. In other words: a connected Norton will detect in real time an unknown threat, whereas a disconnected one will be pwned…

In conclusion to this test, we can see that out of four antiviruses, only one has apparently send the binary to its servers to analyze it. (We aren’t absolutely sure that the binary was send to the cloud… To confirm this hypothesis, we would need to understand what antivirus’ module is responsible of that behavior and reverse engineer it.) The other antiviruses still don’t detect anything harmful in our custom meterpreter after 3 months. What we’ve seen moreover is that one of these 4 antiviruses has a different behavior when connected to the internet. When connected, the software asks the cloud to know if some behavior could be harmful. This can lead to a very specific attack where an attacker temporary disconnect a system in order to execute its malware on the target. Again, this test cannot be considered as exhaustive, however it lets us understand that differences can exist between a connected and a disconnected solution.

Final thoughts

At first when we began this study, we mainly wanted to understand what antivirus software use the cloud for. Some vendors say they would execute analyses faster, some say they would detect threats better, and some say they would protect you from unknown malware… What we showed with these 2 simple tests is that once again with antivirus software, results can be full of surprises. From our results, some antiviruses will detect less threats when connected, some will execute slower when connected, and some will act differently in real time when connected! Once again, these tests are however non-exhaustive and we will have to be more specific and precise if we wanted to conduct a real study on what vendors call these “cloud features”… Hopefully, if we keep on having interesting results regarding this work, we’ll continue to keep you updated.