Visualizing Nessus Working Harder For You

Recently, several images were uploaded to the SecViz - Security Visualization web site which visualize how hard the Nessus, Saint and Retina vulnerability scanners actually work. Default scans for each scanner were performed in full view of a Snort sensor and the alerts from Snort were sent to Prelude for visualization with "pig". The visualization allows understanding of how many different and unique techniques are performed by each scanner. Below are screen shots for the results from each scanner:

Saint Results	Retina Results	Nessus Results

When I first saw these results, I didn't think they were entirely relevant. The visualization is using Snort events, which means that all of the scanners might be trying techniques that Snort might not detect. For example, when Nessus performs a variety of non-credentialed Windows checks over ports 445 and various Windows RPC services, Snort generates some events, but it does not generate a unique event for every custom probe. However, after the author of these posts to SecViz contacted me and pointed out some of the test results, I thought it was a good blog topic. The raw results for Nessus included 1019 alerts, 166 alerts for Saint and 76 alerts for Retina which was fairly significant.

Since I didn't perform this test and did not duplicate it, I am not asking readers to draw any specific conclusions from these numbers. However, I strongly hope that readers perform their own tests of this nature to gain an understanding of how hard your assessment technology is working.

If you have multiple scanner technologies and various methods (NIDS, HIDS, SIM, NBAD, .etc) of detecting these probes, then doing comparison scans of the same targets can have interesting results. You may find that your default scanning policies aren't aggressive enough. Looking at how scanning can generate IDS events, netflow, logs and other types of data from scanner to scanner can help you understand what the scanner is doing and when. Keep in mind that if you are performing credentialed audits, ideally your security monitoring shouldn't see much other than an agent's processes working or remote logins via the Windows domain, Secure Shell or SNMP.

To illustrate this concept, I performed a scan of my target network with a remote Nessus scan and a free Qualys scan. Below are screen shots of from the Security Center and Log Correlation Engine I use for testing. The targets had http, https, ftp, vnc, ssh and several other protocols open during the scan. The results were both "zoomed" to the start and stop time of the scans.

Nessus Results	Qualys Results

If you are not familiar with the LCE, each log, flow or event gets a name. Something like an Apache 404 error log will be normalized as an "Apache-404_Error" name. The graphs above show a time line from left to right. So for the Nessus scan, there are some initial surges in events of SnortET-Scanning (a normalized name for Emerging Threats Snort port scan rules), a Snort-TCP_Scan event and a bunch of NetGear-Blocked_TCP events. What follows are a variety of Snort and system logs that get mapped into a variety of normalized web events. The Nessus scan also had 765 VNC login and logoff events.

Being able to take these types of activity summaries and compare them is very useful. For example, the Qualys scan performed far many fewer VNC logins than the Nessus scan did. Does this mean the user running the scans configured the scans incorrectly, or does this mean there were less actual checks being performed?

Lastly, performing this type of analysis can not only help you understand how hard your scanning technology is working, but you can make sure that your logging infrastructure is also working. When I started this blog entry, I realized I was getting Snort port 80 web events, but the actual logs from my web servers were not being sent to my Log Correlation Engine. I had reverted a target virtual machine which was sending syslogs to the wrong Log Correlation Engine.

Conclusions and Recommendations

I'm sure some readers will interpret this blog as slight against non-Nessus vulnerability scanners, but the point I'm really trying to make is that if you look at the effects of a scan through some sort of network monitoring solution, you may be able to learn not only how your scanner works, but how it interacts with your network.

This technique can be used for several different tasks:

• If you are evaluating a vulnerability scanning technology that is not leveraging credentials, looking at the logs generated by your SIM, NBAD or NIDS might help you see some differences in the scanner's performance for your environment.
• If you are evaluating an MSP scanner offering (regardless if it is PCI certified or not), using your NIDS, SIM or NBAD solution can show how these services might be performing different types of checks. I've seen a great deal of variety in the techniques and implementations of MSPs that use Nessus for their scanning and they all do things much different and unique than I would have thought.
• If you are evaluating a SIM or NBAD solutions, the solution should be able to look at all of the data generated by a scan and show very detailed analysis of how the scan took place and what steps it went through.