The inadequacies inherent in current defenses have driven the development of a new breed of security products known as intrusion-prevention systems (IPS). These systems are proactive defense mechanisms designed to detect malicious packets within normal network traffic and stop intrusions dead, blocking the offending traffic automatically before it does any damage, rather than simply raising an alert as, or after, the malicious payload has been delivered.

Within the IPS marketplace, there are two main categories of product: host IPS and network IPS, with the latter being further sub-divided into content-based and rate-based (or attack mitigation) systems. This report looks at the latest crop of network IPS (NIPS) products to make it through our labs. (Note: It is impossible to cover all of the leading products in a single article of this nature due to the amount of time spent on testing each individual product.)

This review will give readers a perspective of the capabilities, maturity and suitability of the products tested. The NSS Group established this test because IPS products are being actively deployed as a new layer in defense-in-depth security architectures.

Put to the Test
As part of its extensive IPS test methodology (see www.nss.co.uk for the detailed methodology), The NSS Group subjected each product to a battery of tests that verify stability and performance, determine the accuracy of security coverage and ensure that the device will not block legitimate traffic. The NIPS test suite contains more than 800 individual tests that evaluate IPS products in three main areas: performance and reliability, security accuracy, and usability.

Detection/blocking performance under load. This group of tests verifies that the IPS does not adversely impact legitimate traffic, even when new TCP connections are being created rapidly. All tests are repeated with 250 Mbps, 500 Mbps, 750 Mbps and 1000 Mbps of background traffic (or up to the maximum rated throughput of the device in 25% increments should this be less than 1 Gbps). The test is conducted with UDP, HTTP and mixed-protocol traffic, and includes packet rates up to 1.48 million packets per second and connection rates up to 20,000 connections per second.

Latency and user response times. This group of tests determines the effect the IPS sensor has on the traffic passing through it under various load conditions. Bi-directional network latency of UDP packets is measured under three test conditions: with no load, with 500 Mbps of HTTP traffic (or half the rated load of the device if this is less than 1 Gbps), and while the device is under a heavy SYN flood attack (up to 10% of the rated throughput of the sensor).

Stability and reliability. These tests verify the stability of the IPS device under various extreme conditions. First, the external interface of the sensor is exposed to a constant stream of millions of attacks over eight hours. The device is configured to block and alert, and thus this test provides an indication of the effectiveness of both the blocking and alert handling mechanisms. The second part of the test stresses the protocol stack of the device under test by exposing it to traffic from the ISIC test tool for eight hours (both management and sensing interfaces).

Detection accuracy and breadth. This group of tests verifies that the NIPS will not block legitimate traffic (accuracy) and is capable of detecting and blocking a wide range of common exploits (breadth). This test demonstrates how accurately the IPS detects and blocks a wide range of common exploits, port scans and denial-of-service attempts. This test is repeated twice: the first run with blocking disabled on the IPS in order to determine which attacks are detected and how accurately they are detected (attack recognition rating); the second run with blocking enabled in order to determine which attacks are blocked successfully regardless of how they are detected or what alerts are raised (attack blocking rating).

Resistance to evasion techniques. These tests verify that the IPS is capable of detecting and blocking basic exploits when subjected to varying common evasion techniques. The tests consist of: packet fragmentation and stream segmentation, URL obfuscation, and miscellaneous evasion techniques.

Stateful operation. This section determines whether the IPS is capable of monitoring stateful sessions established through the device at various traffic loads, without either losing state or incorrectly inferring state. This test determines whether the sensor is capable of preserving state across increasing numbers of open connections, as well as continuing to detect and block new exploits while not blocking legitimate traffic when the state tables are filled. Various numbers of TCP sessions, from 10,000 to one million are tested.

BroadWeb NetKeeper NK-3256T V3.6.0

The BroadWeb NetKeeper series is a family of intrusion detection and prevention appliances designed to detect and prevent attacks across multiple network segments at up to 100-Mbps speeds. The NK-3256T is designed to support up to 256,000 open connections. A dedicated 1U appliance designed to monitor a single network segment at 100-Mbps speeds, the device sports two copper 10/100-Mbps ports for in-line operation, and a single 10/100-Mbps port for management.

The NK-3256T was tested up to 100 Mbps, the rated speed of the device. Performance at all levels of the load tests was impeccable, with 100% of all attacks being detected and blocked under all load conditions.

Basic latency figures were well within acceptable limits for a device of this type at all traffic loads and with all packet sizes, ranging from 80 µs with 25 Mbps of 256-byte packets, to 218 µs with 100 Mbps of 1,000-byte packets. Placing the device under a half load of 50 Mbps of HTTP traffic, acceptable increases in latency were noted, and all of the figures reported in the tests are well inside the limits expected for a 100-Mbps device. This means the NK-3256T could be situated anywhere on a 100-Mbps network, either internally or at the perimeter.

In the SYN flood test, 10 Mbps of SYN flood traffic caused minimal increases in latency, and the SYN flood was mitigated completely, thus eliminating any adverse effect on the protected network or target servers.

The NK-3256T performed consistently and reliably throughout the tests, continuing to block 100% of all exploits, even when under extended attack. Under eight hours of continuous attack (one million exploits), the device passed 99.994% of legitimate traffic, a small percentage of failures under an extreme load of continuous attack traffic.

Exposing the sensor interface to IP stack integrity checker (ISIC)-generated traffic had no adverse effect, and the device continued to detect and block all other exploits throughout and following the ISIC attack.

Signature recognition (with blocking disabled) was acceptable out of the box (71%), but was increased to 97% after the application of a signature pack update. The quality of the new signatures seemed to be as high as the existing ones, and performance of the box was not affected by the large update. Blocking performance was identical throughout the tests.

Performance in the “false negative” tests was reasonable out of the box, though still not perfect following the signature update. Following the update, resistance to false positives was also acceptable.

Evasion tests caused problems for NetKeeper initially, but a code update rectified this, following which resistance to known evasion techniques was excellent, with the NK-3256T achieving almost a clean sweep across the board in the evasion tests. Not only were the fragmented and obfuscated attacks blocked successfully, almost all of them were decoded accurately, as well, with the latest release of code.

Although NetKeeper demonstrated some initial problems in the area of false positives and resistance to evasion techniques, the engineers worked hard to rectify these during the tests. In order to take advantage of these improvements, current users should update to version 3.6.0.

The Java-based GUI is divided into separate utilities for “super users” and day-to-day administrators, and the role-based access control is effective at ensuring that access to individual NetKeeper devices is restricted on a per-user basis, with each user being allocated different privileges, as required.

The day-to-day administrative functions are not as well catered for, however. The policy-management and alert-handling capabilities are too basic, and there is no correlation capability. The management GUI allows NetKeeper to function effectively as an IPS device, but policy management and forensic analysis are more difficult than they should be due to the necessity of managing each NetKeeper device separately.

Reporting and analysis is handled much better, although there are few built-in reports. Instead, there is a flexible query facility, which allows a number of different graphical and text-based reports to be produced. Those reports can be scheduled to run at regular intervals, saved as HTML files, and copied to a secure server or e-mailed to administrators.

From any report, drilling down the summary screens to access more detailed event information is possible, eventually accessing the relevant packet contents if they were saved along with the original alert. This is a powerful facility, and helps to make up for shortcomings in the alert handling. In treating every NetKeeper device separately instead of allowing cross-device reporting, however, it does limit its usefulness in larger deployments.

Fortinet Fortigate-800

Fortinet’s FortiGate series of ASIC-accelerated antivirus firewalls are real-time network protection systems designed to detect and eliminate the most damaging, content-based threats from e-mail and Web traffic, such as viruses, worms, intrusions, spam and inappropriate Web content. In addition to application-level protection, the FortiGate systems deliver a range of network-level services, including firewall, virtual private network (VPN), intrusion detection/prevention and traffic shaping. All FortiGate antivirus firewalls employ the FortiASIC content-processing chip and the FortiOS operating system.

The FortiGate-800 Antivirus Firewall features four 10/100/1000-Mbps Ethernet ports for networks running at gigabit speeds, and four user-definable 10/100-Mbps ports that provide granular security through multizone capabilities, allowing administrators to segment their network into zones and create policies between zones.

The FortiGate-800 firewall is rated at 600 Mbps, but Fortinet rates the device at 400 Mbps once the IDS/IPS protection module is enabled. At this bandwidth rating, the FortiGate-800 has horsepower to spare, and couples outstanding performance with very low latency when under load.

It achieved at least 400 Mbps of throughput in all the extreme tests (and with all signatures enabled), and much higher in a real-world scenario when used as a dedicated IPS device (performance may be affected if other security modules are activated).

Basic latency figures were well within acceptable limits for a device of this type at all traffic loads and with all packet sizes, ranging from 249 µs with 100 Mbps of 256-byte packets, to 280 µs with 400 Mbps of 1,000-byte packets.

Once the device was placed under load using HTTP traffic, latency figures actually improved. When the FortiGate-800 was loaded with 200 Mbps of genuine HTTP traffic, latency ranged from 138 µs with 256-byte packets to 198 µs with 1,000-byte packets. This is due to the use of NAPI drivers for the network cards, which switch from interrupt-driven to polling mode when under heavy load.

40 Mbps of SYN flood traffic increased latency to 188 µs with 256-byte packets and to 216 µs with 1,000-byte packets (still less than when the device is under a pure UDP load). HTTP response times also increased only slightly during the SYN flood tests, from 214 ms under normal load to 219 ms with the SYN flood. SYN flood mitigation was handled well throughout the test, with the flood being completely mitigated once the initial threshold had been passed. Overall, latency figures were considered to be very good for a device of this type.

The FortiGate-800 continued to block attack traffic in a consistent manner while passing 100% of the legitimate traffic, even when under extended attack. Exposing the sensor interface to ISIC-generated traffic had no adverse effect, and the device continued to detect and block all other exploits throughout and following the ISIC attack.

Signature recognition (with blocking disabled) was low out of the box (75%), but was increased to 95% after the application of a signature pack update.

Blocking performance was slightly higher than detection throughout the tests due to the fact that the firewall module silently drops certain packets, which means the IPS module could not detect them. Blocking performance was increased from 80% to 100% following the application of the signature update.

All false negative (modified exploit) cases were detected accurately following the signature update, and none of the false positive cases triggered once the update had been applied.

The FortiGate-800 achieved almost a clean sweep across the board in evasion tests. Not only were the fragmented and obfuscated attacks blocked successfully, but almost all of them were decoded accurately, as well.

The Web Manager included out of the box is a basic browser-based GUI that has a long way to go before it could be described as an enterprise-class management solution for an IPS device. The Web Manager is severely lacking in the areas of policy definition and deployment, searching, making mass changes to signatures, reporting and forensic analysis.

The company does provide alternative offerings, which should address most of the concerns in this area if policy definition and forensic analysis/reporting are important. For those customers wishing to deploy multiple appliances or with more advanced reporting and analysis requirements, these alternative offerings should be investigated.

The other key point is that the FortiGate-800 is not actually a dedicated IPS device. It is a multifunction security appliance that includes several other security modules, all of which need to be managed via the same interface. From a usability point of view, therefore, Fortinet has actually done a reasonable job in making all of those modules configurable via a single interface in a straightforward and intuitive way.

Securesoft Absolute IPS NP5G V1.1

The SecureSoft Absolute IPS series is a family of hardware appliances designed to detect and prevent attacks across multiple network segments at up to 8 Gbps. The NP5G tested is a dedicated 4U appliance designed to monitor and protect multiple network segments at up to 4 Gbps. The device sports four 10/100/1000-Mbps ports (both copper and fiber) for detection and protection, and three additional ports for management. The four detection ports can be configured in various combinations of two in-line pairs, four single ports, one single four-port device and so on.

The Absolute IPS also supports virtual sensors, allowing up to 200 to be defined (based on physical ports, IPS addresses or VLAN tags), and a unique security policy applied to each. Signature coverage requires some work, however.

The NP5G, although rated at 4 Gbps, was tested as a 1-Gbps device on this occasion. It turned in an outstanding performance in all the tests, achieving 100% detection rates across the board, with plenty of headroom to spare.

Basic latency figures were also outstanding–almost switch like–across the board. They ranged from 23 µs with 250 Mbps of 256-byte packets, to 46 µs with 1 Gbps of 1,000-byte packets.

Behavior was extremely consistent and predictable, latency hardly increasing at all as additional network load was applied from 250 Mbps to 1 Gbps. This was also the only device tested that achieved zero packet loss and low latency at all packet sizes (including 64 bytes) up to 1 Gbps. Neither normal traffic nor SYN flood attack had any appreciable effect on latency, and HTTP response times were also excellent at all network loads.

Overall, latency figures were considered to be outstanding for a device of this type under all load conditions and packet sizes. Clearly, this device can be placed anywhere on the corporate network–from the perimeter to a heavily-loaded high-speed backbone–without impacting overall network performance.

Absolute IPS blocked attack traffic in a consistent manner while passing 100% of the legitimate traffic, even when under extended attack. Exposing the sensor interface to ISIC-generated traffic had no adverse effect, and the device continued to detect and block all other exploits throughout and following the ISIC attack.

Signature recognition (with blocking disabled) was poor out of the box (69%), but was increased to 96% after the application of a signature pack update. Blocking performance was identical throughout the tests. The performance of the box was not affected by the large update.

A minimum of “noise” was noted, with very few test cases raising multiple alerts for a single exploit. Performance in “false negative” tests was once again poor out of the box, though the device missed only one test case following the signature update.

The product’s resistance to false positives was perfect in the tests (although the Back Orifice 2000 signature caused problems).

Out of the box, the Absolute IPS is designed to handle two million open connections without tuning. It was thus able to handle the one million open connection test with ease.

Following a firmware update, resistance to known evasion techniques was excellent, with the Absolute IPS achieving a clean sweep across the board in all the evasion tests. Not only were the fragmented and obfuscated attacks blocked successfully, all of them were decoded accurately, as well.

While the Absolute IPS Manager software is reasonably easy to use, the two-tier management system with a limit of five sensors per console does not scale well, and has some severe limitations in policy management and deployment. The limit of five sensors per console makes effective deployment of policies and consolidating of alerts and reports across more than five sensors impossible.

Alert handling is basic, too, although the summary reports, real-time monitoring and extensive log search capabilities do redeem the software somewhat and make day-to-day monitoring and forensic analysis tasks reasonably straightforward. At the moment, the software is lacking in features that you would expect of an enterprise-level system, and it is not scalable.

The existing software is more than adequate to allow the Absolute IPS appliance to perform its allotted task of detecting and preventing intrusions, and will work well enough in smaller deployments with five sensors or less and a single administrator.

Top Layer IPS 5500 V3.3

The Attack Mitigator IPS 5500 is Top Layer’s new family of network intrusion prevention systems. The IPS 5500 automatically mitigates attacks from both external and internal network sources, while allowing legitimate traffic to pass. The current platform–which offers both rate-based and content-based IPS capabilities–is based on an extension of Top Layer’s previous Attack Mitigator appliances.

The IPS 5500 offers high-availability configurations, redundant capabilities, hot-swappable power supplies and hot-swappable fan tray, secure custom operating system, and flexible port-bypass capabilities to provide a high degree of reliability.

The IPS 5500 is the first device to be tested by NSS Group against both content-based IPS and rate-based IPS methodologies, and it performed well in both. This allows a single device to be deployed to protect fully against both types of attack.

The IPS 5500 is rated for a single gigabit link (2 Gbps aggregate throughput) and was tested to 1 Gbps. It achieved 100% detection rates across the board, with some headroom to spare. This device can be rated at a minimum of 1 Gbps under all network loads.

Basic latency figures were outstanding–almost switch like–across the board under all traffic loads. They ranged from 20 µs with 250 Mbps of 256-byte packets, to 41 µs with 1 Gbps of 1,000-byte packets. Behavior throughout the tests with no background traffic was consistent and predictable, hardly increasing at all as additional network load was applied from 250 Mbps to 1 Gbps.

The IPS 5500 was also one of the few devices tested that achieved zero packet loss and low latency at all packet sizes (including 64 bytes) up to 1 Gbps.

The latency with 64-byte packets at 1 Gbps was just 17 µs (which also includes the basic latency of the test infrastructure).

Placing the device under a half load of 500 Mbps of HTTP traffic, some slight increases in latency were noted, though the figures never climbed above 62 µs. HTTP response times were also excellent.

100 Mbps of SYN flood traffic had a negligible effect on the device, increasing the base latencies at all packet sizes by around a microsecond only. The SYN flood was mitigated completely once it had been detected. With many other types of attack, the IPS 5500 reduced the effects of the attack rather than block it completely.

Overall, latency figures were considered to be outstanding for a device of this type under all load conditions and packet sizes. This device can be placed anywhere on the corporate network–from the perimeter to a heavily loaded high-speed backbone–without impacting overall network performance.

The IPS 5500 continued to block attack traffic in a consistent manner while passing 100% of the legitimate traffic, even when under extended attack. In the rate-based attacks, the IPS 5500 performed equally well. Performance at all levels of the load tests was impeccable, with 100% of all attacks being detected and mitigated under all load conditions, and no interruptions to legitimate sessions. Latency, too, was very low across all tests, even when under heavy DoS attack.

DDoS attacks (multiple source IPs) proved trickier to handle, with CPU becoming a bottleneck much earlier (between 200 Mbps and 400 Mbps), causing packet loss. The IPS 5500 device is rated for DDoS protection at up to 500,000 pps, (approximately 333 Mbps with 64-byte packets), and the ProtectionCluster feature (not tested) can be used to scale this solution to higher rates.

Overall latency performance under all normal and DoS conditions was considered to be excellent, and HTTP response times remained consistent throughout all DoS attacks.

Signature recognition (with blocking disabled) was good out of the box (85%), and was increased to 94% after the application of a signature pack update. Blocking performance was identical throughout the tests.

A minimum of “noise” was noted, with very few test cases raising multiple alerts for a single exploit. Performance in “false negative” tests was reasonable out of the box, but did not improve following the signature update.

The IPS 5500 resistance to false positives is good, but when the various attack mitigation features are employed, careful tuning of the mitigation parameters is required in order that legitimate traffic is not blocked accidentally.

There is no automatic “learning” capability, meaning the responsibility for determining the optimum mitigation thresholds lies squarely with the administrator.

The IPS 5500 achieved a clean sweep across the board in most evasion tests. Fragroute and Whisker both failed to trick the device into ignoring valid attacks. The fragmented and obfuscated attacks were not only blocked successfully, many of them were decoded accurately, as well.

Out of the box, the IPS 5500 is designed to handle one million open connections without tuning. It was thus able to handle the one million open-connection test with ease. Stateless “exploits,” or mid-flows, are handled correctly.

Configuring for the rate-based attacks required more care and consideration in order to avoid self-imposed DoS conditions. Once configured, however, the device detected and mitigated most attacks successfully.

The IPS 5500 is lacking in its ability to detect and mitigate certain scan and probe attempts, relying on its firewall filtering to catch some, and mitigating others only partially. Scan and probe attempts aside, however, the DoS and DDoS mitigation proved to be excellent, as did the resistance to common evasion techniques.

Performance in the high-volume detection/mitigation test was almost impeccable across the board, with perfect detection and mitigation at all load levels. Some problems were noted in passing legitimate traffic at the highest load levels of some of the DDoS attacks, due to high CPU utilization and subsequent packet loss. These load levels can be considered excessive, however, and the device performed almost impeccably up to the 600-Mbps level of attack traffic.

V-Secure V-100 V7.0

The only pure rate-based product in the test, the V-Secure IPS series is a family of attack detection and mitigation appliances designed to detect and prevent rate-based attacks across multiple network segments at speeds of more than 100 Mbps. The V-100 tested is the midrange 100-Mbps model. A dedicated 1U appliance designed to monitor a single network segment, the device sports two copper 10/100-Mbps ports for in-line operation, and a single 10/100-Mbps port for management. Under normal network conditions, the 100-Mbps rating of this device can be verified.

With a range of innovative technologies under the hood, the V-100’s detection and mitigation capabilities tested up to 100 Mbps, and performance at all levels of the load tests was impeccable, with 100% of all attacks being detected and mitigated under all load conditions.

Basic latency figures were excellent at all traffic loads and with all packet sizes, ranging from 52 µs with 25 Mbps of 256-byte packets, to 72 µs with 100 Mbps of 1,000-byte packets. Behavior throughout the tests with no background traffic was even and predictable, remaining at well under 100 µs under all load conditions and with all packet sizes.

Placing the device under increasing loads of HTTP traffic also had minimal effect on the latency figures, which ranged from 57 µs with 25 Mbps of 256-byte packets, to 95 µs with 100 Mbps of 1,000-byte packets.

All of these figures are well inside the limits expected for a 100-Mbps device, meaning the V-100 could be situated anywhere on a 100-Mbps network, either internally or at the perimeter.

Mitigation was handled well at almost all levels, and latency was excellent when under attack with all but the heaviest loads, ranging from just 72 µs at 20 Mbps of SYN flood traffic to 250 µs at 60 Mbps. Performance under the heaviest loads was hampered by the overall packet-processing limit of the device, which is around 70,000 packets per second.

While the overall packet-processing limit of the device seems low for a 100-Mbps device, this limitation is on traffic passing through the appliance. Higher loads of mitigated traffic can be handled, since they are rejected at the external interface rather than being passed through the appliance.

Under eight hours of extended attack (comprising millions of sessions of legitimate traffic interspersed with some attacks) the V-100 continued to mitigate 100% of attack traffic, while passing 99.992% of legitimate traffic. Exposing the sensor interface to ISIC-generated traffic had no adverse effect, and the device continued to detect and block all other exploits throughout and following the ISIC attack.

Attack detection/mitigation was excellent, with the V-100 detecting and successfully mitigating all but one attack. The Back Orifice controller used in the tests was able to contact the Back Orifice server installed on the protected network, receiving a large number of packets in response. These outbound packets were detected as an outbound UDP flood, but the V-100 was unable to mitigate it successfully.

Performance in the high-volume detection/mitigation test was impeccable across the board, with perfect detection and mitigation at all load levels. Once the trusted groups had been configured and the protected hosts approved, the V-100 ran through every single one of the tests without raising a single false positive alert.

The V-100 successfully detected all fragmented and slow evasion attacks, and all but one of the Whisker evasion techniques. Evading this device by simply slowing down port scans and connection floods would appear to be difficult–perhaps impossible–thanks to the fuzzy logic mechanism employed to compare “normal” vs. “abnormal” traffic.

Initial configuration and deployment is straightforward, especially when configuring the basic network-level global protection in detection mode and allowing the V-Secure device to alert which hosts and services it finds to configure further. If there are too many protected hosts (in the hundreds), however, the GUI has some issues handling them en masse (i.e., when creating a protected group) and the overall performance of the system suffers, as well. Most organizations are likely to have tens rather than hundreds of specifically protected servers behind each individual V-Secure appliance, however, so this should not be much of a problem.

SuperVisor Client offers an intuitive and usable means of managing, monitoring and configuring a single V-Secure device. Reporting is basic, with the Web-based reporter utility offering high-level reports with no drill-down capability, and no extensive query or report scheduling capabilities. Nevertheless, monitoring capabilities are excellent, with a wide range of monitoring screens providing good insight into the device behavior and underlying network traffic.

To obtain more information about the products reviewed here:

Broadweb:
www.rsleads.com/509cn-251

Fortinet:
www.rsleads.com/509cn-252

Securesoft:
www.rsleads.com/509cn-253

The NSS Group:
www.rsleads.com/509cn-256

Top Layer Networks:
www.rsleads.com/509cn-254

V-Secure:
www.rsleads.com/509cn-255

Bob Walder is a leading authority on network security and one of the founders of The NSS Group, one of the world’s foremost independent security testing facilities. With British headquarters, and security and network infrastructure testing facilities in the South of France, The NSS Group offers a range of specialist IT, networking and security-related services to vendors and end-user organizations worldwide.