An independent test and evaluation of 15 different network intrusion-protection system products from seven vendors showed none were fully effective in warding off attacks against Microsoft, Adobe and other programs.
NSS Labs, which conducted the evaluation, found that the Sourcefire IPS showed 89% effectiveness against a total of 1,159 attacks on products such as Windows, Adobe Acrobat and Microsoft SharePoint, while the Juniper IPS scored lowest at only 17% effectiveness. NSS Labs, which conducted the test without vendor sponsorship of any kind, also evaluated the 15 network IPS offerings for their capability in responding to “evasions,” attacks delivered in an obfuscated and stealthy manner in order to hide. In that arena, the McAfee and IBM IPS held up particularly well.
Rick Moy, president of NSS Labs, says he was disappointed overall that none of the 10Mbps to 10Gbps IPS products tested achieved 100% effectiveness in detecting and blocking the attacks, including buffer overflow exploits.
Products tested came from Cisco, IBM, Juniper, McAfee, Sourcefire, Stonesoft and TippingPoint. Check Point, Enterasys, Nitro Security, Radware, StillSecure, Top Layer and Trustwave declined to participate in this round of tests, which were conducted in October and November.
“The threats are continuing to get worse and everyone says they're keeping up with them, so we wanted them to prove it,” Moy says.
The vendors that did participate were allowed to tune their equipment in one round of tests designed to find out how long it took to make changes to the default settings in order to try and improve performance based on policy. Under this measurement, McAfee, IBM and Stonesoft did well. The Sourcefire IPS, however, took the most time, which Moy says would translate into time needed for professionals to manage it in an enterprise.
McAfee, which on Tuesday will make major announcements related to new network-security gear, was left at a loss to explain why the its IPS didn't achieve 100% effectiveness in the NSS Labs tests.
“There are a variety of reasons you might not achieve 100%,” says Greg Brown, McAfee's senior director of products marketing, who adds he hasn't read the NSS Labs report yet. Sometimes lab tests simply “don't look like a real attack” to equipment. He says McAfee focuses its efforts on “very new exploits.”
Details on the IPS effectiveness, evasion attacks, tuning, performance and cost-of-ownership issues are included in depth in the 50-plus page report “Network Intrusion Prevention Group Test” that NSS Labs is selling for $1,800. NSS Labs also anticipates conducting a round of tests for host-based IPS products in the near future.
