TESTING AND DIAGNOSTICS

by Ray Peckham, Associate Editor

At application service provider Surebridge, Mark Clayman, director of hosting, and his staff monitor more than 1,200 devices, including routers and switches. Clayman found he did not have the right tools to convert the generic reports of problems from customers into focused task lists for technicians to resolve issues quickly.

A customer that was connected to Lexington, MA-based Surebridge, was having problems opening Excel files over the network. The problem, like so many that are reported to IT departments each day, involved only a generic description of an intermittent issue with certain machines. Clayman’s staff spent substantial time trying to troubleshoot the problem, but with no success.

“We literally spent months with manpower running around,” says Clayman, “switching out network cards, switching out desktops and changing configurations on the server itself. We were unable to resolve the issue.”

Situations like this haunt many enterprises, creating a growing market for enterprise network monitoring and management tools that, according to IDC, is expected to continue increasing to $24.9 billion in 2005. These beneficial tools limit the negative impact poor network performance has among customers, allow companies to get more productivity from existing IT staff and reduce costly business delays as a result of network downtime.

A determined Clayman implemented the Eye of the Storm product from Entuity in August of 2000. “Within two to three weeks, we had gotten a good baseline of the network,” he says, “and good data that indicated that we had misconfigurations between the new types of desktops the customer was ordering and the switches they were connected to, as far as if they were set to full-duplex or half-duplex.”

Armed with the detailed information provided by his network monitoring tool, Clayman’s staff could finally resolve the problem. “We could see that some of these desktops that were coming in were misconfigured by default,” he says. “It was a simple matter of going back out there and changing either the switch or the desktop, and that resolved the issue right away.”

Clayman had relied mostly on vendor-supplied tools in the past, but these did not give him the insight into the network he needed to get the most out of his technical employees. “People would say that the network appears slow, or we are having another type of issue,” he says. “It was taking us weeks, if not months, to nail down a very simple issue that we now can solve within a day. It gives us a target, giving us the information we are looking for, as opposed to complete guesswork.”

The benefits of using the network monitoring tool have been direct and measurable for Surebridge’s 150 employees. If Clayman did not have the tool, “instead of the three network engineers we have today,” he says, “we would be looking at five. It cuts the time that network engineers spend on troubleshooting the problem considerably.”

The monitoring tool has also given Clayman and his staff an increased level of confidence about the status of the network when responding to customer problems. “Now, we have a comfort level that the network is close to 100% clean,” he says. “It has created a stable environment for us, and has reduced many errors that we had on our network.”

MONITORING FOR LEGACY APPLICATIONS

John Ahearn, national information systems director for Audio Visual Headquarters, a provider of high-end audiovisual equipment rentals, also has an appreciation for the importance of accurate network monitoring and how it can make a direct impact on a company.

“We have to integrate older legacy applications from other companies that have merged with us,” says Ahearn. “They are real particular about going through firewalls and going through the network, and having people being able to connect to them. We use the protocol analyzer as a way to diagnose or find problems with these legacy applications.”

The $200-million-a-year business has offices throughout the U.S., and a full-time labor force of 500. The company uses a global VPN, stretching from Hawaii to Washington, with 2,000 nodes and 1,200 computers connected.

Each location has its own network servers or custom Oracle databases. These provide data that must be integrated into the central financial system at corporate headquarters in Los Angeles, where the company also has data warehousing, and the server farm and major Internet access for e-commerce.

These older, custom-written or poorly ported client/server applications are time sensitive. According to Ahearn, without the more recent QoS enhancements to ethernet, many of them simply would not run. “You will find out that a business will stop very fast when the people out in shipping and receiving can’t receive or issue,” he says.

With protocol analyzer software from Network Instruments, called Observer, Ahearn is able to identify and analyze the traffic generated by these older applications. With this knowledge, he can approach the software developers with specific requests for changes to accommodate ethernet communications.

In some cases, the applications cannot be changed, because they are too old or there is no access to the developers. In these cases, the port and traffic information gained from using the protocol analyzer allows Ahearn to program his routers and switches to allow those applications to work immediately, rather than delaying the business while waiting for a software solution.

“We were able to identify all the traffic that the software generated,” says Ahearn, “including the port information, and we bought switches and routers and programmed them to put some sort of quality of service on the traffic for the application. This solved the problem until the software was corrected, some year and a half later.”

SECURITY TROUBLESHOOTING

Network monitoring tools can also provide an irreplaceable source of information when troubleshooting security issues threaten to stop productivity in a network.

“We use the protocol analyzer to check what is going in and out of our VPN tunnels,” says Ahearn. “We have used it for identifying intruders and strange traffic that has been coming from the Internet, to tracing strange behaviors and monitoring traffic usage. If you are really worried about security, then you should be able to look at your network, see what is in it and what is going around it.”

Ahearn relates an experience with another company involving a poorly ported Unix application that would drop connection to all terminals each time someone would copy large files on the now NT-based system. “We took the protocol analyzers and were able to identify who was generating all this traffic,” he says. “The funny thing is, they didn’t need to be generating the traffic; they were copying graphic files off the Internet.

“The big issue we have today, especially as we move into economical times like this,” he adds, “is that it is more important than ever that people start to make sure they are getting full use of their assets. The network, computers and communications equipment are very expensive company assets. I do not want people wasting time, losing productivity and utilizing bandwidth that we have paid for to do stuff that is not work related.”

EASE-OF-USE IMPORTANCE

To get the most out of all the detail available from monitoring tools, finding a system that provides the information in an easy-to-understand format is important. “The biggest thing when you are looking for a network management tool,” says Clayman, “is finding something that is simple to use. There are plenty of tools that are extremely technical, but they are not presenting the information in a manner that someone who is not a seasoned engineer could really understand.”

Kirk Jones, senior product verification engineer for Nortel Networks, agrees that monitoring and testing tools are essential for an enterprise, but should not be hard to use. He uses products from Empirix to simulate enterprise installations and to look at real-world IP impairments that the Nortel equipment could confront when implemented with a customer.

“With today’s technology,” says Jones, “sometimes people do not have the necessary knowledge in the area they are trying to focus on. If the test set can nurture them along, then they do not have to be a rocket scientist or a nuclear physicist. My rule of thumb is: if you can’t get me up to speed so that I can actually use it within a half hour, I do not want to look at it.”

What Jones does look at is what a monitoring system ultimately costs, not only the initial purchase for the equipment or software, but also the training to get an employee up to speed to use it. Jones was able to use a summer student with no telephony background with Empirix’s Hammer PacketSphere product.

Ideally, future advancements in network monitoring systems could make the opportunity for less-skilled staff to play a key role in managing the network even easier. “In a perfect world,” says Clayman, “it would be great if the application was integrated into the knowledge base from all the major networking (equipment) providers. When an error message pops up, it might search that database and spit out a configuration solution.”

Even without that knowledge base link, modern network monitoring tools are essential for any IT manager to keep a network working. Without them, talented technical staff and other resources can easily be wasted. “You can’t work in the dark,” says Ahearn. “You absolutely have to know what is in that wire.”

“I would not recommend anybody flying blind in managing a network,” says Clayman. “If they are not aware of the problems on their network, they are wasting valuable resources.”

Resources

Empirix
781-993-8500

Entuity
212-489-0559

Network Instruments
800-526-7919

For more companies offering network monitoring and testing related products and services, see our online buyers guides at www.comnews.com/cgi-bin/bg/default.asp.