Management tool helps Tennessee Guard stay on top of network performance
Managing change and ensuring performance on a complex network are difficult and time-consuming, which is why the Tennessee Military Department turned to NetMRI to get and keep its network in shape for a new generation of Web-based applications.
Capt. Randy Floyd, network operations and telecommunications manager at the Tennessee Military Department, does not have a lot of time to waste. In addition to managing the Guard’s statewide network with a staff that constantly changes because of active-duty deployments, he is working on a doctorate in business administration, with a focus on information technology.
“I don’t have a lot of time for reactive stuff,” Floyd said. Life is easier for Floyd when he gets the information he needs to proactively manage his network and monitor its performance.
Although his network is state-owned, it falls under Defense Department requirements. And like his network, Floyd’s status is a hybrid of military and civilian, state and federal. He is a federal military technician — a civilian employee who works in uniform for a state but is paid by the federal government. If Tennessee puts him on active duty, the state pays him, and if the federal government activates him, DOD pays him.
Related story:
Even with automation, staffing is the key to managing a network
Several years ago, a series of DOD Information Assurance Vulnerability Alerts (IAVAs) explained the need for a tool that could provide proactive information and automated updates.
“I remember we received an IAVA directive from the DOD and Guard bureau” in late 2006, Floyd said. “We had to have our network devices upgraded by a certain date.”
The first challenge was simply finding out what devices were on the network, a process accomplished by sneakernet — walking from place to place and looking at what was there.
“We had a lot of devices we weren’t aware of,” he said. In addition to forming an inventory, Floyd needed to upgrade each device manually. And it was not a one-time event. “That kind of directive typically comes out once or twice a year,” he said.
Clearly, some automation was needed. Floyd chose Netcordia’s NetMRI appliance. Other tools had similar features, but NetMRI “was just the right mix of benefit vs. cost,” Floyd said. “The learning curve was pretty flat.”
He made the decision to go with the tool after talking with other network managers who used it. An Army major at Fort Hood, Texas, told him that every unit deploying to Iraq and Afghanistan should have NetMRI as standard equipment for tactical and nontactical networks. “This guy didn’t want to leave home without it,” Floyd said.
Although it can be used as a software-only product, NetMRI is usually sold as a stand-alone appliance that runs on a Linux operating system with a MySQL database. It sits on the network, typically in a data center or network operations center, and it gathers data from Layer 2 and Layer 3 devices, such as routers, switches, firewalls, gateways, wireless controllers and wide-area network optimizers.
It collects Simple Network Management Protocol data, configuration data, syslog events and data records of voice-over-IP calls. It passes that information to a database that has built-in best practices and networking policies. It identifies elements that are out of compliance with policies and shows changes in near-real time. It can also be used to schedule work and send changes to devices, and it evaluates a network’s compliance with regulatory requirements.
NetMRI displays network performance information graphically, and when an error or other problem appears, administrators can isolate changes in the network and work backward to find the cause of the problem. It is a tool to help administrators find needles in haystacks and keep track of the needles, said Matt Gowarty, Netcordia’s product marketing manager.
Tracking and managing change produce a more consistent network, which is a more stable network. That can be even more important in an environment in which employees frequently come and go, as they do at the Tennessee Army National Guard.
Floyd was fortunate to be able to start with an up-to-date network that had plenty of bandwidth. The network, which serves 94 locations statewide, recently upgraded from a point-to-point frame-relay network to a mostly meshed Multiprotocol Label Switching network, with a T1 or faster connection to each location.
The network shifted to MPLS about two-and-a-half years ago when the Guard began implementing VOIP.
“We ran out of bandwidth in a hurry,” Floyd said. “With the increased demands of that and other Web-based applications, we had to find another [medium] to make it happen.”
When Verizon offered to provide the Guard with a meshed MPLS network with full T1 connections for less than the Guard had been paying for fractional circuits, “that was a good deal,” Floyd said. “We can easily scale up the MPLS circuits and add new ones.”
The VOIP deployment is not yet complete. “We were in the middle of that when I was deployed to Afghanistan,” Floyd said. “There was nobody to follow up on it. When I got back six months ago, I found it still waiting for me.”
Floyd had three primary requirements for his new network management tool. First, it had to help automate changes to network devices as required by IAVA directives. Doing that by hand took a while, Floyd said. NetMRI can automate the process for most devices.
Second, Floyd wanted a dashboard view that would let him arrive in the morning and quickly check the network’s condition. NetMRI does this with a 10-point scale.
“I take 10 seconds to look at it, and I can see it’s right about nine [on the 10-point scale] every day,” he said. If it is significantly below that, “it gives me the immediate opportunity to start drilling down and see what is going on.”
Third, the database of compliance benchmarks, including the National Security Agency’s metrics, gives him more confidence in the security of the network.
“If I’m doing everything NSA says I should be doing, then I’m likely to be doing all the other things, too,” he said.