When it comes to monitoring and managing complex IT networks, especially those that are high value targets for cyber criminals and rogue states, it would seem that the ideal solution would be to build custom tools uniquely suited for the environment and to be able to support the tools with an army of domain experts.
Or is it?
As David A. Richards, Senior Technical Manager EOSS/GuardNet, one of the largest Department of Defense (DoD) networks, shared recently at the SolarWinds Federal User Group, “When it comes to ensuring the safety, security, and continuous operation of GuardNet, it’s vitally important that we be able to customize our tools to achieve organizational objectives and make actionable decisions.”
Far too often, he said, he and his team found themselves in a position where they were inundated with information and couldn’t find the proverbial needle in the haystack. In the case of ensuring the continuity and integrity of network operations, being unable to identify and isolate the issue degraded the trouble ticket management system and resulted in more issues rather than faster fixes and trouble ticket resolutions.
For Richards the opportunity to break out of the cycle of information overload, analysis paralysis, and circular discussions came in the form of network management and monitoring tools from SolarWinds. While the assumption might be that ‘out of the box’ tools might not be able to cope with the rigors of a complex environment like GuardNet and simply reinforce the challenges that Richards was already grappling with, the tools came with a significant strategic advantage: native customization capabilities.
Starting with a customer-generated architecture diagram, Richards and his team were able to rebuild the network to meet not only today’s needs, but also to prepare for additional demands on the network that will come as the DoD rolls out the Joint Regional Security Stacks (JRSS), which will add more nodes on a global scale and will also require compliance with new DISA security standards that will apply across the DoD, including GuardNet.
So what advice does Richards have for other government IT leaders who are responsible for complex networks?
- Stop thinking in terms of single devices. Start thinking of the network as an ecosystem and identify dependencies within the ecosystem.
- Use monitoring tools to help visualize the network and draw a map, color code it, and share the map.
- Identify patterns of failure and recurrent problem areas, overlay them on the map, and target those areas for remediation.
- Move from a technical diagram to a format that can communicate the business value in order to secure funding for additional network monitoring tools that will help automate routine tasks, such as load balancing and patch updating.
These lessons were invaluable to Richards and his team during last year’s hurricane crisis in Puerto Rico. Following Hurricane Maria, sites in Puerto Rico could no longer monitor GuardNet. Over the course of a weekend, Richards and his team ensured that sites on the mainland were able to add that workload. While the initial step was just to see what parts of the network and devices were up or down, they were able to quickly access credentials and add specific device monitoring and management to ensure continuity of operations.
As Richards shared, “the ability to create a regional view of the situation in a very short period of time gave better insight into areas of most damage and criticality and got us on the right track to normal operations much more quickly than anyone anticipated.”
Learn about solutions that offer network management and monitoring in any circumstance here.