As the Biden administration settles in with new priorities, there will be many changes for agency CIOs, but what remains constant is the need for an effective data management strategy. As the volume of structured and unstructured data continues to grow, the opportunity to use – and reuse – that data in mission critical activities grows too. Agencies need a data management strategy to not only ensure that data is available and accessible for applications like AI, ML and advanced analytics, but to also ensure that the cost of data management doesn’t break their budgets.
For most agencies, the focus of their nascent data management strategies has been on managing structured data, but unstructured data is, in fact, the fastest growing source of data for federal agencies. Gartner research shows that unstructured data is growing between 30 – 60% year over year and by 2024, 50% of global unstructured data will be deployed as software defined storage, up for less than 20% in 2020. As agencies look to create a richer user experience for their constituents, manage high-resolution video from satellite feeds, or simply contend with the constant addition of user file data, traditional storage solutions for unstructured data will struggle to keep up with demand.
“Unstructured data is essential for federal agencies as they look to capitalize on AI, the Internet of Things and edge computing to deliver on the mission,” shared Michael Lamb, infrastructure and storage expert with ViON. “Typically, unstructured data has been managed on lower-tier, lower performance storage, such as spinning disk. Spinning disk storage has been highly reliable, scales well and is cost effective,” Lamb added.
Historically, spinning disk has long track recored of managing unstructured data in a scalable and cost-effective way. It leads to the question of why should agencies look for other solutions when building a comprehensive data management strategy?
For Lamb, there are obvious reasons to make the move from storing unstructured data on spinning disk to storing these files – or objects – as-a-Service. “Today, agencies aren’t just using unstructured data once,” he explained. “They’re re-using this data multiple times to fuel the mission and spinning disk isn’t suited to this rapid ingress and egress. Moreover, the way agencies are required to purchase capacity for spinning disk storage, isn’t a good fit for how agency budgets are actually allocated, or how they are using storage.”
Like all as-a-Service offerings, one of the greatest advantages is that an agency can start small and add capacity as needed. “Because an agency is purchasing a service, costs are predictable each month and not variable which works much better with agency budgets, funding and procurement processes,” explained Lamb. “Today, as-a-service offerings are easily accessible through customer marketplaces which provide a service catalog, management and governance to simplify procurement.
As the federal government invests in becoming Cloud Smart and agencies adopt a multicloud strategy, as-a-Service storage options for all types of data will make even more sense. “Being able to access unstructured data from remote storage locations, from the public cloud, and development environments and bring it altogether to meet the mission is invaluable,” he shared.
“Conversations about data storage shouldn’t be conducted in either-or terms,” concluded Lamb. “There are as many options as there are types of data and agencies should follow best practices to build a storage strategy that fits the data, rather than the other way around.”
As agencies continue to aggregate data from edge to core to cloud for advanced applications like AI, ML and advanced analytics, object storage, delivered in an as-a-Service model allows them to start with a small test bed and grow from there. If it isn’t already, object storage as-a-Service should be part of federal agencies data management strategy.