From constituent information to tax revenues to the daily management of programs and services, local governments are data rich, and they are increasingly recognizing the potential of putting this information to work in a modern DataOps environment. DataOps is an agile approach to data management and processing where reducing the end-to-end cycle time of data analytics is key. This new-found agility lays the groundwork for governments to be forward-thinking and drive innovation. When governments are able to put their data to work using a modern system of data analysis, they advance their ability to make fast yet strategic policy decisions.
Pima County in Southern Arizona, with a population of 1.047 million, wanted to upgrade to modern DataOps and make use of their data, which was spread across many departments, agencies, and organizations in a multitude of formats and systems. In 2019, Pima County formed a Department of Analytics and Data Governance (ADG) with the goal of combining departmental data systems and creating an automated data pipeline that would minimize time-to-value with real-time reporting. This turned out to be a fortuitous decision when, in short order, the COVID-19 pandemic pushed governments to make swift decisions to address urgent economic and social impacts. Since the start of the pandemic, adopting a DataOps methodology has become a priority for many state and local governments.
Scott Zufelt, Deputy Director of ADG, said, “without timely data, we’d be forced to make concessions that were not acceptable to the county.” Even a two-day turnaround would run the risk of dated information. In order to address this issue, Pima County focused on improving data availability and processing time by implementing a sophisticated data analysis platform, centralizing their data in a cloud-based data warehouse, and making sure the two solutions could work together seamlessly.
Pima County began by identifying a modern data platform that could draw data from multiple sources to quickly create advanced reports and dashboards, and partnered with Qlik. In selecting a platform for data analysis, Zufelt prioritized speed. “[We moved] to a modern platform that allowed us to move only the data that is being changed as it is being created,” he said.
Centralizing data on a cloud-based data warehouse was Pima County’s next priority. Cloud Software-as-a-Service (SaaS) solutions have been a boon for governments looking to modernize their data systems, as these solutions offer full management of data through cloud software distribution. With Snowflake Data Cloud, which supports data warehousing workloads and other workloads, governments only need to provide the data, and Snowflake hosts it on an infrastructure that is both scalable and cost-effective. Snowflake then manages all data administration, so governments don’t have to take on the costly and time-consuming burden of additional IT management. This not only improves interdepartmental accessibility to data, but it also increases both internal and external transparency. Departments that are accustomed to working in silos become better able to understand how a government works together as a whole; in turn, they become better able to relay information to citizens.
Zufelt said that the new data architecture reduced report time dramatically—creating a report at Pima County used to take 24 hours, but after the implementation of the Snowflake platform, which could interact seamlessly with its data analysis platform, that time was reduced to 9 milliseconds. When data analysis platforms are equipped to integrate with data warehouses such as those provided by Snowflake, reports can be generated from data captured off transaction logs, eliminating processing time while reporting in near real time.
With state and local governments being asked to do more with less, a modern integrated data architecture allows these organizations to improve citizen services and response by continually embracing innovation. Governments can achieve this by investing in advanced data analysis platforms that work well with a platform such as Snowflake that supports data warehouse workloads. This approach is necessary to embrace the modern DataOps concept, which will enable governments to spend less time on day-to-day administration and more time addressing citizen concerns.
To learn more, watch the webinar Moving to Modern DataOps for State and Local Governments.