Government wants to transition to cloud for its scalability and pay-as-you-go consumption model, among other benefits. But with some systems in the cloud and others on-premises, it gets hard to surface needed information.
As agencies modernize, they must be able to find, correlate and act on real-time data, regardless of location and format. In a hybrid cloud architecture, they must deliver search in a new way — one where “users don’t have to worry about data that’s distributed and disconnected sometimes,” said Dave Erickson, Elastic’s Public Sector Distinguished Architect. “We need to make it simple for users to ask the same question everywhere.”
Improved Data Availability
A partnership between Elastic and AWS is helping agencies to re-host and re-factor applications in the cloud and to re-architect their systems in support of improved information availability. “That partnership has been transformative,” Erickson said, effectively building “a big coalition of all the places where all the best data is, and making sure users can get to it.”
Elastic already empowers more than 100 federal civilian agencies in support of CISA’s Continuous Diagnostics and Mitigation (CDM) program. Cyberthreat hunters, for example, “can run federated queries across Amazon regions” when new threats emerge, Erickson said. With results returned in milliseconds, “people are transforming what was a week-long process before, into something that’s more at the speed of users’ attention spans.”
The ready availability of such capabilities in the AWS ecosystem helps government meet the challenge of doing more with less. “We want those in our GovCloud environments to be able to buy from the marketplace, just like everybody else does,” Erickson said. As serverless technologies come increasingly to the fore, “we want to ensure that the cost curve for government stays predictable.”
The AI Factor
As agencies pursue modernization, they’re also wondering how artificial intelligence (AI) will factor in. Here, too, modernized search has a role to play.
Typically, generative AI-service providers want to feed AI models all necessary data, to ensure the most accurate outputs. But with privacy and other considerations, agencies want to control their data. “How do we give private data to AI, and stay in this realm of protecting privacy?” Erickson said.
The rise of the “vector database” can help resolve the issue. Such a database stores numerical representations of data, rather than the data itself, allowing AI models to securely access an agency’s proprietary data for context-rich, meaningful outputs.
“A vector database [is] being used as the long-term memory for off-the-shelf AI models,” Erickson said. By keeping the vector in Elasticsearch, government’s private knowledge can be stored efficiently for rapid use.
That could help agencies use AI to support over-burdened workers. Today, “the person who is busiest is the person who’s getting questions all day: Where do I find this? How do I do that?” Erickson said. With AI, people will get sensible answers to plain-language questions. This promises “a world where every tool around me is really trying to help me,” he said.
This article appears in our guide, “A Fresh Look at Data.” For more ideas about how to use data in important and innovative ways, download it here:
