Challenges
The California Natural Resources Agency (CNRA) organizations’ collective mission is to restore, protect, and manage the state’s natural, historical and cultural resources for current and future generations.
The Agency’s mission is a big job. With key organizations, such as the Department of Water Resources, CalFire, Department of Fish and Wildlife, Department of Conservation, California Energy Commission, and the Department of Parks and Recreation, working on major initiatives and critical operations, the need to create and maintain strong and innovative business-enabling applications is critical.
The overall application portfolio managed by CNRA organizations currently totals over 1,400 applications, both custom and commercial. As the number of systems proliferated, the organizations needed a clearer picture of how their applications were performing.
“Our challenge was simple. We needed to know that an application was underperforming before it went down,” said Dana Fernandez, Enterprise Application Solution Architect.
The CNRA IT organizations had many existing monitoring tools, but adoption was very low. “Our previous tools were cumbersome. Using them was time-consuming. In the end, the only result was a few metrics on our infrastructure. They just did not address our application-level monitoring needs in an effective manner,” said Fernandez.
The gaps in their previous performance management tool helped build-out requirements for the Agency’s future monitoring state, “What we really needed was a tool that gave us a comprehensive picture of both infrastructure and applications. Also, it had to be easy to use, or we wouldn’t use it,” said Fernandez.
Lacking a clear picture of how applications were performing, where the pain points were, and connectivity problems between environments motivated CNRA IT organizations to consider integrating an APM tool. In early 2018, they deployed AppDynamics.
Solution
The initial deployment of AppDynamics was a proof of concept on a single application. Clyde Blaisdell, Enterprise Development Chief, was pleased that the initial deployment was only a few hours. “Initially, our primary goal with an APM tool was to have fewer calls from users. Less calls means less downtime,” said Blaisdell.
Several of the CNRA organizations’ custom applications have very high usage at peak times. One example is scientific data. Scientific data is consumed by a broad range of users, from individual farmers to water management vendors accessing the data via APIs. Tens of thousands of users access this data on a daily basis, especially in the morning, before the workday begins. This tax on the application and resulting infrastructure resulted in roughly two calls a month from business users. A call meant the application responsiveness was slow or down completely.
“With AppDynamics, we now have months that go by without a single call. That means the application is not crashing. When we have issues, we’re able to see them and fix them—before it impacts an end user,” said Blaisdell.
Quick wins
CNRA organizations saw value in AppDynamics on the first few days of deployment. “One of our applications was very problematic. It was underperforming during spikes, but the spike times were inconsistent. This made it very difficult to diagnose. AppDynamics allowed us to pinpoint the lack of capacity in a specific virtual machine. So, we increased the capacity. We fixed that day one. That was six months ago. We’ve not had an issue since,” said Fernandez.
Looking ahead—speed & migration
CNRA organizations are hoping to capitalize on AppDynamics capabilities to speed up response times and help manage their gradual transition to the cloud. “Right now, we get the biggest bang for our buck from monitoring and troubleshooting. Simple things, like moving an alert from an email to an SMS, really impact response times.