Finding Insights From Noise with AIOps
These days, every company is producing vast amounts of logs and data, and it can be a daunting task to find meaningful insights from this collection. At CoreLogic, using AIOps, or AI Operations, is critical to detecting anomalies, alerting teams, automating responses and even foreseeing future issues before they arise.
AIOps is the shortened term for applying artificial intelligence on a vast amount of operational data. AIOps provide intelligent, actionable insights that help in monitoring and managing IT environments that are hybrid, distributed, and componentized by using operational data, artificial intelligence (AI), and machine learning (ML) to:
- Collect and aggregate ever-increasing volumes of operations data generated by multiple IT infrastructure components, applications, and performance-monitoring tools.
- Develop and deploy machine learning bots that continuously sniff for anomalies that could lead to significant events and patterns related to system performance and availability issues.
- Diagnose root causes and report them to IT for rapid response and remediation—or, in some cases, automatically resolve these issues without human intervention.
It replaces multiple separate, manual IT operations tools with a single, intelligent and automated IT operations platform, thus enabling IT operations teams to respond more quickly—even proactively—to slowdowns and outages, with a lot less effort. This process can help to create predictive outcomes that drive improved system availability and minimize customer impacts.
At CoreLogic, we use a data fabric platform built on the Elastic foundation to aggregate siloed IT operations data in one place. This data includes historical performance, streaming real-time operations events, system logs, network data, incident-related data, ticketing and related document-based data.
We’ve found incredible benefits in this process, and we use it for:
In the CoreLogic ecosystem, about 8 billion to 10 billion operational information lines are generated by the systems daily. A team of engineers are developing advanced algorithms using machine learning techniques to comb through data, identify abnormal events, and trigger timely alerts to the support staff.
We are building a unified dashboard that serves as a single source of truth, delivering valuable information that’s presented in a meaningful way. SPOG also provides a capability to drill down and get more granular data where needed.
As a first step, we built capabilities that automatically route alerts and recommended solutions to the appropriate IT teams. This even creates incident triage bridges based on the nature of the problem and the solution.
We are currently building intelligent triggers that automatically respond to system issues on a real-time basis before users are even aware of the problem.
One of our future goals is to use past data to map out trends utilizing machine learning capabilities to predict future issues and take necessary preventative steps long before the problem impacts the customer.
Ultimately, at CoreLogic, we want to enable IT teams to identify, address, and resolve slow-downs and outages faster. By removing manual processes, we can:
By cutting through IT operations noise and correlating operations data from multiple IT environments, we will be able to identify root causes and propose solutions faster and more accurately than humanly possible. This in turn should significantly improve the MTTR.
Identifying anomalies in the system and projecting/predicting the issues using advanced correlation capabilities will enable development and operations teams address potential problems before they lead to slow-downs or outages.
Instead of being bombarded with every alert from every environment, with AIOps we can remove noise and distracting alerting, instead publishing alerts that meet specific service level thresholds or parameters—complete with all the context required to make the best possible diagnosis and take the best and fastest corrective action.
By exploring new technologies and utilizing these efficiencies, CoreLogic is better able to support our clients—and make better use of our IT teams’ time and energy.
Senior Leader, Software Engineering