Anomaly Detection Optimization in Cloud Technology Metric Time Series

Abstract

In modern cloud IT infrastructures, hundreds of metrics must be monitored to detect anomalies and ensure stable operation. Many of these metrics exhibit multi-seasonal characteristics, requiring decomposition into three components: trend, seasonality, and residuals. However, most known decomposition methods, including the well-known Multiple Seasonal-Trend decomposition using Locally Estimated Scatterplot Smoothing (MSTL), require significant computational resources. This article proposes an alternative, more computationally efficient approach for detecting anomalies in a large array of metrics, particularly for real-time applications. The method`s core idea is to identify linearly dependent metrics and model their trend and seasonal components using linear regression with other metrics. Anomaly detection is then performed on the residual component. This reduces the required computational power, thereby optimizing the expenses of cloud infrastructure monitoring centers. This work also demonstrates how to select a suitable subset of metrics for linear regression modeling and decomposition, calculate the optimal time series length for determining linear regression model coefficients, and establish reliable criteria for detecting anomalous metric values. To verify the effectiveness of this linear regression-based decomposition approach, the experiment was conducted on a real cloud infrastructure. The experiment involved a containerized web application with heavy traffic on the Google Cloud Platform. The results showed that the Chebyshev inequality-based approach was the most suitable anomaly detection criterion in this case. Furthermore, a connection was established between the system's architecture, which defines the nature of the metrics, and their statistical properties, which influences their selection for the proposed optimization approach.