Method of Local Anomalies Identification for Environmental Indicators Values Using Half-Wave Decomposition

Abstract

In the era of mass digitalization of all existing spheres of human activity, the amount of data is constantly growing and it is crucial to be able to work with such volume of data for the solution of various problems. One of the most common data structures is a time series — a sequence of data points, collected over some period of time, usually in chronological order. The time series comprise various financial indicators, environmental monitoring data, medical information, etc. Wide range of application areas makes the problem of time series analysis important and relevant. The quality of the time series forecast greatly depends on the quality of the performed analysis, which may include data standardization, detection of significant indicators, correlation analysis, etc. Anomaly detection occupies very important place among these steps. Anomalies are data points that differ in some way from other values in the dataset or violate certain data behavior patterns. The presence of similar records greatly affects the ability of machine learning models make accurate predictions, is why it is necessary to have the possibility for the identification of these anomalies.

New method of local anomalies identification of the environment state indices using half-wave decomposition has been developed. Main idea of the method is to decompose the time series into half-waves, using trend points where the fall changes growth or vice versa and split the series into fragments. Each fragment is analyzed separately and is checked for anomalies by combining numerous methods. The accuracy of the methods is verified, applying the expert method. Main steps of the proposed method are described and the example of the method usage on real air quality monitoring data obtained from one of the stations of the EcoCity public monitoring network within the international program “Clean Air for Ukraine” is given.

The proposed method was implemented and tested on the Kaggle platform’s notebook. The result of the anomaly detection was used for the construction of the Facebook Prophet model and the accuracy of the time series approximation was compared with the results of the Prophet model operation with the default parameters. Tests have shown 11 % decrease of approximation error of time series for RMSE metric and 8 % decrease for MAE metric. This result confirms the effectiveness of the method.