October 20, 2017

Cross-correlation based clustering and dimension reduction on multivariate time series for KPIs in a data processing system

In this paper, we investigate multi-dimensional time series data and we introduce a graph based clustering approach using the cross-correlation between the time series. The procedure is applied to Key Performance Indicators (KPIs) measured during a specific data processing system in order to identify connections between the various KPIs. The proposed technique consists of two main steps: introducing a novel similarity measure (for measuring cross-correlations) and then clustering (based on the correlations). The two parts are both compared to some existing methods; sensitivity analysis is carried out for various (dis-)similarity measures, and various clustering methods are also compared. The proposed method is suitable for efficient visualization to reveal dependencies and connections between the attributes and also detects a number of "relevant" attributes., i.e., select important features. Our method keeps background assumptions as minimal as possible.