Identify trends and anomalies by using wavelets - CMG'12 paper

Twelve years ago being hired as a Capacity Planner first time and not knowing anything about the specialty, I was asked to look at computer measurements to do any analysis. Applying my academic researcher skills I noticed that server's capacity usage usually shows different seasonality patterns (shifts, weekends, lunch times and so on); so my first impulse was to decompose that to several signals using Fourier Transform. I formulate the task to my manager, but for some reason (to much math and computing power requirements?) he suggested to use just standard statistical approaches instead, and I did…

I did develop and implement SEDS methodology (statistical exception detection) that naturally covers daily and weekly cycles and potentially can uncover monthly and yearly ones if there is enough data history to build baseline.

But Fourier analysis was still in my mind, and I have just been waiting for the time when our computational capacity would be enough to apply the method against system performance data….

That’s why I was positively surprised when I noticed the following paper in CMG’12 agenda:

Introduction to Wavelets and their Application for Computer Performance Trend and Anomaly Detection  Dima Seliverstov, BMC Software

ABSTRACT: This paper presents a technique to identify trends and anomalies in Performance data using wavelets.

And of course, I planed (see my past post here), attended and enjoyed that, I know Dima via other CMG events, spoke with him a few times and have already analyzed some other his papers in this blog (The Exception Value Concept to Measure Magnitude of Systems Behavior Anomalies).

His "wavelet" paper is about implementation of my old dream to use something like Fourier analysis and his idea of decomposing the performance data to combination of typical wavelets is a good attempt to do that. Especially impressive was to see the "Scalogram  that is a type of a heat-map to show the location of the energy as a function of frequency and time”: “Scalogram is a heat map for wavelets transformation”. Some interesting examples (including against VMware VCenter data) were presented and was calculated by MATLAB tool:

BTW stock market is already adopting this idea, the paper references that: Wavelets for Stock Market Analysis
Why not us?

… Ironically the same manager asks me now to analyze how business cycles drive capacity usage….