Some data (metrics) does not have obvious thresholds. For instance, overall disk I/O rate for a server. You have to analyze each particular disk to find some hotspots using I/O rate in conjunction with the disk busy metric, but it is very intensive work which is hard to automate. How to do that I explained in my CMG’03 paper “Disk Subsystem Capacity Management, Based on Business Drivers, I/O Performance Metrics and MASF”.
In that paper I also suggested to use Control Limits as a Dynamic Threshold. Plus I suggested to collect and analyses EV meta-metric called there the “Extra IOs” becouse for I/O rate parent metric the EV (Exception Value) has physical meaning (additional and unusual number of I/Os system processed).
EV meta-metric behaves like 1st derivative of the parent metric. If metric stays constant (between limits) EV=0; If it linearly grows, EV=CONSTANT>0. If it linearly goes down, EV=-CONSTANT<0 and="" on.="" so="">
That fact can help to identify automatically some important patterns in the data history using very simple and universal threshold EV=0. Analyzing the EV trend even could help to predict some future states of the system.
If EV is positive and then got mostly zero that could be indication of the saturation. And the suturation usualy indicates some capacity issue.
If EV was mostly zero and started to be positive that could be indication of the trend beginning.
To illustrate that I am useing the population growth (logistic) curve to simulate some trend starting and the saturation point reaching as that curve naturaly has both pints (S-curve).
Plus I have randomized that curve by adding some random component to simulate a volatility. See in the picture how EV behaves indicating both events:
Of couse the experienced analyst can see just eyeballing when the saturation started, but in the "big data" era we have to deal with dozens thousends systems so we got to automate this type of patterns capture!
And EV could help.
No comments:
Post a Comment