Anomaly Detection on Seasonal Metrics via Robust Time Series Decomposition
AAnomaly Detection on Seasonal Metrics via Robust Time Series Decomposition
Tianwei Li , ∗ , Yitong Geng † , Huai Jiang Rutgers University, NJ eBay [email protected], { yigeng, huajiang } @ebay.com Abstract
The stability and persistence of web services areimportant to Internet companies to improve userexperience and business performances. To keepeyes on numerous metrics and report abnormal sit-uations, time series anomaly detection methods aredeveloped and applied by various departments incompanies and institutions. In this paper, we pro-posed a robust anomaly detection algorithm (MED-IFF) to monitor online business metrics in real time.Specifically, a decomposition method using robuststatistical metric–median–of the time series was ap-plied to decouple the trend and seasonal compo-nents. With the effects of daylight saving time(DST) shift and holidays, corresponding compo-nents were decomposed from the time series. Theresidual after decomposition was tested by a gener-alized statistics method to detect outliers in the timeseries. We compared the proposed MEDIFF algo-rithm with two open source algorithms (SH-ESDand DONUT) by using our labeled internal busi-ness metrics. The results demonstrated the effec-tiveness of the proposed MEDIFF algorithm.
To ensure stabilization or increase in revenue, technologycompanies especially Internet companies providing onlinestreaming services to monitor their business metrics and solvemetric related troubles in real time. Anomaly detection isapplied to discover unexpected breakouts caused by exter-nal factors such as vicious Internet attacks or internal factorssuch as services breakdowns. Corresponding actions such asrestoration and maintenance will be took immediately basedon the alarms.Monitoring and detecting anomalies in multiple businesskey performance indicators (KPIs) in real time is a challeng-ing problem in industrial. First, the number of anomalies inthe millions of real-time business KPIs time series is much ∗ Tianwei Li is currently a PhD candidate at Rutgers University.This research was done when Tianwei worked as a full-time internat eBay. † Author to whom any correspondence should be addressed. smaller than that of non-anomalies. The detection algorithmis required to accurately capture the anomalies from hugeamount of metrics. Second, no standard feature of anomaliesleads to the lack of labels. As a result, supervised learningalgorithms [Laptev et al. , 2015][Liu et al. , 2015][Shipmon et al. , 2017], in general, are not suitable for industrial appli-cation. Moreover, it is inevitable that missing or/and errorvalues exist in the business metrics time series under servicemaintenance. Thus, the anomaly detection algorithm shouldbe robust to these missing or/and error values. Finally, toachieve monitor business metrics in real time, the response ofthe anomaly detection algorithm should be fast, such as lessthan 20 % of metrics sampling period. Existing deep learningalgorithms [Zhang and Chen, 2019][Xu et al. , 2018] may notbe suitable for this requirement as the training process is timeconsuming. Therefore, a fast and robust anomaly detectionmethod is required to be developed.In this paper, a statistical based robust anomaly detectionalgorithm, MEDIFF, was proposed to monitor the online busi-ness metrics in real time at eBay. Our goal was to detectand report both short term deviations, e.g., large spikes dur-ing several minutes, and long term deviations, e.g., a skew-ing during an hour caused by unstable Internet service pro-vided by telecommunication companies, in the metrics withone-minute sampling period in a short response time. TheMEDIFF technique was developed to meet the requirementof robust and fast response detection on streaming businessmetrics. The proposed method was based on a robust sta-tistical trend-seasonal decomposition model and the general-ized extreme Studentized deviate (ESD) many-outlier detec-tion technique [Rosner, 1983]. Specifically, the time serieswas decomposed into the trend and the seasonal componentswith a moving median smoothing and a short window week-over-week median values, respectively. Then, the residualwas computed by removing the trend and the seasonal com-ponents from the time series. Finally, the outliers were de-tected by applying the generalized ESD test on the residual.As applying the robust median metric and the statistical anal-ysis, the response of MEDIFF was both fast and unimpairedby anomalies that happened in history. The proposed MED-IFF algorithm was evaluated by implementing experimentson our labeled internal business production dataset.The pattern of the time series that we considered was sensi-tive to the effect of daylight saving time (DST) shift and hol- a r X i v : . [ s t a t . A P ] A ug days due to the small sampling period. We analyzed the per-formance of MEDIFF on the time series during the DST andholiday periods and proposed a solution to compensate theeffect of DST and holidays. Specifically, the seasonal trendcomponent during DST and the effect component during hol-idays were captured by moving median smoothing, respec-tively. Then, the DST seasonal was obtained by combiningthe week-over-week seasonal component with the seasonaltrend component under chosen weights. After removing thesecomponents (trend, DST seasonality, and holiday effect) fromthe time series, the residual was tested by ESD to detect theoutliers. The performance of the DST/holidays compensationmethod was also evaluated by our internal business metrics.The rest of the paper is organized as follows. In section2, previous works related to time series anomaly detectionwere discussed. In section 3, the proposed robust anomalydetection algorithm MEDIFF and the compensation methodfor DST and holidays were presented in details. The exper-iment implementation and results were described and evalu-ated in section 4. Conclusions were commented in section5. Current anomaly detection techniques used by large Inter-net companies are basically developed in three ways basedon respective requirements. The statistical SH-ESD algo-rithm developed by Twitter [Vallis et al. , 2014; Hochenbaum et al. , 2017] works for their cloud business metrics withstrong seasonality. The technique employs STL decompo-sition [Cleveland et al. , 1990] to determine the seasonal com-ponent of a given time series and then applies extreme Stu-dentized deviate (ESD) [Rosner, 1975] on the residual to de-tect the anomalies. The STL decomposition [Cleveland etal. , 1990] requires at least seven season periods to capturethe accurate seasonal component. However, this requirementresults in huge amount of data for the time series with oneminute sampling period and weekly seasonality in our appli-cation at eBay. Fbprophet developed by Facebook [Taylorand Letham, 2018] was proposed to solve challenges asso-ciated with large variety of time series. This method can beused to solve anomaly detection problems by fitting a regres-sion model with interpretable parameters and predicting thefuture behaviors.Supervised learning methods such as EGADS [Laptev etal. , 2015] and Opprentice [Liu et al. , 2015] take advantageof machine learning technique to improve the detection ac-curacy. Anomaly detectors are trained by using user feed-backs as labels and anomaly scores as features. However, thesmall percentage of anomalies in time series leads to unbal-anced classes in the training dataset. Furthermore, the labelsprovided by users might be with low accuracy and thereforeaggravates the unbalance. SR-CNN model proposed by Mi-crosoft is a combination of unsupervised algorithm with su-pervised learning model [Ren et al. , 2019]. Spectral residualprovides high accuracy labels to the convolutional neural net-works to further improve the output accuracy of the anomalydetection.State-of-the-art unsupervised learning models, such as
Time series DecomposeDST/Holiday? Residual ESD test AnomaliesMEDIFF
Figure 1: The schematic representation of the MEDIFF detector:Time series is first decomposed to extract trend component, seasonalcomponent, and DST shift or holiday effect components (if applica-ble). Then, the residual is tested by ESD technique to detect theanomalies (outliers).
Gaussian Mixture Model (GMM) [Laxhammar et al. , 2009],Support Vector Machine (SVM) based classifier [Erfani et al. ,2016], Long Short Term Memory networks (LSTM) [Malho-tra et al. , 2016], Variational autoencoders (VAE) [Zhang andChen, 2019] and its extension DONUT [Xu et al. , 2018], havebeen developed in recent years to overcome the challenge ofinaccuracy or lack of anomaly labels. These models basicallyfocus on learning normal patterns rather than learning abnor-mal patterns whenever possible. The anomalies detected bythese methods are generally based on the built-in anomalyscores, resulting in uninterpretable outliers.
In this section, the proposed robust anomaly detection tech-nique, MEDIFF, is presented in detail. The algorithm ismainly divided into the decomposition phase and the testphase as shown in Fig. 1. In general, the time series datais decomposed to extract the trend and the seasonal compo-nents. The components of DST and holiday effects, if appli-cable, are also extracted from the time series. Next, the resid-ual obtained by removing the extracted components (trend,seasonality, DST and holiday effects) from the time series istested by the generalized extreme Studentized deviate (ESD)many-outlier detection technique [Rosner, 1983] to detect theanomalies in the time series.
In general, time series data can be decomposed into threecomponents [Cleveland et al. , 1990][Kawasaki, 2006], i.e., y = µ + s + r (1)where y , µ , s , and r denote the time series raw data, the trendcomponent, the seasonal component, and the residual, respec-tively. To decrease the skewing of outliers and avoid the effectof missing or error data in time series, here, we propose theMEDIFF decomposition to capture the components in Eq. (1)by using the robust statistical metric—median. Specifically,given a time series y [ k ] with length (cid:96) y (i.e., k ∈ N (set ofnatural numbers) and (cid:54) k (cid:54) (cid:96) y , denotes as k ∈ N [1 , (cid:96) y ] in the rest of the paper), the trend component is decomposedby using moving median technique with the window lengthof w µ , i.e., µ [ k ] = median (cid:16) w µ − (cid:91) i =0 y [ k − i ] (cid:17) , for k ∈ N [ w µ , (cid:96) y ] (2) ime M e t r i cs Before DST After DST
Time M e t r i c Holidays Regular days (b)(a)
Figure 2: The effect of DST (a) and holidays (b) on time series met-rics, respectively. where the window length w µ is chosen in practice as thelength of one season (cid:96) s of the time series. As the moving me-dian in Eq. (2) uses history data with length w µ , the lengthof the trend component is (cid:96) y − w µ + 1 , i.e., the first w µ − elements are truncated. Then, the seasonal component is de-composed by first removing the trend component µ [ k ] fromthe time series y [ k ] , i.e., the detrended time series ˆ y [ k ]ˆ y [ k ] = y [ k ] − µ [ k ] , for k ∈ N [ w µ , (cid:96) y ] (3)and then evaluating the median of the data at the same win-dow area position [ − w s , w s ] in each season, i.e., s [ k ] = median (cid:16) n s − (cid:91) i =0 w s (cid:91) j = − w s ˆ y [ k ± i(cid:96) s + j ] (cid:17) , for k ∈ N [ w µ , (cid:96) y ] , and ( k ± i(cid:96) s + j ) ∈ N [ w µ , (cid:96) y ] (4)where n s and (cid:96) s denote the number of seasons in the de-trended time series ˆ y [ k ] and the length of each season, respec-tively. The residual r [ k ] is obtained by removing the trend andseasonal components from the time series, i.e., r [ k ] = y [ k ] − µ [ k ] − s [ k ] , for k ∈ N [ w µ , (cid:96) y ] (5)The MEDIFF decomposition is fast and computational sav-ing compared to the STL decomposition [Cleveland et al. ,1990] as no need to compute the local weight for each data[Cleveland, 1979] in STL. Moreover, MEDIFF decomposi-tion is robust to both single outliers and short term skewing.Specifically, as the seasonal component is unchangeable over-time (i.e., seasonal component in each season is the same),the short term skewing happened in only one season (due tonetwork attacks or unexpected short term sales promotion) isnot considered as part of the seasonal component, therefore,will be detected as anomalies. The time series related to customer behaviors such as loggingin and checking out is heavily affected by daylight saving time (DST) shift and holidays [Kamstra et al. , 2000][McEl-roy et al. , 2018]. Specifically, customers keep their daily ac-tivities according to their local time regardless of the DSTshift, i.e., customer activities happen at the similar time ofeach day/week before or after local DST shift. However, thetime series recorded on the server present different (laggingor advancing) patterns as the server time does not shift withDST. Thus, the time series in one season (e.g., one week) afterDST is not overlapped with that before DST (see Fig. 2(a)).Such a changing pattern caused by DST shift results in in-accurate seasonal component by MEDIFF. Similar effect byholidays exists. Specifically, customer activities may burst aspromotions provided by merchants during the Black Fridayor reduce as people are away from Internet to enjoy vacationsduring Christmas and New Year (see Fig. 2(b)).Due to the effects of DST and holidays, the decompositioncomponents (trend and seasonality) introduced in Subsec. 3.1cannot represent the feature of the time series. We proposeto include components in the decomposition to compensatethe effects of DST and holidays [Taylor and Letham, 2018],respectively. Specifically, the time series is decomposed asfollowing y = µ + s D + γe + (cid:15) (6)where γ ∈ { , } is a user defined constant ( γ = 1 when con-sidering the effect of holidays/events in the time series, other-wise γ = 0 ), and s D , e and (cid:15) denote the seasonal componentduring DST effect period (called as “ DST seasonal compo-nent ” in the rest of the paper), the effects of events (holidays,etc.) and the residual which is considered as Gaussian noise,respectively. For the time series y [ k ] with length (cid:96) y , the trendcomponent is first obtained as in Eq. (2). Then, the DSTseasonal component s D [ k ] is obtained as s D [ k ] = βs [ k ] + (1 − β )ˆ s [ k ] (7)where β ∈ [0 , is a user defined weight to balance the DSTeffect, s [ k ] is the seasonal component obtained by Eq. (4),and ˆ s [ k ] denotes the seasonal trend obtained by using movingmedian with a small window length ˆ w s , i.e., ˆ s [ k ] = median (cid:16) ˆ w s − (cid:91) i =0 ˆ y [ k − i ] (cid:17) , for k ∈ N [ w µ , (cid:96) y ] , and ( k − i ) ∈ N [ w µ , (cid:96) y ] (8)After removing the trend and the DST seasonal componentsfrom the time series, the event component is decomposed byusing moving median with the window length of w r , i.e., r [ k ] = y [ k ] − µ [ k ] − s D [ k ] , for k ∈ N [ w µ , (cid:96) y ] , then e [ k ] = median (cid:16) w r − (cid:91) i =0 r [ k − i ] (cid:17) , for k ∈ N [ w µ + w r − , (cid:96) y ] (9)Finally, the decomposed residual is computed as (cid:15) [ k ] = r [ k ] − γe [ k ] , with γ ∈ { , } (10)In general, the time series decomposition by MEDIFF canbe presented as Eq. (6)–Eq. (10) with γ = 0 and β = 1 whenhe effects of holidays/events and DST are near to negligiblelevel in the time series y , i.e., during normal days/weeks withno DST or holidays/events. The dates/time of DST shift andholidays are saved in the sever to indicate the detector to ap-ply appropriate parameters γ and β . Specifically, γ = 1 or β will be defined by the user ( β ∈ [0 , ) when time horizonof the time series y contains the date of holiday/event or theDST shift date/time, respectively. After the MEDIFF decomposition, the decomposed residual (cid:15) [ k ] for k ∈ N [ w µ + w r − , (cid:96) y ] will be checked by the general-ized ESD test [Rosner, 1983] to detect the unspecified numberof outliers. For completeness, we briefly describe below theimplementation of the generalized ESD test in the proposedMEDIFF detector.Given the upper bound on the suspected number of out-liers m , the generalized ESD test first computes correspond-ing statistic z -score of the decomposed residual (cid:15) for m itera-tions by removing the observation that maximizes the z -scorefrom the decomposed residual in each iteration, i.e., z i = max | (cid:15) i − ¯ (cid:15) i | σ i , i = 1 , , · · · , m (11)where (cid:15) i , ¯ (cid:15) i and σ i denote the remaining sample series of thedecomposed residual (cid:15) in the i th iteration, the mean and thestandard deviation of (cid:15) i in the i th iteration, respectively. Tofurther avoid the skewing effect of the outliers, we modifiedEq. (11) by replacing the mean ¯ (cid:15) i and the standard deviation σ i with the median ˜ (cid:15) i and median absolute deviation (MAD) ˜ σ i of (cid:15) i , i.e., ˜ z i = max | (cid:15) i − ˜ (cid:15) i | ˜ σ i , i = 1 , , · · · , m (12)where ˜ (cid:15) i = median ( (cid:15) i ) and ˜ σ i = median ( | (cid:15) i − ˜ (cid:15) i | ) .Then, the ESD computes the critical value for m times λ i = ( n − i ) t p,n − i − (cid:113) ( n − i − t p,n − i − )( n − i + 1) ,i = 1 , , · · · , m (13)where t p,ν is the p percentage point from the t -distribution with ν degrees of freedom and p = 1 − α n − i + 1) (14)with α the significance level. The number of anomalies isdetermined by finding the largest i such that ˜ z i > λ i . Then,the anomalies are determined at the corresponding observa-tions with statistic z -score index less than the number of theanomalies. In general, ˜ z i may not be always larger than thecritical value λ i before permanently dropping below it.The above MEDIFF algorithm (decomposition and theESD test) is summarized in Algorithm 1. In this section, we presented and discussed the performanceof the proposed MEDIFF algorithm, compared with two opensource anomaly detection algorithms—SH-ESD [Vallis et al. ,2014; Hochenbaum et al. , 2017] and DONUT [Xu et al. ,2018].
Algorithm 1
MEDIFF detector
Input : Time series y [ k ] Parameter : w µ , w s , ˆ w s , w r , β , γ Output : Anomalies in the time series y [ k ] Decomposition: Extract the trend component µ [ k ] by Eq. (2); Remove µ [ k ] from y [ k ] by Eq. (3); Extract the seasonal component s [ k ] and the seasonaltrend component ˆ s [ k ] by Eq. (4) and Eq. (8), respec-tively; if DST effect exists then Obtain the DST seasonal component s D [ k ] by Eq. (7)with β ∈ [0 , ; else Obtain s D [ k ] by Eq. (7) with β = 1 ; end if Remove µ [ k ] and s D [ k ] from y [ k ] and extract the eventcomponent e [ k ] by Eq. (9); if event/holiday effect exists then Obtain the residual (cid:15) [ k ] by Eq. (10) with γ = 1 ; else Obtain (cid:15) [ k ] by Eq. (10) with γ = 0 . end if ESD test:
Specify the suspected number of outliers m ; while i (cid:54) m do Calculate the z -score ˜ z i of the decomposed residual (cid:15) [ k ] by Eq. (12) and remove the maximum observationfrom (cid:15) [ k ] ; end while Calculate the critical values λ i by Eq. (13); Compare ˜ z i and λ i , and determine the outliers in (cid:15) [ k ] ; return the outliers as the anomalies in y [ k ] . We applied two groups of internal business metrics duringdifferent months (group A from September to the end of Oc-tober and group B from November to the end of December)with five pieces of metrics in each group. The metrics hadstrong weekly seasonality, i.e., the week-over-week seasonalcomponents were nearly overlapped. Each piece of metricscontained about 80,640 data points with time interval of oneminute, i.e., one data point per minute for about 60 days. Theanomalies in each metric were labeled manually by expertoperators. Any successive anomalies (skewing) were consid-ered as one anomaly and denoted by labeling the first one ofthese anomalies. By visualizing the metrics in each group, noeffect of DST or holidays was found on the metrics in group A , while both DST (in November) and holidays (Black Fri-day and Christmas) effected on the metrics in group B . Thosedates of DST and holidays were recorded in the system forMEDIFF decomposition on the metrics in group B .In the experiment, we implemented the MEDIFF detectionon time series batches with length 40,320 (i.e., 4 weeks) asthe application in our online real-time production environ-ment. Specifically, we partitioned each piece of metrics inboth group A and B into two batches with 40,320 data points able 1: Performance on metrics in group A (w/o DST and holiday) Algorithm Precision Recall F1-score Running time (s)MEDIFF ( γ = 0 , β = 1 ) 0.5361 0.8500 0.6575 0.9754SH-ESD 0.6215 0.4128 0.4961 11.4971DONUT 0.5572 0.875 0.6808 189.0562 (training) + 0.0678 Table 2: Performance on metrics in group B (with DST or holiday) Algorithm Precision Recall F1-score Running time (s)MEDIFF ( γ = 0 , β = 1 ) 0.5952 0.4832 0.5334 0.9562MEDIFF ( γ = 1 , β = 0 . ) 0.775 0.6603 0.7131 0.9847SH-ESD 0.5774 0.3846 0.4617 11.9975DONUT 0.5386 0.6240 0.5782 185.7478 (training) + 0.0737in each batch. Then, the time series in each batch in group A was decomposed and the corresponding residual was ob-tained by following the Algorithm 1 with fine-tuned param-eters w µ = 10080 , w s = 3 , ˆ w s = 30 , and w r = 60 , andweight γ = 0 and β = 1 , while for the batches in group B ,the same window length w µ , w s , ˆ w s , and w r were appliedand the weight were chosen as γ = 1 and β = 0 . . After thedecomposition, the residual of each batch was tested by ESDwith suspected number of outliers m ≈ (i.e., anomalyrate is . ) and the significance level α = 0 . .To evaluate the performance of MEDIFF, open sourceanomaly detection algorithms SH-ESD [Vallis et al. , 2014;Hochenbaum et al. , 2017] and DONUT [Xu et al. , 2018] werealso tested on the labeled metrics in both group A and B . ForSH-ESD, the algorithm was also applied on each time seriesbatch of length 4 weeks. For DONUT, we trained the mod-els for each piece of metrics with the time series in the first4 weeks (September) in group A and then tested the metricsin the next 4 weeks (October). To test the performance ofDONUT during DST and holiday periods, the model was firsttrained on the time series in October (i.e., metrics in the last4 weeks in group A ) and then applied to detect anomalies inthe two batches in group B (November and December), re-spectively. We tried our best to achieve good performance ofthe algorithms on the experiment dataset. We evaluated the performance of the algorithms by calculat-ing the commonly used machine learning metrics
Precision ( P ), Recall ( R ), and F1-score ( F ), i.e., P = TPTP + FP , R = TPTP + FN , F = 2 RPR + P (15)where TP, FP, and FN denote the number of positive valuesthat are predicted correctly, the number of negative values thatare wrongly predicted as positive, and the number of pos-itive values that are wrongly predicted as negative, respec-tively. Similar as the special labeling method in the exper-iment dataset, a set of successive anomalies detected by allthese three algorithms were condensed to one detected posi-tive and denoted as the first one in this set. This kind of pos-itive (condensed from successive anomalies) was consideredas a true positive if the detected positive was within 10 min- Date R e s i dua l DST parameter =1DST parameter =0.4
DST started
Figure 3: The residuals after decomposing a piece of time serieswith DST effect by β = 1 (blue) and β = 0 . (red), respectively. utes delay of the labeled positive in the experiment dataset,otherwise a false positive.The detection results of each batch metrics in both group A and B by all three algorithms were averaged and presentedin Table 1 and Table 2, respectively. Overall, the proposedMEDIFF was more effective (faster and better performance)on our internal seasonal metrics than other methods. First,MEDIFF detected more true positive anomalies than SH-ESD(higher recall value) on the metrics in both group A and B .Such an advantage was due to the accurate robust MEDIFFdecomposition, i.e., most of the prominent outliers in the de-composed residuals were true positives. However, by the STLdecomposition in SH-ESD, the residual might be contami-nated by the decomposed components (trend, seasonality),therefore, true positives might be left out. Then, the perfor-mance of both MEDIFF and SH-ESD declined on the metricseffected by DST (group B ) as neither of the decompositionmethods was suitable. The results were improved by apply-ing the MEDIFF decomposition for DST and holiday periodwith parameters γ = 1 and β = 0 . . As the seasonal pat-tern shifted by DST was captured by the MEDIFF decom-position with γ = 1 and β = 0 . , the residual was clearfrom distortion of the seasonal component (see Fig. 3). Thus,the ESD test captured more true positives and reported lessrong anomalies, resulting in increased precision and recallvalues. Finally, the DONUT algorithm had higher overallperformance on the metrics in both groups compared to theMEDIFF (with γ = 0 and β = 1 ) and the SH-ESD. How-ever, it may still be effected by DST or holiday as we foundseveral unexpected and unexplainable anomalies on the met-rics in group B , resulting in decreased precision value. We proposed a robust anomaly detection algorithm (MED-IFF) to monitor online business metrics in real time at eBay.A robust decomposition method using the moving median ofthe time series was first applied to decouple the trend, the sea-sonal, and the event components (if applicable). The effectsof DST and holidays on the business metrics were analyzedand corresponding decomposition method was also handledby the proposed MEDIFF. The residual after removing thedecomposed components was tested by the generalized statis-tics test algorithm (ESD) to detect anomalies. The proposedMEDIFF algorithm was tested on our internal business met-rics, compared with the SH-ESD and DONUT methods. Theresults were evaluated to demonstrate the proposed MEDIFFalgorithm.
References [Cleveland et al. , 1990] Robert B Cleveland, William SCleveland, Jean E McRae, and Irma Terpenning. Stl: aseasonal-trend decomposition.
Journal of official statis-tics , 6(1):3–73, 1990.[Cleveland, 1979] William S Cleveland. Robust locallyweighted regression and smoothing scatterplots.
Journalof the American statistical association , 74(368):829–836,1979.[Erfani et al. , 2016] Sarah M Erfani, Sutharshan Ra-jasegarar, Shanika Karunasekera, and Christopher Leckie.High-dimensional and large-scale anomaly detectionusing a linear one-class svm with deep learning.
PatternRecognition , 58:121–134, 2016.[Hochenbaum et al. , 2017] Jordan Hochenbaum, Owen SVallis, and Arun Kejariwal. Automatic anomaly detec-tion in the cloud via statistical learning. arXiv preprintarXiv:1704.07706 , 2017.[Kamstra et al. , 2000] Mark J Kamstra, Lisa A Kramer, andMaurice D Levi. Losing sleep at the market: The daylightsaving anomaly.
American Economic Review , 90(4):1005–1011, 2000.[Kawasaki, 2006] Yoshinori Kawasaki. A structural time se-ries model facilitating flexible seasonality. In
Proceed-ings of the Conference on Seasonality, Seasonal Adjust-ment and Their Implications for Short-term Analysis andForecasting . Citeseer, 2006.[Laptev et al. , 2015] Nikolay Laptev, Saeed Amizadeh, andIan Flint. Generic and scalable framework for automatedtime-series anomaly detection. In
Proceedings of the 21thACM SIGKDD International Conference on Knowledge Discovery and Data Mining , pages 1939–1947. ACM,2015.[Laxhammar et al. , 2009] Rikard Laxhammar, Goran Falk-man, and Egils Sviestins. Anomaly detection in sea traffic-a comparison of the gaussian mixture model and the kerneldensity estimator. In , pages 756–763. IEEE, 2009.[Liu et al. , 2015] Dapeng Liu, Youjian Zhao, Haowen Xu,Yongqian Sun, Dan Pei, Jiao Luo, Xiaowei Jing, andMei Feng. Opprentice: Towards practical and automaticanomaly detection through machine learning. In
Proceed-ings of the 2015 Internet Measurement Conference , pages211–224. ACM, 2015.[Malhotra et al. , 2016] Pankaj Malhotra, Anusha Ramakr-ishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agar-wal, and Gautam Shroff. Lstm-based encoder-decoderfor multi-sensor anomaly detection. arXiv preprintarXiv:1607.00148 , 2016.[McElroy et al. , 2018] Tucker S McElroy, Brian C Monsell,and Rebecca J Hutchinson. Modeling of holiday effectsand seasonality in daily time series.
Statistics , 1, 2018.[Ren et al. , 2019] Hansheng Ren, Bixiong Xu, Yujing Wang,Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing,Mao Yang, Jie Tong, and Qi Zhang. Time-seriesanomaly detection service at microsoft. arXiv preprintarXiv:1906.03821 , 2019.[Rosner, 1975] Bernard Rosner. On the detection of manyoutliers.
Technometrics , 17(2):221–227, 1975.[Rosner, 1983] Bernard Rosner. Percentage points for ageneralized esd many-outlier procedure.
Technometrics ,25(2):165–172, 1983.[Shipmon et al. , 2017] Dominique T Shipmon, Jason MGurevitch, Paolo M Piselli, and Stephen T Edwards. Timeseries anomaly detection; detection of anomalous dropswith limited features and sparse examples in noisy highlyperiodic data. arXiv preprint arXiv:1708.03665 , 2017.[Taylor and Letham, 2018] Sean J Taylor and BenjaminLetham. Forecasting at scale.
The American Statistician ,72(1):37–45, 2018.[Vallis et al. , 2014] Owen Vallis, Jordan Hochenbaum, andArun Kejariwal. A novel technique for long-term anomalydetection in the cloud. In { USENIX } Workshop on HotTopics in Cloud Computing (HotCloud 14) , 2014.[Xu et al. , 2018] Haowen Xu, Wenxiao Chen, NengwenZhao, Zeyan Li, Jiahao Bu, Zhihan Li, Ying Liu, You-jian Zhao, Dan Pei, Yang Feng, et al. Unsupervisedanomaly detection via variational auto-encoder for sea-sonal kpis in web applications. In
Proceedings of the 2018World Wide Web Conference , pages 187–196. InternationalWorld Wide Web Conferences Steering Committee, 2018.[Zhang and Chen, 2019] Chunkai Zhang and YingyangChen. Time series anomaly detection with variationalautoencoders. arXiv preprint arXiv:1907.01702arXiv preprint arXiv:1907.01702