Please enable JavaScript.

Coggle requires JavaScript to display documents.

Anomaly Detection Techniques (todo - Techniques for anomaly detection…

- - - - W. Wu, X. Cheng, M. Ding, K. Xing, F. Liu, and P. Deng, Localised Outlying and Boundary Data Detection in Sensor Networks: Use spatial correlation of readings in neighbouring sensor nodes to distinguish outlying sensors and event boundaries. Accuracy not usually high as these techniques (comparing the difference of a node to its neighbouring node's median then being too far above a set threshold) ignore temporal correlation of sensor readings.
      - Y. Hida, P. Huang, and R. Nishtala, Aggregation Query under uncertainty: Make max and avg more reliable under faulty sensor readings and failed nodes. Relies on spatial correlation of sensor data. One dimension data and too much memory required.
    - - M.C. Jun, H. Jeong, and C.C.J. Kuo, Distributed Spatio-Temporal Outlier Detection in Sensor Networks: uses a technique called symmetric α-stable (SαS) distribution to model outliers in form of impulsive noise. Utilises spatio-temporal correlations of sensor data to locally detect outliers. Each node in a cluster detects and corrects temporal outliers by comparing predicted data and the sensing data. It reduces commincation cost and cmputational cost as cluster heads carry mostof the computation taks. May not be suitable for real data as the cluster based structure cant cope with typical dynamic changes.
      - DBN with new nodesrelated to time, which enables the detection of time-dependentbehavioral anomalies. Four types of anomalies, Type 1, 2, and 3 are point anomalies, while Type 4 is a contextual anomaly: 1—Spatial anomaly, 2—Timing anomaly, 3—Duration anomaly, 4—Sequence anomaly. Uses maximum-likelihood estimation and Laplace smoothing to learn histroical data then online anomaly detection (based on probabilities). Chun Zhu, Wearable Sensor-Based Behavioral Anomaly Detection in Smart Assisted Living Systems, 2015
  - - - T. Palpanas, D. Papadopoulos, V. Kalogeraki, and D..Gunopulos,Distribute Deviation Detection in Sensor Networks: Online identification of outliers in streaming sensor data, requires no a priori data, uses kernel estimator to approximate underlying distribution of sensor data. Nodes locally find outliers than deviate signifigantly from the model of estimated distribution. Outliers = values in neighbourhood are less than a user-specified threshold. High dependancy on threshold, not suitable for mulitdimensional data. no maintenace considered
      - S. Subramaniam, T. Palpanas, D. Papadopoulos, V.Kalogerakiand, and D. Gunopulos, Online Outlier Detection.in Sensor Data using nonparametric
        Models: Solve the problem of Palpanas (single threshold for multidimensional data) and Sheng (maintaining the built model) by proposing two global outlier detection techniques: 1.) allows each node to locally identify outlier thens transmit outliers to its parent to be checked until the sink identifies all global outliers. 2.) each node employs LOCI to locally detect global outliers by having a copy of global estimator model from the sink. Both have high accuracy in terms of data distribution and detection rate using low memory and message transmission. Still cant detect spatial outliers.
    - - One dimension data considered only. B. Sheng, Q. Li, W. Mao, and W. Jin, Outlier Detection in Sensor
        Networks: Identify global outliers from data applications of sensor networks. Sink uses histogram info to extract data distribution from the network and filters non-outliers using a threshold distance or rank among all outliers to detect outliers.
      - For each hour of the day, the average proportion of time spent within a specific room is estimated. Large deviations from this pro- portion are considered as abnormal - G. Virone, N. Noury, J. Demongeot, “A system for automatic measurement of circadian activity deviations in telemedicine” IEEE Transactions 2002, and G. Virone, M. Alwan, S. Dalal, S. Kell, B. Turner B., J.A. Stankovic, R. Felder, “Behavioral Patterns of Older Adults in Assisted Living” IEEE Transactions
      - The average time spent in each room is estimated for each day (not each hour) and they also propose to monitor other variables such as the daily distance moved by a person - S. Ohta, H. Nakamoto, Y. Shinagawa, T. Kishimoto, “Home Telehealth: Connecting Care Within the Community”, Medical telematics, 2006
      - Probabilistic Model of Behaviour (PMB) based. A set of attributes or features is associated with each occurrence of this type of activity and a probabilistic model of this type activity is learned from historical data. Deviation from the model is then considered as an anomaly - this approach uses GMM with parameters chosen using Expectation Maximization algorithm. Features: Start time, duration, weekdy\weekend, Activity Level (Number of sensors triggered during the observation). Rule-based algorithms used to detect the acivities. e.g. sleeping - when sensor is on and bed is occupied for more than 30 mins the person is sleeping. This pproach builds a model for each type of activity - sleeping and watching tv. Modelling experiemnt: GMM parameters (means, variances and weights) are learned using the 26 weeks of training data. - Modelling of behavioural patterns for abnormality detection in the context of lifestyle reassurance
        Fabien Cardinaux, 2008
- - - - Elnahrawy and
        Nath [24] - detection of faulty sensors
    - - Janakiram et al. [23]: identifying local outliers in streaming sensor data.
    - - Hill et al. [43]: local outliers in environmental data streams.
  - - - Algorithms according to Goldstein and Uchida, 2016 (comparative eval)
        
        K-Nearest-neighbour global anomaly detection - knn global unsupervised anomaly detections algorithm - NOT TO BE CONFUSED WITH Knn CLASSIFICATION -
        
        Local Outlier Factor LOF: most well known local anomaly detection algorithm: basically a ratio of local densities. Low density = larger score.
        
        1.) knn must be found for each record x
        
        2.) Using these knn, local density for a record is estimated using the local reachability density (LRD). (In highly dense clusters euclidean distance is used (very rare)).
        
        3.) LOF is computed by comparing the LRDof a record with the LRD f it's k neighbours.
        
        Note: if local outliers are not of interest, alot of false alarms are found. K setting is crucial for the algorithm.
        
        Connectivity-based Outlier Factor: similar to LOF but density estimation is calculated differently. Uses shortest path approach (chaining distance) which is a mini-sum of all the distances connecting k neighbours and the instance.
        
        Clustering based
        
        Influenced outlierness (INFLO). LOF can fail scoring instances at the borders of clusters. INFLO uses knn and nearest neoghbourood set (which records are stored for crrent record and neighbour). INFLO score combines both neighbourhood sets.
        
        Statisical
        
        Subspace techniques
        
        TODO - Local Outlier Probability (LoOP), Local Correlation Integral (LOCI), Approximate Local Correlation Integral (aLOCI), Cluster-based Local Outlier Factor (CBLOF\uCBLOF), Local Density Cluster-based Outlier Factor (LDCOF), Clustering-based Multivariate Gaussian Outlier Score (CMGOS), Histogram-based Outlier Score (HBOS), One class support vector machine (OCSVM), Robust Component Principal Analysis (rPCA)