News + Publications

There are many instances where you might want to identify anomalies in data, from quality assurance to process control. And there are a variety of different methods that can be used for anomaly detection. These include:

• Statistical Methods – which involve modelling the typical data distributions and identifying instances that differ from these expected patterns. This can range from simple approaches like the z-score to more complex methods like Markov Chain Monte Carlo (MCMC) modelling.
• Machine Learning (ML) and Deep Learning Methods – where ML algorithms are trained to identify anomalies by learning expected patterns from labelled data (supervised learning) or from unlabeled data (unsupervised learning). Some popular ML algorithms could include tree-based models (e.g., boosted-trees), autoencoders, or support vector machines.
• Clustering Methods – where clustering algorithms group similar instances together. Anomalies are identified as instances which do not belong to any cluster or which form a sparse cluster.
• Time series analysis – where techniques such as moving average, and autocorrelation models are used to identify outliers over time.
• Rule Based method – where specific rules and thresholds are defined to detect anomalies. This can sometimes be a simple and effective way to identify anomalies; however, creating the rules typically requires some expert knowledge about the data or the processes behind the data.

Which methods to use for a given application depends on the characteristics of the data, type of anomalies being considered, quantity of data and of labelled data, and the overall objectives of the analysis. It can often be beneficial to use more than one method to achieve better anomaly detection results.