System & Methods to detect Anomalies in data using Machine Learning Techniques

Author(s): Prashik Waghmare, Vivek Pawar

Abstract

Anomaly detection has received significant attention from the researcher/practitioner communities since few years due to its ability to identify anomalous incidents in data which could help to mitigate future risks. Recent advancements in Big Data technologies have enabled to process huge data. While working on Big Data (Sensor, Transactional, etc.), Traditional Anomaly detection techniques have certain limitations which can be resolved with state-of-the-art machine learning models. In this study, we give a thorough assessment of the literature on anomaly detection methods. There are both conventional and machine learning techniques which can be applied to several data types. We describe the most popular supervised machine learning algorithms, i.e., KNN, Neural Networks and unsupervised i.e., Isolation Forests and Support Vector Machines. We also present comprehensive case studies about the use of anomaly detection in banking and manufacturing sectors. In banking, Anomaly detection can be effectively used to detect fraudulent activities such as Credit card fraud, Anti Money Laundering, etc. In Manufacturing it can be used to detect abnormal behavior of machines using sensor-based data. Anomaly detection is extremely critical today to avoid potential future risks with early identification of anomaly as businesses deal with trillions of data points and millions of metrics.

Picture of Association of Data Scientists

Association of Data Scientists

The Chartered Data Scientist Designation

Achieve the highest distinction in the data science profession.

Elevate Your Team's AI Skills with our Proven Training Programs

Strengthen Critical AI Skills with Trusted Generative AI Training by Association of Data Scientists.

Our Accreditations

Get global recognition for AI skills

Chartered Data Scientist (CDS™)

The highest distinction in the data science profession. Not just earn a charter, but use it as a designation.

Certified Data Scientist - Associate Level

Global recognition of data science skills at the beginner level.

Certified Generative AI Engineer

An upskilling-linked certification initiative designed to recognize talent in generative AI and large language models

Join thousands of members and receive all benefits.

Become Our Member

We offer both Individual & Institutional Membership.