Novelty Detection: Deep AI Algorithm, Machine Learning

What is Novelty Detection?

As the name implies, novelty detection is the identification of novel, or unusual, data from within a dataset. Often, these outliers, also called anomalies, are discovered due to their differences from the rest of the data. Sometimes, however, novelty detection algorithms need to be tuned to look for not just single incidents of unusual data, but rather groups or bursts of unusual information. This alternative is called cluster analysis and is a common technique in bank fraud algorithms to track patterns of suspicious activity.

How does Novelty Detection work?

Novelty detection works by identifying outlying data and information. In order to properly classify this data, a machine learning algorithm may be trained exclusively on correct data. This way, any time outlying or unusual data is detected, it is flagged as a novel instance. As referenced above, however, sometimes the algorithm is trained to look for groupings of unusual data, known as clusters.

Novelty Detection and Machine Learning

Source

Novelty detection is specified use of machine learning algorithms. While a single function may be applied in the detection of novel or unusual data, often layers of functions are applied consecutively, each with a specified purpose. For example, a layer of the network can be trained to identify all anomalous data, while an additional layer atop the other can be trained to decide whether the anomalous data is relevant to the networks goal, or outlying and irrelevant. This way, neural networks can filter out unnecessary novelty detections and be trained to look only for specific instances of anomalous data. 

This article was first published on: https://deepai.org/machine-learning-glossary-and-terms/novelty-detection