# Naive Bayes classification

Naive Bayes classifiers can be used for example, to mark email (spam or not), to classify news articles, to check a piece of text for it containing expressions of positive or negative emotions, and in face recognition applications.

## The math

Naive Bayes classifiers are based on the Bayes Theorem:

with

Posterior: The probability of our hypothesis being true given the data collected. | |

Prior: The probability of our hypothesis being true before collecting data. | |

Likelihood: Probability of collecting this data when our hypothesis is true. | |

Marginal: The probability of collecting this data under all possible hypotheses. |

May seem complex. It isn't.

## The infamous dot example

There are 15 orange dots and 30 green dots in this set of dots.

Twice as many green dots as orange dots. The priors for Green resp Orange:

When a new dot appears, we'd expect it to be twice as likely to be green as orange.

Suppose we wish to classify a new object, a white dot. The dots seem neatly clustered, so it is reasonable to assume that the more green (or orange) dots in the vicinity of a new dot X, the more likely it is that the new dot X belongs to that particular colour. To measure this likelihood, draw a circle around X which encompasses a number (to be chosen a priori) of dots irrespective of class labels. Then calculate the number of dots in the circle belonging to each class label. Calculating likelihoods:

The prior probabilities indicate that X may belong to Green (because there are twice as many Green dots compared to Orange dots) and these likelihoods indicate otherwise; that it is more likely for class membership of X to be Orange (because there are more Orange dots in the vicinity of X than Green dots).

In Bayesian analysis, the final classification is produced by combining both sources of information, the prior and the likelihood, to form a posterior probability using the so-called Bayes' rule (the equation at the start):

`= 2/3 x 1/30 = 1/45`

`= 1/3 x 4/15 = 4/45`

In Bayesian analysis the White dot X is classified as Orange since its class membership achieves the largest posterior probability. Note that it is white though. And think facebook.