The Naive Bayes Classifier
Classifiers based on Bayesian methods utilize training data to calculate an observed
probability of each class based on feature values. When the classifier is used later on
unlabeled data, it uses the observed probabilities to predict the most likely class for the
new features. It's a simple idea, but it results in a method that often has results on par
with more sophisticated algorithms.
Begin Exercise.
Getting the required packages:
First, down the required packages for the classifier. The NLTK package is widely used for the purposes of Natural Language Processing in Python. You can read more about NLTK here. TextBlob allows simpler interface for the same. We are going to use both of them to achieve our goal. You can download them from your Linux Shell using
pip.
sudo pip install nltk
sudo pip install textblob
After your installation of these packages is done, we are now ready to code.
Start coding!
1. Collect training data set
All machine learning algorithms require training. We all know that Naive Bayes classifies the unclassified text by learning from past. Luckily there is a well shaped data set which actually contains a messages tagged with spam vs ham. You can get it from
here. This is how the first few lines of the data set look like:
ham Even my brother is not like to speak with me. They treat me like aids patent.
ham As per your request 'Melle Melle (Oru Minnaminunginte Nurungu Vettam)' has been set as your callertune for all Callers. Press *9 to copy your friends Callertune
spam WINNER!! As a valued network customer you have been selected to receivea 900
spam Had your mobile 11 months or more? U R entitled to Update to the latest colour Free! Call The Mobile Update Co FREE on 08002986030
ham I'm gonna be home soon and i don't want to talk about this stuff anymore tonight, k? I've cried enough today.
As you can see, there is the type on the left side and separated by a tab, there is a message. This is going to be o
2. Import the libraries
There are the libraries that we need to import. These include few from the two we have downloaded and some more to perform some debugging tasks on our Python code.
import random #to shuffle the training set
import time #to time learning and and classification
from textblob import TextBlob #to tokenize our sentences into words
from nltk.corpus import stopwords #to remove unwanted stop words
from text.classifiers import NaiveBayesClassifierur training data set.
3. Creating training tuples for the classifier
The NaiveBayesClassifier function accepts tuples in the order, (body, class). Where body is the text that is to be classified and class referes to the class of text. For this experiment, we are going to classify each word of the message with respect to its class. Making our text of tuples look like:
list_of_tuples = [
('call','ham')
('free', 'spam')
('word','ham'
]
To achieve this, I have written a function that reads all lines of the tab separated file and generated list of tuples. The algorithm that I have followed to achieve this is:
initialize an empty list of tuples
for each line in file:
split the line using the tab delimiter
extract the second part of the split as sentence
split sentence into words
for each word in sentence:
if word is not a stop word or a number:
create tuple (word,type)
append tuple to the list
return the list
To work with sentences and words, we are going to use the NLTK corpus and TextBlob APIs. Here is how the function is going to look like:
def get_list_tuples(read_file):
list_tuples = []
with open(read_file,"r") as r:
c=0
for line in r:
tabsep = line.strip().split('\t')
msg = TextBlob(tabsep[1])
try:
words=msg.words
except:
continue
for word in words:
if word not in stopwords.words() and not word.isdigit():
list_tuples.append((word.lower(),tabsep[0]))
c+=1 #limiting factor begins
if c==500:
break #limiting factor ends
return list_tuples
For the sake of simplicity, we have created a TextBlob for every sentence that makes it easier to extract words and save few lines of code to remove punctuation and all. Also, if we encounter some words that cannot be rendered by the TextBlob, we just skip the line, and continue scanning our data set. You might have also observed that I have introduced a variable
c that actually limits the number of lines that we are going to import from our data set. All these modifications are made just for simplicity and so that the code be executed faster. You can increase this counter or remove that part of code to let the classifier train over all the 5700+ lines present in the data set.
Now, we will have to call the function with the parameter being the path to file where our data set resides. For me, I have called the function like this:
a = time.time()
entire_data = get_list_tuples("~/Documents/DataSci/DataSets/sms/SMSSpamCollection")
print "It took "+str(time.time()-a)+" seconds to import data" #10.031548
As you might have seen I have also calculated the amount of time required to import data from the file and create our entire_data structure. This is helpful is measuring the performance of our program. For me, it took around 10 seconds to import 500 lines from the data set. You can also omit those lines if you are not bothered about it.
4. Some shuffling required
Before beginning our training, if we look at the tuples we generated, most of them are partly sequentially arranged. For example, many 'ham' classified text are contiguous. We will now simply shuffle our array using the random.shuffle() function in Python. This will enable us to create the train and test data sets for our classifiers to be created from the same list.
random.seed(1)
random.shuffle(entire_data) #shuffling the data
train = entire_data[:250] #list of training tuples for classifier
test = entire_data[251:500] #list of tuples for testing the accuracy of the classifier
5. Train the classifier!
Now that everything is ready, we can train the classifier using NaiveBayesClassifier present in the text.classifiers package. Yet again, I have introduced timers to check how much time was required for the training to be completed.
a = time.time()
cl = NaiveBayesClassifier(train)
print "It took "+str(time.time()-a)+" seconds to train data" #7.003
Testing the classifier!
Now, we will test the classifier using the testing set that we have created earlier.
accuracy = cl.accuracy(test)
print "accuracy: "+str(accuracy)
Well, when I tried it, I got an accuracy of around 78%.
You can also play around with the classifier by typing in your favorite messages and see how they are classified. All you have to do is to create the tuple of (message,class) as I have told you people before.
To classify individual sentences and getting the result, you can use this one line of code:
print cl.classify("Hey bud, what's up") #ham
print cl.classify("Get a brand new mobile phone by being an agent of The Mob! Plus loads more goodies! For more info just text MAT to 87021") #spam
The entire python file:
Conclusion:
We have designed a simple SPAM vs HAM classifier using Naive Bayes Classification algorithm. You can use this tutorial to develop various other systems of classifications. There are many data sets present on
this website which can be used for classification purposes. Using similar algorithm that we have used here, you can also use various sentiment analysis databases to classify a given sentence to be positive or negative. It is just that as our classes here were
spam and
ham, for that experiment, they would be
positive or
negative. Do you have something more to add to this? Please let me know your suggestions and comments below. Thank you.
More coding tutorials at
Let's Code
You can also connect me on various social networks, links are on top of the page. Also, if you like it, share it :)
