Python NLTK Saving Classifier

Python NLTK Saving Classifier


Training classifiers and machine learning algorithms can take a very long time, especially if you're training against a larger data set.

Ours is actually pretty small. Can you imagine having to train the classifier every time you wanted to fire it up and use it?

we can do is use the Pickle module to go ahead and serialize our classifier object, so that all we need to do is load that file in real quick.

how do we do this?

The first step is to save the object. To do this first, you need to import pickle at the top of your script, then, after you have trained with .train() the classifier, you can then call the following lines:

This opens up a pickle file, preparing to write in bytes some data. Then, we use pickle.dump() to dump the data.

The first parameter to pickle.dump() is what are you dumping, the second parameter is where are you dumping it.

After that, we close the file as we're supposed to, and that is that, we now have a pickled, or serialized, object saved in our script's directory!

Next, how would we go about opening and using this classifier?

The .pickle file is a serialized object, all we need to
do now is read it into memory, which will be about as quick as reading any
other ordinary file. To do the following code:

Here, we do a very similar process. We open the file to

read as bytes. Then, we use pickle.load() to load the file, and we save the
data to the classifier variable.

Then we close the file, and that is that. We now have the
same classifier object as before!

Now, we can use this object, and we no longer need to train our classifier every time we
wanted to use it to classify.

While this is all fine and dandy, we're probably not too content with the 60-75% accuracy

we're getting.

What about other classifiers?

Turns out, there are many classifiers, but we need the scikit-learn (sklearn) module.

 Luckily for us, the people at NLTK recognized the value of incorporating the sklearn module into NLTK.


Using saving the classifier we have to train classifiers at the one time only. Then if we have to use that classifier then we just have to fire that classifier.