Previous topic

models.lsimodel – Latent Semantic Indexing

Next topic

models.rpmodel – Random Projections

models.tfidfmodel – TF-IDF model

class gensim.models.tfidfmodel.TfidfModel(corpus, id2word=None, normalize=True)

Objects of this class realize the transformation between word-document co-occurence matrix (integers) into a locally/globally weighted matrix (positive floats).

This is done by combining the term frequency counts (the TF part) with inverse document frequency counts (the IDF part), optionally normalizing the resulting documents to unit length.

The main methods are:

  1. constructor, which calculates IDF weights for all terms in the training corpus.
  2. the [] method, which transforms a simple count representation into the TfIdf space.
>>> tfidf = TfidfModel(corpus)
>>> print = tfidf[some_doc]
>>> tfidf.save('/tmp/foo.tfidf_model')

Model persistency is achieved via its load/save methods.

normalize dictates whether the resulting vectors will be set to unit length.

initialize(corpus)
Compute inverse document weights, which will be used to modify term frequencies for documents.
classmethod load(fname)
Load a previously saved object from file (also see save).
save(fname)
Save the object to file via pickling (also see load).