gensim.interfaces.TransformedCorpus - How use?

问题

I'm relative new in the world of Latent Dirichlet Allocation. I am able to generate a LDA Model following the Wikipedia tutorial and I'm able to generate a LDA model with my own documents. My step now is try understand how can I use a previus generated model to classify unseen documents. I'm saving my "lda_wiki_model" with

id2word =gensim.corpora.Dictionary.load_from_text('ptwiki_wordids.txt.bz2')

    mm = gensim.corpora.MmCorpus('ptwiki_tfidf.mm')

    lda = gensim.models.ldamodel.LdaModel(corpus=mm, id2word=id2word, num_topics=100, update_every=1, chunksize=10000, passes=1)
    lda.save('lda_wiki_model.lda')

And I'm loading the same model with:

new_lda = gensim.models.LdaModel.load(path + 'lda_wiki_model.lda') #carrega o modelo

I have a "new_doc.txt", and I turn my document into a id<-> term dictionary and converted this tokenized document to "document-term matrix"

But when I run new_topics = new_lda[corpus] I receive a 'gensim.interfaces.TransformedCorpus object at 0x7f0ecfa69d50'

how can I extract topics from that?

I already tried

`lsa = models.LdaModel(new_topics, id2word=dictionary, num_topics=1, passes=2)
corpus_lda = lsa[new_topics]
print(lsa.print_topics(num_topics=1, num_words=7)

and

print(corpus_lda.print_topics(num_topics=1, num_words=7) `

but that return topics not relationed to my new document. Where is my mistake? I'm miss understanding something?

**If a run a new model using the dictionary and corpus created above, I receive the correct topics, my point is: how re-use my model? is correctly re-use that wiki_model?

Thank you.

回答1:

I was facing the same problem. This code will solve your problem:

new_topics = new_lda[corpus]

for topic in new_topics:

      print(topic)

This will give you a list of tuples of form (topic number, probability)

回答2:

From the 'Topics_and_Transformation.ipynb' tutorial prepared by the RaRe Technologies people:

Converting the entire corpus at the time of calling corpus_transformed = model[corpus] would mean storing the result in main memory, and that contradicts gensim’s objective of memory-independence.

If you will be iterating over the transformed corpus_transformed multiple times, and the transformation is costly, serialize the resulting corpus to disk first and continue using that.

Hope it helps.

来源：https://stackoverflow.com/questions/45317151/gensim-interfaces-transformedcorpus-how-use

标签

gensim

lda