Skip to content

Add smart information retrieval system for TFIDF #1785

@markroxor

Description

@markroxor

https://en.wikipedia.org/wiki/SMART_Information_Retrieval_System

The current TFIDF model uses natural TF and IDF for computing TFIDF. The idea is to try various transformation like logarithmic, augmented,boolean etc. before computing the vectors.

More about this - http://www.cs.odu.edu/~jbollen/IR04/readings/article1-29-03.pdf and https://nlp.stanford.edu/IR-book/pdf/06vect.pdf

Will send a PR tomorrow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    difficulty mediumMedium issue: required good gensim understanding & python skillsfeatureIssue described a new featurewishlistFeature request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions