Gensim

Gensim is a Python library for topic modelling , document indexing and similarity retrieval with large corpora. Target audience is the natural language processing (NLP) and information retrieval (IR) community. ⚠️ Gensim is in stable maintenance mode: we are not accepting new features, but bug and documentation fixes are still welcome! ⚠️ Features All algorithms are memory-independent w.r.t. the corpus size (can process input larger than RAM, streamed, out-of-core), Intuitive interfaces easy to plug in your own input corpus/datastream (trivial streaming API) easy to extend with other Vector Space algorithms (trivial transformation API) Efficient multicore implementations of popular algorithms, such as online Latent Semantic Analysis (LSA/LSI/SVD) , Latent Dirichlet Allocation (LDA) , Random Projections (RP) , Hierarchical Dirichlet Process (HDP) or word2vec deep learning . Distributed computing : can run Latent Semantic Analysis and Latent Dirichlet Allocation on a cluster of computers. Extensive [documentation and Jupyter Notebook tutorials]. If this feature list left you scratching your head, you can first read more about the [Vector Space Model] and [unsupervised document analysis] on Wikipedia. Or, if you have instead downloaded and unzipped the [source tar.gz] package: Gensim is being continuously tested under all supported Python versions. Support for Python 2.7 was dropped in gensim 4.0.0 – install gensim 3.8.3 if you must use Python 2.7. How come gensim is so fast and memory efficient? Isn’t it pure Python, and isn’t Python slow and greedy? Memory-wise, gensim makes heavy use of Python’s built-in generators and iterators for streamed data processing. Memory efficiency was one of gensim’s [design goals], and is a central feature of gensim, rather than something bolted on as an afterthought. Raise bugs on Github but please make sure you follow the issue template . Issues that are not bugs or fail to provide the requested details will be closed without inspection. [design goals]: [RaRe Technologies]: [rare tech]: //rare-technologies.com [Talentpair]: [citing gensim in academic papers and theses]: [documentation and Jupyter Notebook tutorials]: [Vector Space Model]: [unsupervised document analysis]: [NumPy]: [linking NumPy to a BLAS library]: [ATLAS]: [OpenBLAS]: [source tar.gz]: [documentation]: