fasttextlt 0.1.0

A pure Python FastText model reader

FastText lite

A pure[^1] Python FastText model reader, to ensure that FastText model stay usable for as long as possible.

This is slower than the original FastText, for some methods by orders of magnitude. It's the price to pay for not using Cython or compiled code. These methods are also not usually bottlenecks in standard uses of FastText.

FastText has been in maintenance mode for some time, with no indication that it will ever change. In the absence of further releases, it will stay stuck in increasingly obsolete Python versions and get increasingly hard to rebuild. Gensim has a working re-implementation that is easier to use, but it too has a relatively slow release pace and does not necessarily keep up with the rest of the Python ecosystem. This project provides a path to keep using FastText models for longer and at low cost, by extracting the relevant parts from Gensim and converting the Cython parts back to pure Python (and optimising them as well as I can). So far it supports loading the binary models and use them for word embedding purposes.

If you want to train new models, I would advise starting a new library from scratch to avoid being stuck in FastText's historical cruft, and take inspiration from Floret instead.

[^1]: The only non-pure Python dependency is NumPy 2, which can reasonably be expected to be available.

Licence

This software is released under the LGPL v2.1 Licence, see LICENCE.md for the details.