Skip to main content

Automated subject indexing and classification tool

Project description

DOI License Container image CI/CD codecov Scrutinizer Code Quality Code Climate OpenSSF Scorecard codebeat badge CodeQL Quality Gate Status docs Code style: black Open in GitHub Codespaces

Annif is an automated subject indexing toolkit. It was originally created as a statistical automated indexing tool that used metadata from the Finna.fi discovery interface as a training corpus.

This repository contains a rewritten production version of Annif based on the prototype.

Finto AI is a service based on Annif; see a 🤗 Hugging Face Hub collection of the models that Finto AI uses.

Basic install

Annif is developed and tested on Linux. If you want to run Annif on Windows or Mac OS, the recommended way is to use Docker (see below) or a Linux virtual machine.

You will need Python 3.9-3.12 to install Annif.

The recommended way is to install Annif from PyPI into a virtual environment.

python3 -m venv annif-venv
source annif-venv/bin/activate
pip install annif

Start up the application:

annif

See Getting Started for basic usage instructions and Optional features and dependencies for installation instructions for e.g. fastText and Omikuji backends and for Voikko and spaCy analyzers.

Shell compeletions

Annif supports tab-key completion in bash, zsh and fish shells for commands and options and project id, vocabulary id and path parameters.

To enable the completion support in your current terminal session use annif completion command with the option according to your shell to produce the completion script and source it. For example, run

source <(annif completion --bash)

To enable the completion support in all new sessions first add the completion script in your home directory:

annif completion --bash > ~/.annif-complete.bash

Then make the script to be automatically sourced for new terminal sessions by adding the following to your ~/.bashrc file (or in some alternative startup file):

source ~/.annif-complete.bash

For details and usage for other shells see Click documentation.

Docker install

You can use Annif as a pre-built Docker container image from quay.io/natlibfi/annif repository. Please see the wiki documentation for details.

Development install

A development version of Annif can be installed by cloning the GitHub repository. Poetry is used for managing dependencies and virtual environment for the development version.

See CONTRIBUTING.md for information on unit tests, code style, development flow etc. details that are useful when participating in Annif development.

Installation and setup

Clone the repository.

Switch into the repository directory.

Install pipx and Poetry if you don't have them. First pipx:

python3 -m pip install --user pipx
python3 -m pipx ensurepath

Open a new shell, and then install Poetry:

pipx install poetry

Poetry can be installed also without pipx: check the Poetry documentation.

Create a virtual environment and install dependencies:

poetry install

By default development dependencies are included. Use option -E to install dependencies for selected optional features (-E "extra1 extra2" for multiple extras), or install all of them with --all-extras. By default the virtual environment directory is not under the project directory, but there is a setting for selecting this.

Enter the virtual environment:

poetry shell

Start up the application:

annif

Demo install in Codespaces

Annif can be tried out in the GitHub Codespaces. Just open a page for configuring a new codespace via the badge below, start the codespace from the green "Create codespace" button, and a terminal session will start in your browser with the contents of the Annif-tutorial repository:

Open in GitHub Codespaces

Getting help

Many resources are available:

Publications / How to cite

See below for some articles about Annif in peer-reviewed Open Access journals. The software itself is also archived on Zenodo and has a citable DOI.

Citing the software itself

See "Cite this repository" in the details of the repository.

Annif articles

  • Golub, K.; Suominen, O.; Mohammed, A.; Aagaard, H.; Osterman, O, 2024. Automated Dewey Decimal Classification of Swedish library metadata using Annif software. Journal of Documentation, in press. https://doi.org/10.1108/JD-01-2022-0026
    See BibTex
    @article{golub2024annif,
      title={Automated Dewey Decimal Classification of Swedish library metadata using Annif software},
      author={Golub, Koraljka and Suominen, Osma and Mohammed, Ahmed Taiye and Aagaard, Harriet and Osterman, Olof},
      journal={J. Doc.},
      year={in press},
      doi = {10.1108/JD-01-2022-0026},
      url={https://www.emerald.com/insight/content/doi/10.1108/JD-01-2022-0026},
    }
    
  • Suominen, O.; Inkinen, J.; Lehtinen, M., 2022. Annif and Finto AI: Developing and Implementing Automated Subject Indexing. JLIS.It, 13(1), pp. 265–282. URL: https://www.jlis.it/index.php/jlis/article/view/437
    See BibTex
    @article{suominen2022annif,
      title={Annif and Finto AI: Developing and Implementing Automated Subject Indexing},
      author={Suominen, Osma and Inkinen, Juho and Lehtinen, Mona},
      journal={JLIS.it},
      volume={13},
      number={1},
      pages={265--282},
      year={2022},
      doi = {10.4403/jlis.it-12740},
      url={https://www.jlis.it/index.php/jlis/article/view/437},
    }
    
  • Suominen, O.; Koskenniemi, I, 2022. Annif Analyzer Shootout: Comparing text lemmatization methods for automated subject indexing. Code4Lib Journal, (54). URL: https://journal.code4lib.org/articles/16719
    See BibTex
    @article{suominen2022analyzer,
      title={Annif Analyzer Shootout: Comparing text lemmatization methods for automated subject indexing},
      author={Suominen, Osma and Koskenniemi, Ilkka},
      journal={Code4Lib J.},
      number={54},
      year={2022},
      url={https://journal.code4lib.org/articles/16719},
    }
    
  • Suominen, O., 2019. Annif: DIY automated subject indexing using multiple algorithms. LIBER Quarterly, 29(1), pp.1–25. DOI: https://doi.org/10.18352/lq.10285
    See BibTex
    @article{suominen2019annif,
      title={Annif: DIY automated subject indexing using multiple algorithms},
      author={Suominen, Osma},
      journal={{LIBER} Quarterly},
      volume={29},
      number={1},
      pages={1--25},
      year={2019},
      doi = {10.18352/lq.10285},
      url = {https://doi.org/10.18352/lq.10285}
    }
    

License

The code in this repository is licensed under Apache License 2.0, except for the dependencies included under annif/static/css and annif/static/js, which have their own licenses; see the file headers for details.

Please note that the YAKE library is licensed under GPLv3, while Annif itself is licensed under the Apache License 2.0. It is commonly accepted that the GPLv3 and Apache 2.0 licenses are compatible at least in one direction (GPLv3 is more restrictive than the Apache License); obviously it also depends on the legal environment. The Annif developers make no legal claims - we simply provide the software and allow the user to install optional extensions if they consider it appropriate. Depending on legal interpretation, the terms of the GPL (for example the requirement to publish corresponding source code when publishing an executable application) may be considered to apply to the whole of Annif+extensions if you decide to install the optional YAKE dependency.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

annif-1.3.1.tar.gz (476.5 kB view details)

Uploaded Source

Built Distribution

annif-1.3.1-py3-none-any.whl (507.1 kB view details)

Uploaded Python 3

File details

Details for the file annif-1.3.1.tar.gz.

File metadata

  • Download URL: annif-1.3.1.tar.gz
  • Upload date:
  • Size: 476.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.2 CPython/3.10.12 Linux/6.8.0-1021-azure

File hashes

Hashes for annif-1.3.1.tar.gz
Algorithm Hash digest
SHA256 a9608aa4dc718326d3af839540133e6d9a4b5db176713f6b2a64cb86033b4c86
MD5 0b8cd7999c5652e376c125ff5ca080d5
BLAKE2b-256 111ffb1fae26701cf069e1a73ba185f9b936e3c9be421b3569dfaac17004a9c4

See more details on using hashes here.

File details

Details for the file annif-1.3.1-py3-none-any.whl.

File metadata

  • Download URL: annif-1.3.1-py3-none-any.whl
  • Upload date:
  • Size: 507.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.2 CPython/3.10.12 Linux/6.8.0-1021-azure

File hashes

Hashes for annif-1.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 da2dda780aa1e81c345523b5667eb63e4cd9046ab0254405b913dec661eae8ec
MD5 0b94b0e11fefd7a115d9a68c958f4d04
BLAKE2b-256 3ad5fb1e2cd617ecbfff3f8dce3c8c35b5d2165e3624d83ce4c3437e665d7a03

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page