Software Heritage - Listers
Collection of listers for source code distribution places like development forges, FOSS distributions, package managers, etc. Each lister is in charge to enumerate the software origins (e.g., VCS, packages, etc.) available at a source code distribution place.
A lister is a component from the Software Heritage stack aims to produce listings of software origins and their urls hosted on various public developer platforms or package managers. As these operations are quite similar, this package provides a set of Python modules abstracting common software origins listing behaviors.
It also provides several lister implementations, contained in the Python swh.lister.* modules. See this documentation for the list of supported listers.
Dependencies
All required python dependencies can be found in the requirements*.txt files located at the root of the repository.
In order to be able to run all the listers (and thus execute the tests), some tools must be available on your system, namely:
opam
tar
psql
On a Debian-like system, you may use:
Local deployment
Lister configuration
Each lister must be configured by following the instructions below (please note that you have to replace <lister_name> by the lister name).
Preparation steps
mkdir ~/.config/swh/
create configuration file ~/.config/swh/listers.yml
Configuration file sample
Minimalistic configuration shared by all listers to add in file ~/.config/swh/listers.yml:
scheduler:
cls: 'remote'
args:
url: 'http://localhost:5008/'
credentials: {}Note: This expects scheduler (5008) service to run locally
Executing a lister
Once configured, a lister can be executed by using the swh CLI tool with the following options and commands:
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister <lister_name> [lister_parameters]Examples:
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister bitbucket
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister cran
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister gitea url=https://demo.gitea.com/
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister forgejo url=https://try.next.forgejo.org/
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister gitlab url=https://salsa.debian.org/
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister npm
$ swh --log-level DEBUG lister -C ~/.config/swh/listers.yml run --lister pypiLicensing
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
See top-level LICENSE file for the full text of the GNU General Public License along with this program.
Metadata
Release files for swh.lister 7.4.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| swh_lister-7.4.0.tar.gz | 1.4 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| swh_lister-7.4.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 2.9 MB
Release files / swh_lister-7.4.0.tar.gz
| Download URL | swh_lister-7.4.0.tar.gz |
|---|---|
| Size | 1.4 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5ef340d78bf6275b569d3c5cc388d94faa697f25e449e14bd0af51e0e01cdb80
|
|
BLAKE2b-256 checksum How to use checksums |
74331e2e30c3afa654976cd25db2efce7b22d70a3a664045adf6a47ef30648af
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.15
|
Release files / swh_lister-7.4.0-py3-none-any.whl
| Download URL | swh_lister-7.4.0-py3-none-any.whl |
|---|---|
| Size | 1.5 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
79b0d83f76e278d5508dac74fcb114ca2806071a672d0a87de7ac3b6ea1bd10b
|
|
BLAKE2b-256 checksum How to use checksums |
bf808746ec6e145dd8ea130ac8fafe3cf365e51e4e9d60df65a0380fd4a4d244
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.11.15
|
