About

GenomeHubs comprises a set of tools to parse index and search and display genomic metadata, assembly features and sequencing status for projects under the Earth BioGenome Project umbrella that aim to sequence all described eukaryotic species over a period of 10 years.

Genomehubs builds on legacy code that supported taxon-oriented databases of butterflies & moths (lepbase.org), molluscs (molluscdb.org), mealybugs (mealybug.org) and more. Genomehubs is now search-oriented and positioned to scale to the challenges of mining data across almost 2 million species.

The first output from the new search-oriented GenomeHubs is Genomes on a Tree (GoaT, goat.genomehubs.org), which has been opublised in: Challis et al. 2023, Genomes on a Tree (GoaT): A versatile, scalable search engine for genomic and sequencing project metadata across the eukaryotic tree of life. Wellcome Open Research, 8:24 doi:10.12688/wellcomeopenres.18658.1

The goat.genomehubs.org website is freely available with no logins or restrictions, and is being widely used by the academic community and especially by the Earth BioGenome Project to plan and coordinate efforts to sequence all described eukaryotic species.

The core GoaT/Genomehubs components are available as a set of Docker containers:

GoaT UI Docker image

A bundled web server to run a GoaT-specific instance of the GenomeHubs UI, as used at goat.genomehubs.org.

Usage

docker pull genomehubs/goat:latest

docker run -d --restart always \
    --net net-es -p 8880:8880 \
    --user $UID:$GROUPS \
    -e GH_CLIENT_PORT=8880 \
    -e GH_API_URL=https://goat.genomehubs.org/api/v2 \
    -e GH_SUGGESTED_TERM=Canidae \
    --name goat-ui \
    genomehubs/goat:latest

Genomehubs UI Docker image

A bundled web server to run an instance of the GenomeHubs UI, such as goat.genomehubs.org.

Usage

docker pull genomehubs/genomehubs-ui:latest

docker run -d --restart always \
    --net net-es -p 8880:8880 \
    --user $UID:$GROUPS \
    -e GH_CLIENT_PORT=8880 \
    -e GH_API_URL=https://goat.genomehubs.org/api/v2 \
    -e GH_SUGGESTED_TERM=Canidae \
    --name gh-ui \
    genomehubs/genomehubs-ui:latest

Genomehubs API Docker image

A bundled web server to run an instance of the GenomeHubs API. The GenomeHubs API underpins all search functionality for Genomes on a Tree (GoaT) goat.genomehubs.org. OpenAPI documentation for the GenomeHubs API instance used by GoaT is available at goat.genomehubs.org/api-docs.

Usage

docker pull genomehubs/genomehubs-api:latest

docker run -d \
    --restart always \
    --net net-es -p 3000:3000 \
    --user $UID:$GROUPS \
    -e GH_ORIGINS="https://goat.genomehubs.org null" \
    -e GH_HUBNAME=goat \
    -e GH_HUBPATH="/genomehubs/resources/" \
    -e GH_NODE="http://es1:9200" \
    -e GH_API_URL=https://goat.genomehubs.org/api/v2 \
    -e GH_RELEASE=$RELEASE \
    -e GH_SOURCE=https://github.com/genomehubs/goat-data \
    -e GH_ACCESS_LOG=/genomehubs/logs/access.log \
    -e GH_ERROR_LOG=/genomehubs/logs/error.log \
    -v /volumes/docker/logs/$RELEASE:/genomehubs/logs \
    -v /volumes/docker/resources:/genomehubs/resources \
    --name goat-api \
genomehubs/genomehubs-api:latest;

Genomehubs CLI Docker image

command line tool to process and index genomic metadata for GenomeHubs. Used to build and update GenomeHubs instances such as Genomes on a Tree goat.genomehubs.org.

Usage

docker pull genomehubs/genomehubs:latest

Parse [NCBI datasets](https://www.ncbi.nlm.nih.gov/datasets/) genome assembly metadata:

docker run --rm --network=host \
    -v `pwd`/sources:/genomehubs/sources \
     genomehubs/genomehubs:latest bash -c \
        "genomehubs parse \
            --ncbi-datasets-genome sources/assembly-data \
            --outfile sources/assembly-data/ncbi_datasets_eukaryota.tsv.gz"

Initialise a set of ElasticSearch indexes with [NCBI taxonomy](https://www.ncbi.nlm.nih.gov/taxonomy/) data for all eukaryotes:

docker run --rm --network=host \
    -v `pwd`/sources:/genomehubs/sources \
     genomehubs/genomehubs:latest bash -c \
        "genomehubs init \
            --es-host http://es1:9200 \
            --taxonomy-source ncbi \
            --config-file sources/goat.yaml \
            --taxonomy-jsonl sources/ena-taxonomy/ena-taxonomy.extra.jsonl.gz \
            --taxonomy-ncbi-root 2759 \
            --taxon-preload"

Index assembly metadata:

docker run --rm --network=host \
    -v `pwd`/sources:/genomehubs/sources \
     genomehubs/genomehubs:latest bash -c \
        "genomehubs index \
            --es-host http://es1:9200 \
            --taxonomy-source ncbi \
            --config-file sources/goat.yaml \
            --assembly-dir sources/assembly-data"

Fill taxon attribute values across the tree of life:

docker run --rm --network=host \
    -v `pwd`/sources:/genomehubs/sources \
     genomehubs/genomehubs:latest bash -c \
        "genomehubs fill \
            --es-host http://es1:9200 \
            --taxonomy-source ncbi \
            --config-file sources/goat.yaml \
            --traverse-root 2759 \
            --traverse-infer-both"

Related projects

Some GenomeHubs components are hosted in separate open source repositories (all under MIT licenses), including:

BlobToolKit GitHub release

Interactive quality assessment of genome assemblies.

Explore analysed public assemblies at blobtoolkit.genomehubs.org/view

GoaT CLI GitHub release

A command line interface for GoaT.

The GoaT CLI builds URLs to query the Goat API, removing some of the complexity of the GoaT API. for the end user.

Changelog

2.0.0 (2020-07-02)

  • First release on PyPI.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for rjchallis from gravatar.com rjchallis

Credits

Author: genomehubs

License

MIT License

Requires

Python <4, >=3.6

Provides Extra

dev test

Tags

bioinformatics

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

genomehubs-2.12.12.tar.gz (133.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

genomehubs-2.12.12-py3-none-manylinux2014_x86_64.whl (165.6 kB view details)

Uploaded Python 3

File details

Details for the file genomehubs-2.12.12.tar.gz.

File metadata

  • Download URL: genomehubs-2.12.12.tar.gz
  • Upload date:
  • Size: 133.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.9.25

File hashes

Hashes for genomehubs-2.12.12.tar.gz
Algorithm Hash digest
SHA256 4546e3a8106b6b53d38d9533a63785856ee483eb936e002a5c77e9ce9a9a34a0
MD5 5e8285b85d4ea9e0b7883a4441a3771c
BLAKE2b-256 1c4217e2d096cc1ad2e6d31d30d583336d17959e0a7d8438408f37cc0111d2e3

See more details on using hashes here.

File details

Details for the file genomehubs-2.12.12-py3-none-manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for genomehubs-2.12.12-py3-none-manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 830f450db7fe130cb2130f6186e6877120df5809e00d82fbf2ee7b166b362875
MD5 a7ef7f3ba1478cdc0cbecb9808d6b67d
BLAKE2b-256 0400e1622348db410ffe44f1a578fcf5c84384549ec5f2586dbaf50fae7a37d4

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

2.12.12 This release

2 files

2.12.11

2 files

2.12.10

2 files

2.12.9

2 files

2.12.8

2 files

2.12.7

2 files

2.12.6

2 files

2.12.5

2 files

2.12.4

2 files

2.12.3

2 files

2.12.2

2 files

2.12.1

2 files

2.12.0

2 files

2.11.17

2 files

2.11.16

2 files

2.11.15

2 files

2.11.14

2 files

2.11.13

2 files

2.11.12

2 files

2.11.11

2 files

2.11.10

2 files

2.11.9

2 files

2.11.8

2 files

2.11.7

2 files

2.11.6

2 files

2.11.5

2 files

2.11.4

2 files

2.11.3

2 files

2.11.2

2 files

2.11.1

2 files

2.11.0

2 files

2.10.21

2 files

2.10.20

2 files

2.10.19

2 files

2.10.18

2 files

2.10.17

2 files

2.10.16

2 files

2.10.15

2 files

2.10.14

2 files

2.10.13

2 files

2.10.12

2 files

2.10.11

2 files

2.10.10

2 files

2.10.9

2 files

2.10.8

2 files

2.10.7

2 files

2.10.6

2 files

2.10.5

2 files

2.10.4

2 files

2.10.3

2 files

2.10.0

2 files

2.9.17

2 files

2.9.16

2 files

2.9.15

2 files

2.9.14

2 files

2.9.13

2 files

2.9.12

2 files

2.9.11

2 files

2.9.10

2 files

2.9.9

2 files

2.9.8

2 files

2.9.7

2 files

2.9.6

2 files

2.9.5

2 files

2.9.4

2 files

2.9.3

2 files

2.9.2

2 files

2.9.1

2 files

2.9.0

2 files

2.8.24

2 files

2.8.23

2 files

2.8.22

2 files

2.8.21

2 files

2.8.20

2 files

2.8.19

2 files

2.8.18

2 files

2.8.17

2 files

2.8.16

2 files

2.8.15

2 files

2.8.14

2 files

2.8.13

2 files

2.8.12

2 files

2.8.11

2 files

2.8.10

2 files

2.8.9

2 files

2.8.8

2 files

2.8.7

2 files

2.8.6

2 files

2.8.5

2 files

2.8.4

2 files

2.8.3

2 files

2.8.2

2 files

2.8.1

2 files

2.8.0

2 files

2.7.45

2 files

2.7.44

2 files

2.7.43

2 files

2.7.42

2 files

2.7.41

2 files

2.7.40

2 files

2.7.39

2 files

2.7.38

2 files

2.7.37

2 files

2.7.36

2 files

2.7.35

2 files

2.7.34

2 files

2.7.33

2 files

2.7.32

2 files

2.7.31

2 files

2.7.30

2 files

2.7.29

2 files

2.7.28

2 files

2.7.27

2 files

2.7.25

2 files

2.7.24

2 files

2.7.23

2 files

2.7.22

2 files

2.7.21

2 files

2.7.20

2 files

2.7.19

2 files

2.7.18

2 files

2.7.17

2 files

2.7.16

2 files

2.7.15

2 files

2.7.14

2 files

2.7.12

2 files

2.7.11

2 files

2.7.10

2 files

2.7.9

2 files

2.7.8

2 files

2.7.7

2 files

2.7.6

2 files

2.7.5

2 files

2.7.1

2 files

2.7.0

2 files

2.6.16

2 files

2.6.15

2 files

2.6.14

2 files

2.6.13

2 files

2.6.12

2 files

2.6.10

2 files

2.6.9

2 files

2.6.8

2 files

2.6.7

2 files

2.6.6

2 files

2.6.5

2 files

2.6.4

2 files

2.6.3

2 files

2.6.2

2 files

2.6.1

2 files

2.6.0

2 files

2.5.47

2 files

2.5.46

2 files

2.5.45

2 files

2.5.44

2 files

2.5.43

2 files

2.5.42

2 files

2.5.41

2 files

2.5.40

2 files

2.5.39

2 files

2.5.38

2 files

2.5.37

2 files

2.5.36

2 files

2.5.35

2 files

2.5.34

2 files

2.5.33

2 files

2.5.32

2 files

2.5.31

2 files

2.5.30

2 files

2.5.29

2 files

2.5.28

2 files

2.5.27

2 files

2.5.26

2 files

2.5.25

2 files

2.5.24

2 files

2.5.23

2 files

2.5.22

2 files

2.5.21

2 files

2.5.20

2 files

2.5.19

2 files

2.5.18

2 files

2.5.17

2 files

2.5.16

2 files

2.5.15

2 files

2.5.14

2 files

2.5.13

2 files

2.5.12

2 files

2.5.11

2 files

2.5.10

2 files

2.5.9

2 files

2.5.8

2 files

2.5.7

2 files

2.5.6

2 files

2.5.5

2 files

2.5.4

2 files

2.5.3

2 files

2.5.2

2 files

2.5.1

2 files

2.5.0

2 files

2.4.26

2 files

2.4.25

2 files

2.4.24

2 files

2.4.23

2 files

2.4.22

2 files

2.4.21

2 files

2.4.20

2 files

2.4.19

2 files

2.4.18

2 files

2.4.17

2 files

2.4.16

2 files

2.4.15

2 files

2.4.14

2 files

2.4.13

2 files

2.4.12

2 files

2.4.11

2 files

2.4.10

2 files

2.4.9

2 files

2.4.8

2 files

2.4.7

2 files

2.4.5

2 files

2.4.4

2 files

2.4.3

2 files

2.4.2

2 files

2.4.1

2 files

2.3.5

4 files

2.3.4

4 files

2.3.3

4 files

2.3.1

4 files

2.3.0

4 files

2.2.0

2 files

2.1.0

2 files

2.0.7

2 files

2.0.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page