Skip to main content

Break WikiData dumps into smaller knowledge graphs

Project description

WikiDataSets

https://img.shields.io/pypi/v/wikidatasets.svg https://img.shields.io/travis/armand33/wikidatasets.svg Documentation Status Updates

Breaking WikiData dumps into smaller knowledge graphs (e.g. graph of human entities).

Data Sets

Data sets are available on this page.

Features

This is a non-exhaustive list of useful functions :

  • wikidatasets.processFunction.get_subclasses : Gets a list of WikiData IDs of entities which are subclasses of the subject.

  • wikidatasets.processFunction.query_wikidata_dump : Goes through a Wikidata dump. It can either collect entities that are instances of test_entities or collect the dictionary of labels. It can also do both.

  • wikidatasets.processFunction.build_dataset : Builds datasets from the pickle files produced by the query_wikidata_dump.

  • wikidatasets.utils.load_data_labels : Loads the edges and attributes files into Pandas dataframes and merges the labels of entities and relations to get.

The example/ folder contains examples of scripts to create datasets (e.g. build_humans.py). Such scripts should be placed in the main directory (along with utils.py, processFunctions.py) and hard-coded paths should be tuned to match your installation.

Citations

If you find this code useful in your research, please consider citing our paper:

@misc{arm2019wikidatasets,
    title={WikiDataSets : Standardized sub-graphs from WikiData},
    author={Armand Boschin},
    year={2019},
    eprint={1906.04536},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.

History

0.1.0 (2019-07-01)

  • First release on PyPI.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

wikidatasets-0.1.5.tar.gz (14.7 kB view details)

Uploaded Source

Built Distribution

wikidatasets-0.1.5-py2.py3-none-any.whl (10.1 kB view details)

Uploaded Python 2Python 3

File details

Details for the file wikidatasets-0.1.5.tar.gz.

File metadata

  • Download URL: wikidatasets-0.1.5.tar.gz
  • Upload date:
  • Size: 14.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.7.1

File hashes

Hashes for wikidatasets-0.1.5.tar.gz
Algorithm Hash digest
SHA256 71cecbd21ad8833451716990f61a6411eac0456e682a2207620cf804f99d2867
MD5 7c74ddf6f6857472f23e8d55e983e859
BLAKE2b-256 74e64f77f670a1f007ffa35071771ff8ee55c713a2814c1ba985c16de8faadf5

See more details on using hashes here.

File details

Details for the file wikidatasets-0.1.5-py2.py3-none-any.whl.

File metadata

  • Download URL: wikidatasets-0.1.5-py2.py3-none-any.whl
  • Upload date:
  • Size: 10.1 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.2 CPython/3.7.1

File hashes

Hashes for wikidatasets-0.1.5-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 22aab373feb8f8c048bda5dc5f84d5a155361813366b54b54752c08d35650dfb
MD5 d384a36b47297330af15917b34a2e934
BLAKE2b-256 e43b24c546235d536c6d4828a8af028c81339eb7e713971676acf4daa9ab8e7e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page