Wikipedia tools (for Humans)
Project description
Python and command-line MediaWiki access for Humans:
get an HTML or plain text “extract” (lead or summary)
get a representative image, pageimage, thumbnail
get an Infobox as a python dictionary
get selected Wikidata properties
get a Wikidata item by title
get info in any language
get random info
This package is intended to make it as easy as possible to get data from MediaWiki instances, expose more Wikidata, and extend the MediaWiki API just for kicks. We say “(for Humans)” because that is a goal. Please see NOTES for details. Questions, feedback, and especially contributions are welcome!
Install
$ pip install wptools
✨🦄✨
Usage
An instance can be initialized by:
title: <unicode> a MediaWiki article title
lang: <str> MediaWiki language code (default=’en’)
wikibase: <str> Wikidata entity ID
The simplest way to begin is with a title:
>>> import wptools
>>> f = wptools.wptools("Flannery O'Connor")
Flannery_O'Connor (en)
{
lang: en
title: Flannery_O'Connor
}
The default language is ‘en’ (English):
>>> t = wptools.wptools('穐吉敏子')
穐吉敏子 (en)
{
lang: en
title: 穐吉敏子
}
Leaving off arguments invokes a random lookup in English:
>>> r = wptools.wptools()
en.wikipedia.org (action=random) None
Sylvia_Rivera (en)
{
lang: en
pageid: 3296309
title: Sylvia_Rivera
}
If you give only lang, you get a random article in that language:
>>> zh = wptools.wptools(lang='zh')
zh.wikipedia.org (action=random) None
哈莉特·塔布曼 (zh)
{
lang: zh
pageid: 211070
title: 哈莉特·塔布曼
}
You can also start with a wikibase:
>>> q = wptools.wptools(wikibase='Q43303')
Q43303 (en)
{
lang: en
wikibase: Q43303
}
Instance attributes are echoed automatically. You can turn that off with silent=True.
>>> r = wptools.wptools(silent=True).get()
>>>
Examples
Get a representative image:
>>> frida = wptools.wptools("Frida Kahlo").get()
>>> frida.Image
u'https://upload.wikimedia.org/wikipedia/commons/0/06/Frida_Kahlo,_by_Guillermo_Kahlo.jpg'
![]()
frida.thumbnail
Get a text (or HTML) extract:
>>> ella = wptools.wptools('Ella Fitzgerald').get_query()
>>> print ella.extext
**Ella Jane Fitzgerald** (April 25, 1917 – June 15, 1996) was an
American jazz singer often referred to as the First Lady of Song,
Queen of Jazz and Lady Ella. She was noted for her purity of tone,
impeccable diction, phrasing and intonation, and a "horn-like"
improvisational ability, particularly in her scat singing.
...
>>> print ella.extract
<p><b>Ella Jane Fitzgerald</b> (April 25, 1917 – June 15, 1996) was an
American jazz singer often referred to as the First Lady of Song,
Queen of Jazz and Lady Ella. She was noted for her purity of tone,
impeccable diction, phrasing and intonation, and a "horn-like"
improvisational ability, particularly in her scat singing.</p>
...
Get an Infobox as a python object:
>>> fela = wptools.wptools('Fela Kuti').get_parse()
>>> fela.infobox['instrument']
'Saxophone, vocals, keyboards, trumpet, guitar, drums'
Get a wikibase by title:
>>> fry = wptools.wptools('Stephen Fry').get_parse()
>>> fry.wikibase
u'Q192912'
Get all the things by wikibase:
>>> jill = wptools.wptools(wikibase='Q6192915').get()
>>> jill.show()
Jill_Lepore (en)
{
Description: American historian
Label: Jill Lepore
extext: <str(1018)> **Jill Lepore** (born August 27, 1966) is an A...
extract: <str(1109)> <p><b>Jill Lepore</b> (born August 27, 1966) ...
g_parse: <dict(3)> {info, query, response}
g_query: <dict(3)> {info, query, response}
g_wikidata: <dict(3)> {info, query, response}
infobox: <dict(39)> {academic_advisors, alma_mater, alt, author_ab...
lang: en
pageid: 22469182
parsetree: <str(19661)> <root><template><title>Infobox scientist</...
random: Spanish immigration to Chile
title: Jill_Lepore
url: https://en.wikipedia.org/wiki/Jill_Lepore
urlraw: https://en.wikipedia.org/wiki/Jill_Lepore?action=raw
wikibase: Q6192915
wikitext: <str(13011)> {{Infobox scientist| name = Jill Lepore| na...
}
Get info from another wiki:
>>> m = wptools.wptools(wiki='en.wikiquote.org')
en.wikiquote.org (action=random) None
Malala_Yousafzai (en)
{
lang: en
title: Malala_Yousafzai
}
Methods
Get help on instance methods like this:
>>> help(wptools.core)
>>> help(<instance>)
get (self)
make all requests necessary to populate all the things
get_query()
get_parse()
get_wikidata()
get_parse (self)
MediaWiki:API action=parse request for:
infobox: <dict> Infobox data as python dictionary
links: <list> interwiki links (iwlinks)
pageid: <int> Wikipedia database ID
parsetree: <unicode> XML parse tree
wikibase: <unicode> Wikidata entity ID or wikidata URL
wikitext: <unicode> raw wikitext URL
get_query (self)
MediaWiki:API action=query
extext: <unicode> plain text (Markdown) extract
extract: <unicode> HTML extract via Extension:TextExtract
images: <dict> {image, pageimages, thumbnail}
pageid: <int> Wikipedia database ID
pageimage: <unicode> pageimage URL via Extension:PageImages
random: <unicode> a random article title with every request!
thumbnail: <unicode> thumbnail URL via Extension:PageImages
url: <unicode> the canonical wiki URL
urlraw: <unicode> ostensible raw wikitext URL
get_random (self)
MediaWiki:API action=query
pageid: <int> Wikipedia database ID
title: <unicode> article title
get_wikidata (self)
Wikidata:API action=wbgetentities
Description: <unicode> Wikidata description
Image: <unicode> Wikidata Property:P18 image URL
Label: <unicode> Wikidata label
show (self)
pretty-print instance attributes
Requests
Detailed request info can be found in these instance attributes:
g_parse: <dict> {info, query, response}
g_query: <dict> {info, query, response}
g_wikidata: <dict> {info, query, response}
The wptools user-agent will look like this:
wptools/0.0.5 (https://github.com/siznax/wptools) PycURL/7.43.0 libcurl/7.43.0 SecureTransport zlib/1.2.5
wptool
Basic functionality on the command-line is provided by the wptool command.
$ wptool -h
usage: wptool [-h] [-H] [-l L] [-n] [-q] [-s] [-t T] [-v] [-w W]
Get Wikipedia article info and Wikidata via MediaWiki APIs.
Gets a random English Wikipedia article by default, or in the
language -lang, or from the wikisite -wiki, or by specific
title -title. The output is a plain text extract unless -HTML.
optional arguments:
-h, --help show this help message and exit
-H, -HTML output HTML extract
-l L, -lang L language code
-n, -nowrap do not wrap text
-q, -query show query and exit
-s, -shh quiet output to stderr
-t T, -title T get a specific title
-v, -verbose HTTP status to stderr
-w W, -wiki W use alternative wikisite
Powered by https://github.com/siznax/wptools/
For example:
$ wptool -t "Jeanne d'Arc" -l fr -s
JEANNE_D'ARC—_sainte et héroïne de l'histoire de France_
![Jeanne d'Arc](https://upload.wikimedia.org/wikipedia/commons/3/39/Joan_of_arc_miniature_graded.jpg)
**Jeanne d'Arc**, née vers 1412 à Domrémy village du duché de Bar dont
une partie relevait du royaume de France pour le temporel et de
l'évêché de Toul pour le spirituel (actuellement dans le département
des Vosges en Lorraine), et morte sur le bûcher le 30 mai 1431 à
Rouen, capitale du duché de Normandie alors possession du royaume
d'Angleterre, est une héroïne de l'histoire de France, chef de guerre
et sainte de l'Église catholique, surnommée depuis le XVIe siècle «
_la Pucelle d'Orléans_ » et, depuis le XIXe siècle, « _mère de la
nation française_ ».
...
https://fr.wikipedia.org/wiki/Jeanne_d%27Arc
https://www.wikidata.org/wiki/Q7226
@siznax
Release History
0.1 (2016-08-25)
Many fixes, enhancements.
ONE CLI SCRIPT TO RULE THEM ALL: wptool.
Much testing of random (title, lang, and wiki).
Allow chaining get_s or just get().
Enabled full-hydration from wikibase only.
Block further requests on fatal error.
Get 240x thumbnails instead of default 50x.
Harmonized silent and verbose arguments.
Mock-up tests/test_advanced.py.
Made NOTES.md a knowledge base.
Use _humans_ in README examples. ;-)
0.0.5 (2016-08-23)
Major re-write.
Exposed core.WPTools as entrypoint.
Added get_parse(), get_query(), and get_wikidata().
Added get(self) to query all APIs.
Added show(self) method to display fetched attrs.
Show instance attributes after each request.
Ignore requests if attrs will not be updated.
Enabled language support across APIs.
Gets random article if no arguments.
CLI scripts and tests disabled pending update.
0.0.4 (2016-08-16)
Added wptools.lead.
Added safe_exit() to CLI scripts.
Removed a fair amount of unused code.
0.0.3 (2016-08-12)
Implemented wptools.image choices.
Added wptools.api to simplify python i/f and CLI scripts.
Merged @0x9900’s CLI dist fixes.
A little more test coverage.
0.0.2 (2016-08-02)
Starting to look like a legit module.
0.0.1 (2015)
Still better than alternatives for working with articles.
0.0.0 (2012)
It seems to work!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for wptools-0.1-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0716507a349816e083c0dab1eda2dd7d6bbd8e702fe2f967f60fd39f5153df01 |
|
MD5 | 594f7edcab8dd3ef714d1be765ebfc3b |
|
BLAKE2b-256 | 37a06331a95e992f4a74330587f447c087795fd29cc9e9741f3ea3ce2efd2cf5 |