Python package containing various utilities relevant in the field of digital humanities.
Project description
Digital Humanities Utilities
Python 3.6+ package containing various utilities relevant in the field of digital humanities.
$ pip install dh-utils
Unicode utilities
Convert Greek beta code to unicode
>>> from dh_utils.unicode import beta2uni
>>> u.beta2uni('lo/gos')
'λόγος'
This is a wrapper of the CLTK converter. We used this converter to also create inverse:
>>> from dh_utils.unicode import uni2beta
>>> uni2beta('λόγος')
'lo/gos'
NB: Since cltk and its dependency nltk are relatively large, cltk is added as an optional dependency. To use the beta2uni
converter, either install cltk separately using pip install cltk
or install dh-utils including this optional depency with pip install dh-utils[betacode]
.
Decompose a unicode string
>>> u.decompose('λόγος')
λ U+03bb GREEK SMALL LETTER LAMDA
ο U+03bf GREEK SMALL LETTER OMICRON
́ U+0301 COMBINING ACUTE ACCENT
γ U+03b3 GREEK SMALL LETTER GAMMA
ο U+03bf GREEK SMALL LETTER OMICRON
ς U+03c2 GREEK SMALL LETTER FINAL SIGMA
TEI utilities
Convert markdown to TEI
A basic converter from markdown to TEI has been added. It will convert a markdown file like:
Some paragraph block
> A blockquote
1. An
2. Ordered
3. List
Another paragraph block with _italics_ and __bold__, and:
* An
* Unordered
* List
using a Python snippet like
>>> from dh_utils.tei import md2tei
>>> with open('file.md') as f:
>>> md = md2tei(f.read())
to the following TEI XML:
<p>Some paragraph block</p>
<quote>
<p>A blockquote</p>
</quote>
<list rend="numbered">
<item>An</item>
<item>Ordered</item>
<item>List</item>
</list>
<p>Another paragraph block with <hi rend="italic">italics</hi> and <hi rend="bold">bold</hi>, and:</p>
<list rend="bulleted">
<item>An</item>
<item>Unordered</item>
<item>List</item>
</list>
The function md2tei
is syntactic sugar for the markdown extension ToTEI
, which can be used in combination with other extensions as follows:
>>> from markdown import markdown
>>> from dh_utils.tei import ToTEI
>>> markdown('some text', extensions=[ToTEI()]) # Other extensions can be added to this list
The extension ToTEI
in turn exists solely of the postprocessor TEIPostprocessor
. It has priority 0, which usually means that it will run after all other postprocessors have finished. If any other behaviour or prioritization is required, the processor TEIPostprocessor
can also be directly imported (from dh_utils.tei import TEIPostprocessor
) and used in a custom markdown extension.
Tag languages
Tag languages in a given string based on its script:
>>> t.tag('A line contaning the hebrew אגוז מלך inline', 'Hebr')
'A line contaning the hebrew <foreign xml:lang="he-Hebr">אגוז מלך</foreign> inline'
It is also possible to tag a given language based on its script in a TEI XML document (NB: file will be overwritten!):
>>> t.tag_xml('path/to/file.xml', 'Arab')
The available scripts are stored in AVAILABLE_SCRIPTS
and are enumerated below:
>>> t.AVAILABLE_SCRIPTS
['Arab', 'Copt', 'Hebr', 'Latn', 'Cyrl']
Default language-script codes are used to tag the scripts (stored in DEFAULT_LCS
), which can be adjusted using the language_code
keyword argument:
>>> t.tag_xml('path/to/file.xml', 'Cyrl', language_code = 'ov-Cyrs')
Refsdecl generator
To generate refsdecl elements, the generator can be used to create etree xml elements:
from dh_utils.tei import refsdecl_generator
refs_decl = refsdecl_generator.generate_for_file("./path/to/file")
refs_decls = refsdecl_generator.generate_for_path("./path/to/files")
It can also be used trough the command line interface:
python -m dh_utils.tei.refsdecl_generator [--update] [PATH]
By default, it does not update the file but outputs the refsdecl xml to the terminal. If the --update
flag is given, the file is updated with the generated refsdecl.
MyCapytain-compatilble critical apparatus
The Python API MyCapytain only serves the main text of a CTS structured text version, and does not support stand-off annotation, bibliographies, critical apparati, etc. To overcome the last problem, we have developed a script that generates a separate text version of the critical apparatus that can be served through MyCapytain. Brill's Scholarly Editions uses these separate text versions, which can be displayed in parallel.
The following snippet creates such a critapp file from textgroup.work.edition-extension.xml
located in path/to/data/textgroup/work
and saves it as textgroup.work.edition-appcrit1.xml
>>> from dh_utils.tei import crit_app as ca
>>> data_dir = "path/to/data/textgroup/work"
>>> filename = "textgroup.work.edition-extension.xml"
>>> ca_ext = "appcrit1" # Or any other extension
>>> ca.create(filename, ca_ext, data_dir)
If a file contains multiple critical apparati, these can be distinguished using /listApp[@type]
, e.g.:
<listApp type="superior">
<app/>
<app/>
...
</listApp>
<listApp type="inferior">
<app/>
<app/>
...
</listApp>
Using the above snippet will combine these apparati into one file. If these should be conerted to separate files, one can pass an additional argument app_type
to ca.create
(e.g., ca.create(filename, ca_ext, data_dir app_type="superior")
) to convert an apparatus separately.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dh_utils-0.1.21-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 24f9f5c67dbdabf70e9b6ce626c9974c78a579174bbc9d640fa4f477dbcc3412 |
|
MD5 | 82ac65b87d1593858cf33f3a6d20cd02 |
|
BLAKE2b-256 | 41f0de3337c36e11d247866d4690eea20f635a1ca407e24d64412bd0d1b08482 |