Skip to main content

Encoding tools for DDHI

Project description

A collection of command-line utilities to assist in the creation of TEI-encoded oral history interviews. Part of the Dartmouth Digital History Initiative.

DDHI Encoder

The ddhi-encoder package is being developed to assist encoders in the DDHI project in encoding oral history interview transcripts in TEI. At present, it contains three command-line utilities:

  1. ddhi_convert: convert a Dartmouth DVP transcript from docx to tei.xml.
  2. ddhi_tag: perform named-entity tagging on a DDHI TEI transcription.
  3. ddhi_mentioned_places: extract places from stand-off markup for processing with OpenRefine
  4. ddhi_update_places: update places in stand-off markup

Installation

You can use pip to install this package:

pip install ddhi-encoder

To peform named-entity tagging with ddhi_tag, you will need a Spacy model. Before running ddhi_tag, install Spacy’s small English model:

python -m spacy download en_core_web_sm

See the Spacy documentation for more information.

Use

Use ddhi_convert to transform a DOCX-encoded transcription into a simply structured TEI document:

ddhi_convert ~/Desktop/transcripts/zien_jimmy_transcript_final.docx -o tmp.tei.xml

Use ddhi_tag to add named-entity tags to a TEI-encoded transcription:

ddhi_tag -o zien.tei.xml tmp.tei.xml

Encoders are then expected to edit the text of the interview, correcting automatically generated named-entity tags and adding new ones. when this phase of editing is complete, use ddhi_generate_standoff to create a <standOff> element in the interview and link the entities to names in the text.

Use ddhi_mentioned_places to extract the places in a TEI file’s standoff markup and print it as tab-separated values:

ddhi_mentioned_places lovely.tei.xml > lovely.tsv

Then use OpenRefine or another tool to refine this list with identifiers and other metadata.

Use ddhi_update_places to update the places in a TEI file’s standoff markup with identifiers and geo-coordinates obtained via OpenRefine or other procedure:

ddhi_update_places lovely.tei.xml lovely_updates.tsv >
updated_lovely.tei.xml

Similarly, use ddhi_mentioned_events and ddhi_update_events to perform the same operations for events.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ddhi-encoder, version 1.2.4
Filename, size File type Python version Upload date Hashes
Filename, size ddhi_encoder-1.2.4-py2.py3-none-any.whl (26.3 kB) File type Wheel Python version py2.py3 Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page