This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

DiscourseGraphs

This library enables you to process linguistic corpora with multiple levels of annotations by:

  1. converting the different annotation formats into separate graphs and
  2. merging these graphs into a single multidigraph (based on the common tokenization of the annotation layers)
  3. exporting your (merged) graphs into several output formats
  4. visualizing linguistic graphs directly in an IPython notebook

Import formats

So far, the following formats can be imported and merged:

  • TigerXML (a format for representing tree-like syntax graphs with secondary edges)
  • Penn Treebank format (an s-expressions/lisp/brackets format for representing syntax trees)
  • RS3 (a format used by RSTTool to annotate documents with Rhetorical Structure Theory)
  • MMAX2 (a format / GUI tool for annotating spans and connections between them (e.g. coreferences)
  • CoNLL 2009 and CoNLL 2010 formats (used for annotating i.a. dependency parses and coreference links)
  • ConanoXML (a format for annotating connectives, used by Conano)
  • Decour (an XML format used by a corpus of DEceptive statements in Italian COURts)
  • EXMARaLDA, a format for annotating spans in spoken or written language
  • an ad-hoc plain text format for annotating expletives (you’re probably not interested in)

Export formats

discoursegraphs can export graphs into the following formats / for the following tools:

  • dot format, which is used by the open source graph visualization software graphviz
  • geoff format, used by the neo4j graph database
  • GEXF and GraphML (common interchange formats for graphs used by various tools such as Gephi and Cytoscape)
  • PAULA XML 1.1, an exchange format for linguistic data (exporter is still buggy)
  • EXMARaLDA, a tool for annotating spans in spoken or written language
  • CoNLL 2009 (so far, only tokens, sentence boundaries and coreferences are exported)

Installation

This should work on both Linux and Mac OSX using Python 2.7 and either pip or easy_install.

Install from PyPI

pip install discoursegraphs # prepend 'sudo' if needed

or, if you’re oldschool:

easy_install discoursegraphs # prepend 'sudo' if needed

Install from source

sudo apt-get install python-dev libxml2-dev libxslt-dev pkg-config graphviz-dev libgraphviz-dev -y
sudo easy_install -U setuptools
git clone https://github.com/arne-cl/discoursegraphs.git
cd discoursegraphs
sudo python setup.py install

Usage

The command line interface of DiscourseGraphs allows you to merge syntax, rhetorical structure, connectives and expletives annotation files into one graph and to store this graph in one of several output formats (e.g. the geoff format used by the neo4j graph database or the dot format used by the graphviz plotting tool).

discoursegraphs -t syntax/maz-13915.xml -r rst/maz-13915.rs3 -c connectors/maz-13915.xml -a anaphora/tosik/das/maz-13915.txt -o dot
dot -Tpdf doc.dot > discoursegraph.pdf # generates a PDF from the dot file

If you’re interested in working with just one of those layers, you’ll have to call the code directly:

import discoursegraphs as dg
tiger_docgraph = dg.read_tiger('syntax/doc.xml')
rst_docgraph = dg.read_rs3('rst/doc.rs3')
expletives_docgraph = dg.read_anaphoricity('expletives/doc.txt')

All the document graphs generated in this example are derived from the networkx.MultiDiGraph class, so you should be able to use all of its methods.

Documentation

Source code documentation is available here, but you can always get an up-to-date local copy using Sphinx.

You can generate an HTML or PDF version by running these commands in the docs directory:

make latexpdf

to produce a PDF (docs/_build/latex/discoursegraphs.pdf) and

make html

to produce a set of HTML files (docs/_build/html/index.html).

Requirements

If you’d like to visualize your graphs, you will also need:

License and Citation

This software is released under a 3-Clause BSD license. If you use discoursegraphs in your academic work, please cite the following paper:

Neumann, A. 2015. discoursegraphs: A graph-based merging tool and converter for multilayer annotated corpora. In Proceedings of the 20th Nordic Conference of Computational Linguistics (NODALIDA 2015), pp. 309-312.

@inproceedings{neumann2015discoursegraphs,
  title={discoursegraphs: A graph-based merging tool and converter for multilayer annotated corpora},
  author={Neumann, Arne},
  booktitle={Proceedings of the 20th Nordic Conference of Computational Linguistics (NODALIDA 2015)},
  pages={309-312},
  year={2015}
}

Author

Arne Neumann

People who downloaded this also like

  • SaltNPepper: a converter framework for various linguistic data formats
  • educe: a library for handling discourse-annotated corpora (SDRT, RST and PDTB)
  • treetools: a library for converting treebanks and grammar extraction (supports i.a. TigerXML and Negra/Tüba-Export formats)
  • TCFnetworks: library for creating graphs from annotated text corpora (based on TCF).

News

0.3.2 (2016-05-30)

Release data: 30-May-2016

  • second attempt to fix the distribution of the data directory with the package
  • added exporter for FREQT, which extracts frequent embedded subtrees

0.3.1 (2016-05-07)

Release data: 7-May-2016

  • attempt to fix the distribution of the data directory with the package
  • document graphs can be converted into PTB-style strings (readwrite/tree.py)
  • node/edge collections are now ordered (OrderedDict)

0.3.0 (2016-04-30)

Release data: 30-April-2016

  • almost two years and countless commits later, finally a new official release
  • added lots of importers and exporters and simplified the API
  • added 80+ tests (py.test), continuous integration (Travis) and docker support

0.1.2 (2014-05-13)

Release data: 13-May-2014

  • added basic Geoff and Neo4j exporter (not yet available via the command line)
  • added sphinx-based documentation

0.1.1 (2014-04-25)

Release date: 25-Apr-2014

  • small improvements
  • added usage examples to readme
  • discoursegraphs script now uses the commandline interface of the merging module

0.1.0 (2014-04-24)

Release date: 24-Apr-2014

  • first public release
  • imports: RS3, TigerXML and an ad-hoc format for expletive annotation
  • merge these formats/files into a single multidigraph
  • generates simple dot/graphviz-based visualization
Release History

Release History

0.3.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
discoursegraphs-0.3.2-py2-none-any.whl (2.7 MB) Copy SHA256 Checksum SHA256 py2 Wheel May 30, 2016
discoursegraphs-0.3.2.tar.gz (125.9 kB) Copy SHA256 Checksum SHA256 Source May 30, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting