Minimalist wrapper around Stanford OpenIE
Project description
Python wrapper for Stanford OpenIE
Open information extraction (open IE) refers to the extraction of structured relation triples from plain text, such that the schema for these relations does not need to be specified in advance. For example, Barack Obama was born in Hawaii would create a triple (Barack Obama; was born in; Hawaii)
, corresponding to the open domain relation "was born in". CoreNLP is a Java implementation of an open IE system as described in the paper:
More information can be found here : http://nlp.stanford.edu/software/openie.html
V2
from openie import StanfordOpenIE
with StanfordOpenIE() as client:
text = 'Barack Obama was born in Hawaii. Richard Manning wrote this sentence.'
print('Text: %s.' % text)
for triple in client.annotate(text):
print('|-', triple)
graph_image = 'graph.png'
client.generate_graphviz_graph(text, graph_image)
print('Graph generated: %s.' % graph_image)
with open('corpus/pg6130.txt', 'r', encoding='utf8') as r:
corpus = r.read().replace('\n', ' ').replace('\r', '')
triples_corpus = client.annotate(corpus[0:50000])
print('Corpus: %s [...].' % corpus[0:80])
print('Found %s triples in the corpus.' % len(triples_corpus))
for triple in triples_corpus[:3]:
print('|-', triple)
V1 (deprecated)
The unofficial cross-platform Python wrapper for the state-of-art information extraction library from Stanford University.
NOTE: Windows is not currently supported! Works on UNIX systems like Linux and Mac OS.
Usage
First of all, make sure Java 1.8 is installed. Open a terminal and run this command to check:
java -version
If this is not the case and if your OS is Ubuntu, you can install it this way:
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer
The code can be invoked either programmatically or through the command line. The program can be invoked with the following command. It will display [['Barack Obama', ' was', ' born'], ['Barack Obama', ' was born in', ' Hawaii']]
git clone https://github.com/philipperemy/Stanford-OpenIE-Python.git
cd Stanford-OpenIE-Python
echo "Barack Obama was born in Hawaii." > samples.txt
python main.py -f samples.txt
The output should be something like:
Barack Obama | was | born
Barack Obama | was born in | Hawaii
Troubleshooting
It's possible that you get an error like that one when using the lib for the first time:
AssertionError: ERROR: Call to stanford_ie exited with a non-zero code status
The error is related to the interaction with Java. To troubleshoot it, I would advise to run the python script with the --verbose
argument. Search for line executing command = <whatever command>
in the logs and copy paste this <whatever command>
in your terminal to see the error.
Large Corpus
Sometimes you just want to run the information extraction tool on something larger than just a couple of sentences. I provide a bash script for that. This example runs the tool on the book: The Iliad by Homer, composed of 1.2M characters and 26K lines.
./process_large_corpus.sh corpus/pg6130.txt corpus/pg6130.txt.out
wc -l corpus/pg6130.txt.out
> 23888
Generate Graph
echo "Barack Obama was born in Hawaii." > samples.txt
python main.py -f samples.txt -g
Will generate a GraphViz DOT graph and its related PNG file in /tmp/openie/
Note: Make sure GraphViz is installed beforehand. Try to run the dot
command to see if this is the case. If not, run sudo apt-get install graphviz
if you're running on Ubuntu.
Support
You can either open an issue or send me an e-mail to premy@cogent.co.jp. Any contributions are welcomed!
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for stanford_openie-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 28c40ebe75761e60cac5bfcb5385cd577c95eb72696ed6c0983cdc3984053756 |
|
MD5 | 5214cbb549bfeefceef68c69481176a8 |
|
BLAKE2b-256 | b8e367061dbba7fd2d41e91e7d110f58ca09b456183dbaba514a052dc16d6eac |