BioC data structures and encoder/decoder for Python
Project description
BioC XML / JSON format can be used to share text documents and annotations.
bioc exposes an API familiar to users of the standard library marshal and pickle modules.
Development of bioc happens on GitHub: https://github.com/bionlplab/bioc
Getting started
Installing bioc
$ pip install bioc
XML
Encoding the BioC collection object collection:
import bioc
# Serialize ``collection`` to a BioC formatted ``str``.
bioc.dumps(collection)
# Serialize ``collection`` as a BioC formatted stream to ``fp``.
with open(filename, 'w') as fp
bioc.dump(collection, fp)
Compact encoding:
import bioc
bioc.dumps(collection, pretty_print=False)
Incremental BioC serialisation:
import bioc
writer = bioc.BioCXMLDocumentWriter(filename)
writer.write_collection_info(collection)
for document in collection.documents:
writer.write_document(document)
writer.close()
Decoding the BioC XML file:
import bioc
# Deserialize ``s`` to a BioC collection object.
collection = bioc.loads(s)
# Deserialize ``fp`` to a BioC collection object.
with open(filename, 'r') as fp:
collection = bioc.load(fp)
Incrementally decoding the BioC XML file:
import bioc
# read from a file
reader = bioc.BioCXMLDocumentReader(filename)
collection_info = reader.get_collection_info()
for document in reader:
# process document
...
# read from a ByteIO
reader = bioc.BioCXMLDocumentReader(open(filename, 'rb'))
collection_info = reader.get_collection_info()
for document in reader:
# process document
...
get_collection_info can be called after the construction of the BioCXMLDocumentReader anytime.
Together with Python coroutines, this can be used to generate BioC XML in an asynchronous, non-blocking fashion.
import bioc
reader = bioc.BioCXMLDocumentReader(source)
writer = bioc.BioCXMLDocumentWriter(dest)
collection_info = reader.get_collection_info()
writer.write_collection_info(collection_info)
for document in reader:
# modify the document
...
writer.write_document(document)
writer.close()
Json
Encoding the BioC collection object collection:
import bioc
# Serialize ``collection`` to a BioC Json formatted ``str``.
bioc.dumps(collection, BioCFileType.BIOC_JSON, indent=2)
# Serialize ``collection`` as a BioC Json formatted stream to ``fp``.
with open(filename, 'w') as fp
bioc.dump(collection, BioCFileType.BIOC_JSON, fp, indent=2)
Compact encoding:
import bioc
bioc.dumps(collection, BioCFileType.BIOC_JSON)
Decoding the BioC Json file:
import bioc
# Deserialize ``s`` to a BioC collection object.
collection = bioc.loads(s, BioCFileType.BIOC_JSON)
# Deserialize ``fp`` to a BioC collection object.
with open(filename, 'r') as fp:
collection = bioc.load(fp, BioCFileType.BIOC_JSON)
Json Lines
Incrementally encoding the BioC structure:
from bioc import BioCJsonIterWriter
with open(filename, 'w', encoding='utf8') as fp:
writer = BioCJsonIterWriter(fp, level=bioc.PASSAGE)
for doc in collection.documents:
for passage in doc.passages:
writer.write(passage)
or
from bioc import toJSON
import jsonlines
with jsonlines.open(filename, 'w') as writer:
for doc in collection.documents:
for passage in doc.passages:
writer.write(toJSON(passage))
Incrementally decoding the BioC Json lines file:
from bioc import BioCJsonIterReader
with open(filename, 'r', encoding='utf8') as fp:
reader = BioCJsonIterReader(fp, level=bioc.PASSAGE)
for passage in reader:
# process passage
...
or
from bioc import fromJSON
import jsonlines
with jsonlines.open(filename) as reader:
for obj in reader:
passage = fromJSON(obj, level=bioc.PASSAGE)
...
Developers
Yifan Peng (yip4002@med.cornell.edu)
Acknowledgment
Hernani Marques (https://github.com/2mh/PyBioC)
Webpage
The official BioC webpage is available with all up-to-date instructions, code, and corpora in the BioC format, and other research on, based on and related to BioC.
Reference
If you use bioc in your research, please cite the following paper:
Peng Y, Tudor CO, Torii M, Wu CH, Vijay-Shanker K. iSimp in BioC standard format: Enhancing the interoperability of a sentence simplification system. Database (Oxford). 2014, 1-8. bau038.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.