Converts FoLiA and TEI files to Alpino XML files
Project description
CHAT, FoLiA, PaQu metadata, plaintext and TEI to Alpino XML or PaQu metadata format
Converts CHAT, FoLiA, PaQu metadata, plaintext and TEI XML files to Alpino XML files. Each sentence in the input file is parsed separately.
Usage
Command Line
pip install corpus2alpino
corpus2alpino -s localhost:7001 folia.xml -o alpino.xml
Or from project root:
python -m corpus2alpino -s localhost:7001 folia.xml -o alpino.xml
Library
from corpus2alpino.converter import Converter
from corpus2alpino.annotators.alpino import AlpinoAnnotator
from corpus2alpino.collectors.filesystem import FilesystemCollector
from corpus2alpino.targets.memory import MemoryTarget
from corpus2alpino.writers.lassy import LassyWriter
alpino = AlpinoAnnotator("localhost", 7001)
converter = Converter(FilesystemCollector(["folia.xml"]),
# Not needed when using the PaQuWriter
annotators=[alpino],
# This can also be ConsoleTarget, FilesystemTarget
target=MemoryTarget(),
# Set to merge treebanks, also possible to use PaQuWriter
writer=LassyWriter(True))
# get the Alpino XML output, combined into one treebank XML file
parses = converter.convert()
print(''.join(parses)) # <treebank><alpino_ds ... /></treebank>
Enrichment
It is possible to add custom properties to (existing) Lassy/Alpino files. This is done using a csv-file containing the node attributes and values to look for and the custom properties to assign.
For example:
python -m corpus2alpino tests/example_lassy.xml -e tests/enrichment.csv -of lassy
See corpus2alpino.annotators.enrich_lassy
for more information.
Development
Unit Test
python -m unittest
Upload to PyPi
See: https://packaging.python.org/tutorials/packaging-projects/#generating-distribution-archives
Make sure setuptools
and wheel
are installed. Then from the virtualenv:
pip install build
python -m build
twine upload dist/*
Requirements
- Alpino parser running as a server:
Alpino batch_command=alpino_server -notk server_port=7001
- Python 3.8 or higher
- libfolia-dev
- libxml2-dev
Installation Instructions for Ubuntu
sudo apt install libfolia-dev libxml2-dev
pip install -r requirements.txt
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file corpus2alpino-0.3.12.tar.gz
.
File metadata
- Download URL: corpus2alpino-0.3.12.tar.gz
- Upload date:
- Size: 22.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.9.19
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | aafc630a77899ee537e225b2cac57b9533bcbe6d2a990f37ab4801fe3e599738 |
|
MD5 | fe4cdc955653f983fd994cbf54e9d8f1 |
|
BLAKE2b-256 | 8ed866d2bd2fa237444232fb1b7ac1d96023f4f6d897929e79ee2149de5128c9 |
File details
Details for the file corpus2alpino-0.3.12-py3-none-any.whl
.
File metadata
- Download URL: corpus2alpino-0.3.12-py3-none-any.whl
- Upload date:
- Size: 31.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.0.0 CPython/3.9.19
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4bde313d1aadfa42c9f26257f4afe3f8fdc43a5f458a3ad82c86c57774e211df |
|
MD5 | 672b3b21b45b3944766099fa967cc2a3 |
|
BLAKE2b-256 | ad1a11896d604c9586be9990c7d612d3b501b40895652a8fae4d9675cca6a06d |