LightNER 0.2.0

A Toolkit for Pre-trained Sequence Labeling Models Inference

# LightNER

[](https://opensource.org/licenses/Apache-2.0)
[](https://badge.fury.io/py/LightNER)
<!-- [](http://tensorboard-wrapper.readthedocs.io/en/latest/?badge=latest) -->
<!-- [](https://pepy.tech/project/LightNER) -->

A Toolkit to conduct inference with models pre-trained by LD-Net / AutoNER / VanillaNER / ...

We are in an early-release beta. Expect some adventures and rough edges.

## Quick Links

- [Installation](#installation)
- [Usage](#usage)

## Installation

To install via pypi:
```
pip install lightner
```

To build from source:
```
pip install git+https://github.com/LiyuanLucasLiu/LightNER
```
or
```
git clone https://github.com/LiyuanLucasLiu/LightNER.git
cd LightNER
python setup.py install
```

## Usage

### Pre-trained Models

| | NER | NP |
| ------------- |------------- | ------------- |
| LD-Net | [pner0.th](http://dmserv4.cs.illinois.edu/pner0.th) | [pnp0.th](http://dmserv4.cs.illinois.edu/pnp0.th) |
...

### Decode API

The decode api can be called in the following way:
```
from lightner import decoder_wrapper
model = decoder_wrapper()
model.decode(["Ronaldo", "won", "'t", "score", "more", "than", "30", "goals", "for", "Juve", "."])
```

The ```decode()``` method also can conduct decoding at document level (takes list of list of ```str``` as input) or corpus level (takes list of list of list of ```str``` as input).

The ```decoder_wrapper``` method can be customized by choosing a different pre-trained model or passing an additional ```configs``` file as:
```
model = decoder_wrapper(URL_OR_PATH_TO_CHECKPOINT, configs)
```
And you can access the config options by:
```
lightner decode -h
```

### Console

After installing and downloading the pre-trained mdoels, conduct the inference by
```
lightner decode -m MODEL_FILE -i INPUT_FILE -o OUTPUT_FILE
```

You can find more options by:
```
lightner decode -h
```

The current accepted paper format is as below (tokenized by line break and ```-DOCSTART-``` is optional):
```
-DOCSTART-

Ronaldo
won
't
score
more
30
goals
for
Juve
.
```

The output would be:
```
<PER> Ronaldo </PER> won 't score more than 30 goals for <ORG> Juve </ORG> .
```