Tiny DSL to generate training dataset for NLU engines
Project description
Tiny DSL to generate training dataset for NLU engines. Based on the javascript implementation of chatl.
Installation
pip
$ pip install pychatl
source
$ git clone https://github.com/atlassistant/pychatl.git
$ cd pychatl
$ python setup.py install
or
$ pip install -e .
Usage
From the terminal
$ pychatl .\example\forecast.dsl .\example\lights.dsl -a snips -o '{ \"language\": \"en\" }'
From the code
from pychatl import parse
result = parse("""
# pychatl is really easy to understand.
#
# You can defines:
# - Intents
# - Entities (with or without variants)
# - Synonyms
# - Comments (only at the top level)
# Inside an intent, you got training data.
# Training data can refer to one or more entities and/or synonyms, they will be used
# by generators to generate all possible permutations and training samples.
%[my_intent]
~[greet] some training data @[date]
another training data that uses an @[entity] at @[date#with_variant]
~[greet]
hi
hello
# Entities contains available samples and could refer to a synonym.
@[entity]
some value
other value
~[a synonym]
# Synonyms contains only raw values
~[a synonym]
possible synonym
another one
# Entities and intents can define arbitrary properties that will be made available
# to generators.
# For snips, `snips:type` and `extensible` are used for example.
@[date](snips:type=snips/datetime)
tomorrow
today
# Variants is used only to generate training sample with specific values that should
# maps to the same entity name, here `date`. Props will be merged with the root entity.
@[date#with_variant]
the end of the day
nine o clock
twenty past five
""")
# Now you got a parsed dataset so you may want to process it for a specific NLU engines
from pychatl.postprocess import snips
snips_dataset = snips(result) # Or give options with `snips(result, language='en')`
# And now you got your dataset ready to be fitted within snips-nlu!
Adapters
For now, only the snips adapter has been done. Here is a list of adapters and their respective properties:
adapter |
snips |
---|---|
type (1) |
✔️ with snips:type |
extensible (2) |
✔️ |
Specific type of the entity to use (such as datetime, temperature and so on)
Are values outside of training samples allowed?
Testing
$ pip install -e .[test]
$ python -m nose --with-doctest -v
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
pychatl-1.2.2.tar.gz
(6.2 kB
view hashes)