Mathics3-Module-nltk 10.0.0

Mathics3 Natural Language Toolkit module

Mathics3 Natural Language Toolkit module.

Example Session

$ mathicsscript
In[1]:= LoadModule["pymathics.natlang"]
Out[1]= pymathics.natlang
In[2]= Pluralize["try"]
Out[2]= tries
In[3]= WordFrequency["Apple Tree and apple", "apple", IgnoreCase -> True]
Out[3]= 0.5
In[4] = TextCases["I was in London last year.", "Pronoun"]
Out[4]= {I}
In[5] = Synonyms["forget"]
Out[5]= {blank out, block, bury, draw a blank, leave}
In[6] = Antonyms["big"]
Out[6]= {little, small}
In[7] = LanguageIdentify["eins zwei drei"]
Out[7]= "German"

Other examples can be found in the test file.

Installing and Running

To build this Mathics3 module, you will need to install the Python module nltk and spacy, and then install some data from Language-specific words:

$ make develop  # or make install

The above make command defaults to the English. If you would like to install for another language, set the variable LANG. For example:

$ make develop LANG=fr

To use the Extended Open Multilingual Wordnet with NLTK and use even more languages, you need to install them manually. Go to http://compling.hss.ntu.edu.sg/omw/summx.html, download the data, and then create a new folder under $HOME/nltk_data/corpora/omw/your_language where you put the file from wiki/wn-wikt-your_language.tab, and rename it to wn-data-your_language.tab.

If you get the message:

OSError: [E050] Can't find model 'en'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

There is a problem with the spacy library for advanced Natural Language Processing in Python.

You might be able to fix this by running:

python -m spacy download en

Adjust “python” and “en” (the language you want) above as needed.

User customization

To use the Extended Open Multilingual Wordnet (OMW) with ‘NLTK’ and use even more languages, you need to install them manually.

Go to http://compling.hss.ntu.edu.sg/omw/summx.html, download the data, and then create a new folder under $HOME/nltk_data/corpora/omw/your_language where you put the file from wiki/wn-wikt-your_language.tab, and rename it to wn-data-your_language.tab.

Adding more languages to Open Multilingual Wordnet:

To use the Extended Open Multilingual Wordnet with NLTK and use even more languages, you need to install them manually. Go to http://compling.hss.ntu.edu.sg/omw/summx.html, download the data, and then create a new folder under $HOME/nltk_data/corpora/omw/your_language where you put the file from wiki/wn-wikt-your_language.tab, and rename it to wn-data-your_language.tab.