A python package to semantically link two lists of texts.
Project description
🖇️ NLP Link
NLP Link finds the most similar word (or words) in a reference list to an inputted word. For example, if you are trying to find which word is most similar to 'puppies' from a reference list of ['cats', 'dogs', 'rats', 'birds']
, nlp-link will return 'dogs'.
🗺️ SOC Mapper
Another functionality of this package is using the linking methodology to find the Standard Occupation Classification (SOC) code most similar to an inputted job title. More on this here.
🔨 Usage
Install the package using pip:
pip install nlp-link
Basic usage
Match two lists in python:
from nlp_link.linker import NLPLinker
nlp_link = NLPLinker()
# list inputs
comparison_data = ['cats', 'dogs', 'rats', 'birds']
input_data = ['owls', 'feline', 'doggies', 'dogs','chair']
nlp_link.load(comparison_data)
matches = nlp_link.link_dataset(input_data)
# Top match output
print(matches)
Which outputs:
input_id input_text link_id link_text similarity
0 0 owls 3 birds 0.613577
1 1 feline 0 cats 0.669633
2 2 doggies 1 dogs 0.757443
3 3 dogs 1 dogs 1.000000
4 4 chair 0 cats 0.331178
SOC Mapping
Match a list of job titles to SOC codes:
from nlp_link.soc_mapper.soc_map import SOCMapper
soc_mapper = SOCMapper()
soc_mapper.load()
job_titles=["data scientist", "Assistant nurse", "Senior financial consultant - London"]
soc_mapper.get_soc(job_titles, return_soc_name=True)
Which will output
[((('2433/04', 'Statistical data scientists'), ('2433', 'Actuaries, economists and statisticians'), '2425'), 'Data scientist'), ((('6131/99', 'Nursing auxiliaries and assistants n.e.c.'), ('6131', 'Nursing auxiliaries and assistants'), '6141'), 'Assistant nurse'), ((('2422/02', 'Financial advisers and planners'), ('2422', 'Finance and investment analysts and advisers'), '3534'), 'Financial consultant')]
Contributing
The instructions here are for those contrbuting to the repo.
Set-up
In setting up this project we ran:
conda create --name nlp-link pip python=3.9
conda activate nlp-link
pip install poetry
pip install pre-commit black
pre-commit install
poetry init
poetry install
Tests
To run tests:
poetry run pytest tests/
Documentation
Docs for this repo are automatically published to gh-pages branch via. Github actions after a PR is merged into main. We use Material for MkDocs for these. Nothing needs to be done to update these.
However, if you are editing the docs you can test them out locally by running
cd docs
<!-- pip install -r docs/requirements.txt -->
mkdocs serve
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file nlp_link-0.1.3.tar.gz
.
File metadata
- Download URL: nlp_link-0.1.3.tar.gz
- Upload date:
- Size: 162.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.9.19 Darwin/23.6.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | a1870b0d8ebc14ca000f3afc795455ba396f12a6bd29c9c6ee097a85c482d9df |
|
MD5 | 35a06f985145829d030740cac37b2696 |
|
BLAKE2b-256 | e4933eec0292fd325d6eeb4c62aa3755595001d105a070e013bff875cde9afc0 |
File details
Details for the file nlp_link-0.1.3-py3-none-any.whl
.
File metadata
- Download URL: nlp_link-0.1.3-py3-none-any.whl
- Upload date:
- Size: 162.3 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.9.19 Darwin/23.6.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93413e7be99d998fefe0b386db932d1ae3ddfccfcfed8c42cffc817a974084fe |
|
MD5 | 8d27a9e23806fb1eb028aaeca96917ac |
|
BLAKE2b-256 | adfa5d438c730207755f1df7513499c253851ecc4a3f8960a9d4bb279843ab02 |