✏️ rpunct - Restore Punctuation
This repo contains code for Punctuation restoration.
This package is intended for direct use as a punctuation restoration model for the general English language. Alternatively, you can use this for further fine-tuning on domain-specific texts for punctuation restoration tasks.
It uses HuggingFace's bert-base-uncased model weights that have been fine-tuned for Punctuation restoration.
Punctuation restoration works on arbitrarily large text. And uses GPU if it's available otherwise will default to CPU.
List of punctuations we restore:
- Upper-casing
- Period: .
- Exclamation: !
- Question Mark: ?
- Comma: ,
- Colon: :
- Semi-colon: ;
- Apostrophe: '
- Dash: -
🚀 Usage
Below is a quick way to get up and running with the model.
- First, install the package.
pip install rpunct
- Sample python code.
from rpunct import RestorePuncts
# The default language is 'english'
rpunct = RestorePuncts()
rpunct.punctuate("""in 2018 cornell researchers built a high-powered detector that in combination with an algorithm-driven process called ptychography set a world record
by tripling the resolution of a state-of-the-art electron microscope as successful as it was that approach had a weakness it only worked with ultrathin samples that were
a few atoms thick anything thicker would cause the electrons to scatter in ways that could not be disentangled now a team again led by david muller the samuel b eckert
professor of engineering has bested its own record by a factor of two with an electron microscope pixel array detector empad that incorporates even more sophisticated
3d reconstruction algorithms the resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves""")
# Outputs the following:
# In 2018, Cornell researchers built a high-powered detector that, in combination with an algorithm-driven process called Ptychography, set a world record by tripling the
# resolution of a state-of-the-art electron microscope. As successful as it was, that approach had a weakness. It only worked with ultrathin samples that were a few atoms
# thick. Anything thicker would cause the electrons to scatter in ways that could not be disentangled. Now, a team again led by David Muller, the Samuel B.
# Eckert Professor of Engineering, has bested its own record by a factor of two with an Electron microscope pixel array detector empad that incorporates even more
# sophisticated 3d reconstruction algorithms. The resolution is so fine-tuned the only blurring that remains is the thermal jiggling of the atoms themselves.
🎯 Accuracy
Here is the number of product reviews we used for finetuning the model:
| Language | Number of text samples |
|---|---|
| English | 560,000 |
We found the best convergence around 3 epochs, which is what presented here and available via a download.
The fine-tuned model obtained the following accuracy on 45,990 held-out text samples:
| Accuracy | Overall F1 | Eval Support |
|---|---|---|
| 91% | 90% | 45,990 |
💻🎯 Further Fine-Tuning
To start fine-tuning or training please look into training/train.py file.
Running python training/train.py will replicate the results of this model.
☕ Contact
Contact Daulet Nurmanbetov for questions, feedback and/or requests for similar models.
Metadata
Release files for rpunct 1.0.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| rpunct-1.0.2.tar.gz | 5.6 kB | Without provenance | Details for rpunct-1.0.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| rpunct-1.0.2-py3-none-any.whl | Python 3 | none | any | Details for rpunct-1.0.2-py3-none-any.whl |
Total release size: 11.5 kB
Release files / rpunct-1.0.2.tar.gz
| Download URL | rpunct-1.0.2.tar.gz |
|---|---|
| Size | 5.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
33415c5efde858b0b4fb3538eb45372fb13bb4440771d106bf798bba8990c8cb
|
|
BLAKE2b-256 checksum How to use checksums |
073267bfdf0c229e26e2fe70df09db89d7d33c495607d9521cc2a7e289af0ac9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.8
|
Release files / rpunct-1.0.2-py3-none-any.whl
| Download URL | rpunct-1.0.2-py3-none-any.whl |
|---|---|
| Size | 5.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
846390eedf8ab7825d82d13e724ea3c58613a5337f2da9bab2ddcd15d14245a5
|
|
BLAKE2b-256 checksum How to use checksums |
969ab8c77cf4105d813e999522c298c25481d3bb2fa41c6efdab156190397647
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.4.1 importlib_metadata/4.0.1 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.60.0 CPython/3.8.8
|
Security for rpunct 1.0.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.