Overview
BibDedupe is an open-source Python library for deduplication of bibliographic records, tailored for literature reviews. Unlike traditional deduplication methods, BibDedupe focuses on entity resolution, linking duplicate records instead of simply deleting them.
Features
- Automated Duplicate Linking with Zero False Positives: BibDedupe automates the duplicate linking process with a focus on eliminating false positives.
- Preprocessing Approach: BibDedupe uses a preprocessing approach that reflects the unique error generation process in academic databases, such as author re-formatting, journal abbreviation or translations.
- Entity Resolution: BibDedupe does not simply delete duplicates, but it links duplicates to resolve the entitity and integrates the data. This allows for validation, and undo operations.
- Programmatic Access: BibDedupe is designed for seamless integration into existing research workflows, providing programmatic access for easy incorporation into scripts and applications.
- Transparent and Reproducible Rules: BibDedupe's blocking and matching rules are transparent and easily reproducible to promote reproducibility in deduplication processes.
- Continuous Benchmarking: Continuous integration tests running on GitHub Actions ensure ongoing benchmarking, maintaining the library's reliability and performance across datasets.
- Efficient and Parallel Computation: BibDedupe implements computations efficiently and in parallel, using appropriate data structures and functions for optimal performance.
Documentation
Explore the official documentation for comprehensive information on installation, usage, and customization of BibDedupe.
Citation
If you use BibDedupe in your research, please cite it as follows:
Wagner, G. (2024) BibDedupe - An open-source Python library for deduplication of bibliographic records. Journal of Open Source Software, 9(97), 6318, https://doi.org/10.21105/joss.06318.
Contribution Guidelines
We welcome contributions from the community to enhance and expand BibDedupe. If you would like to contribute, please follow our contribution guidelines.
License
BibDedupe is released under the MIT License, allowing free and open use and modification.
Contact
For any questions, issues, or feedback, please open an issue on our GitHub repository.
Happy deduplicating with BibDedupe!
1 maintainer
Metadata
1 maintainer
Release files for bib-dedupe 0.11.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| bib_dedupe-0.11.0.tar.gz | 405.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| bib_dedupe-0.11.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 474.3 kB
Release files / bib_dedupe-0.11.0.tar.gz
| Download URL | bib_dedupe-0.11.0.tar.gz |
|---|---|
| Size | 405.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d61c26575e5f054e5207d8cea933c112a8b9e0151a8265ec8e9d0969325a95bf
|
|
BLAKE2b-256 checksum How to use checksums |
a827c17aed45c04a5a5c43662af47d88dd3a9615c0c5348187d03f4b51ed2f32
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.9.11 {"installer":{"name":"uv","version":"0.9.11"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / bib_dedupe-0.11.0-py3-none-any.whl
| Download URL | bib_dedupe-0.11.0-py3-none-any.whl |
|---|---|
| Size | 69.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
acc4f20f7ead74f88fba1a8e194e8cff8a79f6e99fbe5233c6aba38dda4a98e0
|
|
BLAKE2b-256 checksum How to use checksums |
9ee75528c262f460baee7ce7a3121cc065339bde62bf1ad6f7aeabfab5ef4da7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.9.11 {"installer":{"name":"uv","version":"0.9.11"},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
