Compare sentences from input document with all sentences from reference documents - find very similar ones.
Project description
Plagiarism Checker
This is a command-line tool for checking the similarity between a given text and a set of reference documents. The tool uses the Jaccard similarity algorithm to compare the input text with the reference documents.
Installation
Install in an isolated environment using pipx (or normal pip):
pipx install sentence-plagiarism
CLI Usage
To run the plagiarism checker, use the following command:
sentence-plagiarism <path-to-input-file> <path-to-reference-file-1> <path-to-reference-file-2> ... [--threshold <threshold-value>] [--output_file <path-to-output-file>] [--quiet]
<path-to-input-file>
: Path to the input file to be checked for plagiarism.<path-to-reference-file-1> ...
: Paths to the reference files to compare against.--threshold
: (optional) The minimum similarity score required to consider a sentence as plagiarized. The value should be between 0 and 1.--output-file
(optional): Path to the output file to save the results in JSON format.--quiet
(optional): Flag to suppress the display of similar sentences in the console.
Example
The following command:
sentence-plagiarism input.txt --reference-files ref1.txt ref2.txt --similarity-threshold 0.8 --output-file results.json
can produce the following output on stdout:
Input Sentence: The retriever and seq2seq modules commence their operations as pretrained models, and through a joint fine-tuning process, they adapt collaboratively, thus enhancing both retrieval and generation for specific downstream tasks.
Reference Sentence: foobar The retriever and seq2seq modules commence their operations as pretrained models, and through a joint fine-tuning process, they adapt collaboratively, thus enhancing both retrieval and generation for specific downstream tasks.
Reference Document: ref1.txt
Similarity Score: 0.9667
Input Sentence: Closing thoughts For a comprehensive understanding of the RAG technique, we offer an in-depth exploration, commencing with a simplified overview and progressively delving into more intricate technical facets.
Reference Sentence: barfoo For a comprehensive understanding of the RAG technique, we offer an in-depth exploration, commencing with a simplified overview and progressively delving into more intricate technical facets.
Reference Document: ref2.txt
Similarity Score: 0.8966
Results saved to results.json
and save results to results.json
.
Programmatic Usage
from sentence_plagiarism import check
check(
examined_file="txt/txt1.txt",
reference_files=["txt/txt2.txt", "txt/txt3.txt"],
similarity_threshold=0.8,
output_file=None,
quiet=False,
)
License
Distributed under the MIT License. See LICENSE
for more information.
Contact
Krystian Safjan - ksafjan@gmail.com
Project Link: https://github.com/izikeros/plagiarism-checker
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for sentence_plagiarism-0.2.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 73d0b45d70a6e863a452727f005bc25e8fbfddbe92aef2e89182d91cb7d923f9 |
|
MD5 | b14958c5b04d61f24bae39c60e8eb168 |
|
BLAKE2b-256 | aa57b939669896a22872e9514f95fd22f9bf4709cbd26a23c31abc00c1cff23a |
Hashes for sentence_plagiarism-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 390b5d74608e10c50b6412a0e1881c09109e0dfb37c7fdb7af8a5d2d0e1210cf |
|
MD5 | 4c435a22d35a0a9e7d14cea240aae33a |
|
BLAKE2b-256 | 3605bf9d030e336f49d6edd53cb58033b002a4d2a82f8009f6b3d3c39e3008df |