AI Detector: Detecting AI Generated Text
Overview
AI Detector is a Python module, based on PyTorch, that simplifies the process of training and deploying a classification model to detect whether a given text has been generated by AI. It is designed to be platform-agnostic, making AI detection capabilities accessible to users across different work environments.
Installation
There are two methods available for installing the AI Detector module:
-
Using pip: You can install AI Detector directly from PyPI using pip by running the following command:
pip3 install aidetector -
From this repository: Alternatively, you can clone this repository and install it locally:
git clone https://github.com/baileytec-labs/aidetector.git cd aidetector pip3 install .
Usage
AI Detector can be operated in two modes: training and inference.
Training
To train a new model, you need a CSV dataset with a classification column (labels: 0 for human-written and 1 for AI-generated text) and a text column (the text data). The script takes the following command-line arguments:
aidetector train --datafile [path_to_data] --modeloutputfile [path_to_model] --vocaboutputfile [path_to_vocab] --tokenmodel [SpaCy model] --percentsplit [percentage_for_test_split] --classificationlabel [classification_label_in_data] --textlabel [text_label_in_data] --download --lowerbound [lower_bound_for_early_stopping] --upperbound [upper_bound_for_early_stopping] --epochs [number_of_epochs]
Inference
To make predictions with a trained model, you need to provide the text you want to classify. The script takes the following command-line arguments:
aidetector infer --modelfile [path_to_trained_model] --vocabfile [path_to_vocab] --text [text_to_classify] --tokenmodel [SpaCy_model] --threshold [probability_threshold_for_classification] --download [flag_to_download_SpaCy_model]
The prediction will be printed to the console: "This was written by AI" or "This was written by a human."
Python API
You can use all the functionality of AiDetector in your python programs, it's as simple as starting with
from aidetector.aidetectorclass import *
from aidetector.inference import *
from aidetector.training import *
from aidetector.tokenization import *
#or
import aidetector as ad
From there, you have access to all of the training, inference, and tokenization capabilities.
for example,
#Getting inference of an AI model in python
from aidetector.tokenization import *
from aidetector.inference import *
from aidetector.aidetectorclass import *
tokenizer=get_tokenizer()
vocab=load_vocab("./myvocabfile.vocab")
model = AiDetector(len(vocab))
testtext="Is this written by AI?"
model.load_state_dict(torch.load("./mymodelfile.model"))
isai=check_input(
model,
vocab,
testtext,
tokenizer=tokenizer,
)
#returns 0 if human, 1 if AI.
Dependencies
The main dependencies for this project include:
PyTorch SpaCy Torchtext scikit-learn pandas argparse Halo
Note: For tokenization, the project uses SpaCy models. By default, it uses the multi-language model xx_ent_wiki_sm, but other models can be specified using the --tokenmodel argument. If the model is not already downloaded, you can use the --download flag to download the model.
Contributing
Contributions to the AI Detector project are welcome. Please review CONTRIBUTION.md for further instructions.
Metadata
Release files for aidetector 0.0.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| aidetector-0.0.2.tar.gz | 13.5 kB | Without provenance | Details for aidetector-0.0.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| aidetector-0.0.2-py3-none-any.whl | Python 3 | none | any | Details for aidetector-0.0.2-py3-none-any.whl |
Total release size: 27.6 kB
Release files / aidetector-0.0.2.tar.gz
| Download URL | aidetector-0.0.2.tar.gz |
|---|---|
| Size | 13.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
4e582d5d206671d5051bc9d32f3a5f8cbdd879f38de91e44a6fac63cce62032b
|
|
BLAKE2b-256 checksum How to use checksums |
f6a10dfe5c8499688333fe1fb59c730e52da81ece3af653624386a5738b8b756
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.11.4
|
Release files / aidetector-0.0.2-py3-none-any.whl
| Download URL | aidetector-0.0.2-py3-none-any.whl |
|---|---|
| Size | 14.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
3bc058ae0a443309cc72e1621ffde655b15a452fc925ab63ffd335f243b40214
|
|
BLAKE2b-256 checksum How to use checksums |
b9d320e3fae4136b417c8a765a2d03a5a64b890e861ecbb1af693152cac97d3f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.1 CPython/3.11.4
|
Security for aidetector 0.0.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.