attest-detector 0.1.0

Detect what percentage of a text was written by AI, with per-sentence and per-paragraph breakdowns.

attest-detector

Detect what percentage of a piece of text was written by AI.

attest-detector is an offline-friendly Python package that analyzes text using three different AI detection models and reports each model's score separately — so you can make your own informed judgement rather than trusting a single black-box score.

Why three models?

No single AI detection model is reliable on its own for modern AI text. After testing, here's what we found:

Model	Catches AI text	False positives on human text	Best used for
fakespot	~100%	~66%	Flagging potential AI writing
roberta	~40%	~6%	Confirming text is human-written
chatgpt	~3%	~0%	"Definitely human" confirmation

Running all three together and reading them side by side gives a much more honest picture than any single score.

⚠️ First run: Each model (~500MB) is downloaded automatically and cached locally. Subsequent runs are fully offline.

---

Installation

pip install attest-detector

With PDF support:

pip install attest-detector[pdf]

With Word (.docx) support:

pip install attest-detector[docx]

---

Quick Start

from ai_detector import Detector

d = Detector()  # runs all three models by default
result = d.analyze("Your text goes here...")

print(result.summary())

Example Output

============================================================
  Overall: 48.8% likely AI-written  [Human]
  Primary model: fakespot
============================================================

  Per-model scores (full document):
    fakespot      100.0%  [✓ AI]
    roberta        40.3%  [✗ Human]
    chatgpt         2.9%  [✗ Human]

  Models flagging as AI: 1 / 3

  Analyzed 2 sentence(s) across 1 paragraph(s).

  Per-paragraph breakdown:
    Para 1:  48.8%  [█████████░░░░░░░░░░░]  [Human]  [1/3 models]

How to read the results

• All three models flag AI → Strong signal the text is AI-written
• Only fakespot flags AI → Uncertain — fakespot has many false positives
• All three say Human → Very likely human-written
• roberta and chatgpt both say Human → Strong signal text is human-written

---

Per-Sentence and Per-Paragraph Breakdown

# Per-sentence scores
for sentence in result.sentences:
    print(f"{sentence.ai_score:.1%}  [{sentence.label}]  {sentence.text}")
    print(f"  Model scores: {sentence.model_scores}")

# Per-paragraph scores
for para in result.paragraphs:
    print(f"{para.ai_score:.1%}  [{para.label}]")
    print(f"  {para.models_flagging_ai()}/{len(para.model_scores)} models flagged as AI")

---

Single Model Mode

# Use a specific model instead of all three
d = Detector(model='fakespot')   # best at catching AI
d = Detector(model='roberta')    # best at confirming human
d = Detector(model='chatgpt')    # near-zero false positives

---

Adjusting Sensitivity

# More sensitive — flags more text as AI
d = Detector(threshold=0.3)

# Less sensitive — only flags strongly AI text
d = Detector(threshold=0.7)

---

CLI Usage

After installation, use attest-detect directly in your terminal — no Python needed:

# Analyze a text file (runs all three models)
attest-detect essay.txt

# Analyze a PDF
attest-detect paper.pdf

# Analyze a Word document
attest-detect report.docx

# Analyze text directly
attest-detect --text "Paste your text here"

# Use a specific model
attest-detect essay.txt --model fakespot

# Show per-sentence breakdown
attest-detect essay.txt --sentences

# Adjust sensitivity
attest-detect essay.txt --threshold 0.3

# List available models
attest-detect --list-models

---

API Reference

Detector(model=None, threshold=0.5, device=None)

Parameter	Type	Default	Description
model	str	None	Run a single model: 'roberta', 'chatgpt', 'fakespot'. If not set, runs all three.
threshold	float	0.5	AI cutoff. Lower = more sensitive
device	str	None	'cpu', 'cuda', 'mps'. Auto-detected

DetectionResult

Attribute/Method	Type	Description
ai_score	float	Primary model's overall AI probability (0–1)
label	str	'AI' or 'Human'
model_scores	dict	Every model's score e.g. {'fakespot': 1.0, 'roberta': 0.4, 'chatgpt': 0.03}
sentences	list	Per-sentence Segment list
paragraphs	list	Per-paragraph Segment list
models_flagging_ai()	int	Count of models that scored above threshold
summary()	str	Formatted text summary

Segment

Attribute/Method	Type	Description
text	str	Original text
ai_score	float	Primary model's AI probability
model_scores	dict	Every model's score for this segment
label	str	'AI' or 'Human'
segment_type	str	'sentence' or 'paragraph'
models_flagging_ai(threshold)	int	Count of models above threshold

---

Available Models

Key	Model	Trained On	Size
fakespot	fakespot-ai/roberta-base-ai-text-detection-v1	Modern AI text	~500MB
roberta	openai-community/roberta-base-openai-detector	GPT-2 output (2019)	~500MB
chatgpt	Hello-SimpleAI/chatgpt-detector-roberta	ChatGPT output (2023)	~500MB

---

Limitations

• No single model reliably detects all modern AI text — use all three together
• All models are English-only
• Very short texts (< 2 sentences) may produce unreliable scores
• AI detection is probabilistic — treat results as signals, not verdicts
• Paraphrased or lightly edited AI text may score lower than expected

---

Running Tests

pip install -e ".[dev]"
pytest tests/ -v

---

Contributing

Contributions are welcome! Please open an issue first to discuss changes.

1. Fork the repo
2. Create a feature branch (git checkout -b feature/my-feature)
3. Commit your changes
4. Push and open a Pull Request

---

License

MIT — see LICENSE for details.