Aletheia-Probe: Automated Integrity Checks for Academic Journals & Conferences

Aletheia-Probe is a command-line tool for evaluating the legitimacy of academic journals and conferences. By aggregating data from authoritative sources and applying advanced pattern analysis, it helps researchers, librarians, and institutions detect predatory venues and ensure the integrity of scholarly publishing.

Dual-Purpose Tool: Aletheia-Probe serves both individual researchers checking publication venues and as research infrastructure for empirical studies on scholarly publishing. Beyond individual use, the tool supports systematic literature reviews, bibliometric analysis, and meta-research workflows leveraging 240M+ publication records. For detailed research applications, see the Research Applications Guide.

Setting Realistic Expectations: If you are an honest researcher using established, well-known journals and conferences through major search engines and databases, you will likely never encounter predatory venues. This tool functions like a virus scanner for academic publishing—you should have it installed and running, but hopefully never receive any warnings. It's designed to catch the edge cases and protect against the less obvious threats that might slip through normal research workflows.

About the Name: The name "Aletheia" (ἀλήθεια) comes from ancient Greek philosophy, where it represents the concept of truth and unconcealment. In Greek mythology, Aletheia was personified as the goddess or spirit (daimona) of truth and sincerity. This reflects the tool's core mission: to reveal the truth about academic journals and conferences, helping researchers distinguish legitimate venues from predatory ones.

System Requirements

Aletheia-Probe requires external system dependencies beyond Python. Before installation, ensure your system has the required programs installed. Platform-specific installation instructions are available in the Quick Start Guide.

Required System Dependencies:

• Git - Required for retraction data synchronization
• Python 3.10+ - Runtime environment

TL;DR

Aletheia-Probe helps answer two critical questions for researchers:

1. Is the journal I want to publish in legitimate?

   aletheia-probe journal "Journal of Computer Science"
   

2. Are the references in my paper legitimate?

   aletheia-probe bibtex references.bib
   

# Install from PyPI or source

# Option 1: Install from PyPI (recommended)
pip install aletheia-probe

# Option 2: Install from source (for development)
git clone https://github.com/sustainet-guardian/aletheia-probe.git
cd aletheia-probe
pip install -e .

# First time: Sync data sources (takes a few minutes)
aletheia-probe sync

# Check the current state of the cache database
aletheia-probe status

# Assess a single journal
aletheia-probe journal "Journal of Computer Science"

# Sync acronym dataset used for acronym-based venue expansion
aletheia-probe acronym sync

# Assess using a journal acronym after syncing the acronym dataset
aletheia-probe journal "JMLR"

# Assess all journals in a BibTeX file (returns exit code 1 if predatory journals found)
aletheia-probe bibtex references.bib

# Get detailed analysis with confidence scores from multiple sources
aletheia-probe journal --format json "Nature Reviews Drug Discovery"

Output: Combines data from multiple authoritative sources and advanced pattern analysis to provide confidence-scored assessments of journal legitimacy.

Note: The first sync downloads and processes data from multiple sources (DOAJ, Beall's List, UGC-CARE discontinued lists, etc.), which takes a few minutes. After that, queries typically complete in under 5 seconds.

Data Sources

This tool acts as a data aggregator - it doesn't provide data itself, but combines information from multiple authoritative sources:

• DOAJ - Directory of Open Access Journals
• Beall's List - Historical predatory journal archives
• UGC-CARE Cloned (Group I) - UGC-CARE discontinued cloned journal list
• UGC-CARE Cloned (Group II) - UGC-CARE discontinued cloned journal list
• UGC-CARE Delisted (Group II) - UGC-CARE discontinued delisted journal list
• UGC-CARE Included from Clone Page (Group I/II) - Left-side included journals from public clone-correction pages
• Algerian Ministry - Algerian Ministry of Higher Education predatory journals list
• OpenAlex - Publication pattern analysis
• Crossref - Metadata quality assessment
• Retraction Watch - Journal retraction history analysis
• PubMed/NLM - MEDLINE and NLM Catalog journal legitimacy signal for biomedical venues
• Scopus - Optional premium journal database
• Institutional Lists - Custom whitelist/blacklist configurations
• Cross-Validator - Cross-source consistency validation system
• Kscien Standalone Journals - Individual predatory journals identified by Kscien
• Kscien Publishers - Known predatory publishers
• Kscien Hijacked Journals - Legitimate journals that have been hijacked by predatory actors
• Kscien Predatory Conferences - Database of predatory conferences
• DBLP Venues - Curated computer science journals and conference series from DBLP XML
• CORE Conferences (ICORE/CORE) - Ranked conference venues from the CORE portal
• CORE Journals (legacy) - Ranked journals from the CORE journal portal (latest list: CORE2020)

The tool analyzes publication patterns, citation metrics, and metadata quality to provide comprehensive coverage beyond traditional blacklist/whitelist approaches.

Note on Conference Assessment: Conference checking now combines curated predatory conference signals (Kscien) with curated legitimacy signals from DBLP and CORE/ICORE ranked venues. Coverage is strongest for computer science venues.

Quick Start

See the Quick Start Guide for installation and basic usage examples.

Assessment Methodology

The tool uses a hybrid approach combining curated databases with advanced pattern analysis to achieve comprehensive coverage and high accuracy.

Backend Types

Curated Databases (High Trust)

These provide authoritative yes/no decisions for journals they cover:

Pattern Analysis (Evidence-Based)

These analyze publication patterns and metadata quality to detect predatory characteristics:

How Assessment Works

1. Multi-Backend Query

The tool queries all enabled backends concurrently for comprehensive coverage:

Journal Query → [Curated Databases + Pattern Analyzers] → Combined Assessment
                 │
                 ├─ DOAJ (legitimate OA)
                 ├─ Scopus (indexed journals)
                 ├─ Beall's List (predatory)
                 ├─ UGC-CARE discontinued lists
                 ├─ PredatoryJournals.org
                 ├─ Kscien databases
                 ├─ CORE Conferences / CORE Journals
                 ├─ Retraction Watch (quality)
                 ├─ OpenAlex Analyzer (patterns)
                 ├─ Crossref Analyzer (metadata)
                 └─ Cross-Validator (consistency)

Note: Not all backends will find every journal. A journal may be:

• Found in DOAJ → strong legitimate evidence
• Found in Beall's → strong predatory evidence
• Found in UGC-CARE cloned/delisted lists → strong predatory evidence
• Not found in any curated database → rely on pattern analysis
• Found in contradictory sources → cross-validation resolves conflicts

Normalization gate:

• Input is normalized into one backend contract (normalized_venue) before backend queries.
• If provided name and ISSN/eISSN disagree after lookup validation, the tool returns INSUFFICIENT_DATA (no forced union of mismatched inputs).

2. Assessment Logic

Curated Database Results (Authoritative):

• DOAJ/Scopus match → Classified as legitimate (high confidence)
• Predatory list match → Classified as predatory (high confidence)
• No matches found → Proceed to pattern analysis

Pattern Analysis (Evidence-Based): When curated databases don't have the journal, pattern analyzers evaluate quality:

🟢 Legitimacy Indicators (OpenAlex/Crossref):

• Consistent publication volume (20-500 papers/year)
• Healthy citation patterns (>3 citations/paper average)
• Complete metadata (abstracts >100 chars, references, author ORCIDs)
• Recognized publisher with history
• Stable growth patterns

🔴 Predatory Indicators (OpenAlex/Crossref):

• Publication mill patterns (>1000 papers/year)
• Extremely low citations (<0.5/paper)
• Incomplete metadata (no abstracts, missing author info)
• Suspicious/unknown publisher
• Sudden publication volume spikes

3. Confidence Scoring

Final confidence is determined by:

• Source authority: DOAJ/Scopus > Pattern analysis > Smaller lists
• Agreement: Multiple sources agreeing → higher confidence
• Evidence strength: Strong indicators > weak signals
• Cross-validation: Consistent data across sources increases confidence
• Retraction data: High retraction rates lower confidence for "legitimate" journals

4. Result Combination

The dispatcher aggregates all backend results:

• Conflicting assessments are resolved by source weight
• Multiple agreeing sources boost confidence
• Pattern analysis supplements curated databases
• Detailed reasoning explains the assessment

Example Assessment Scenarios

Scenario 1: Well-Known Legitimate Journal

Input: "Nature"
│
├─ DOAJ: ✗ Not found (subscription journal, not open access)
├─ Scopus: ✓ Found → "legitimate"
├─ Predatory Lists: ✗ Not found
├─ Retraction Watch: ✓ Found → 153 retractions, 0.034% rate (within normal)
├─ OpenAlex: ✓ Found → 446,231 publications, healthy citations
├─ Crossref: ✓ Found → Complete metadata, Nature Publishing Group
│
Result: LEGITIMATE (confidence: 0.95)
Reasoning: "Found in Scopus with excellent publication patterns and metadata quality"

Scenario 2: Known Predatory Journal

Input: "Journal Appearing in UGC-CARE Cloned Group II"
│
├─ DOAJ: ✗ Not found
├─ Predatory Lists: ✓ Found in UGC-CARE Cloned (Group II) → "predatory"
├─ Retraction Watch: ✗ Not found
├─ OpenAlex: ✓ Found → suspicious pattern indicators
├─ Crossref: ✓ Found → weak metadata quality
│
Result: PREDATORY (confidence: 0.90)
Reasoning: "Listed in UGC-CARE cloned journal list, corroborated by pattern analysis"

Scenario 3: Unknown Journal (Pattern Analysis)

Input: "Emerging Regional Journal"
│
├─ DOAJ: ✗ Not found
├─ Scopus: ✗ Not found
├─ Predatory Lists: ✗ Not found
├─ OpenAlex: ✓ Found → 150 papers/year, 5 citations/paper average
├─ Crossref: ✓ Found → Good metadata, established publisher
│
Result: INSUFFICIENT_DATA (confidence: 0.45)
Reasoning: "Not in major databases; pattern analysis suggests legitimate practices but low confidence"

Optional: Manually Provided Data Sources

Three backends read a file that you download yourself, because their providers offer no directly downloadable URL:

See Manually Provided Data Sources for download links and where to place each file so that aletheia-probe sync picks it up.

Documentation

User Documentation

• Quick Start Guide - Installation and basic usage
• User Guide - Comprehensive usage examples and features
• Manually Provided Data Sources - Scopus, DOAJ and DBLP files you download yourself
• Research Applications - Using the tool in research workflows (SLR, bibliometrics, meta-research)
• Media Mentions - Curated external coverage and institutional references
• Configuration Reference - All configuration options
• Troubleshooting - Common issues and solutions

Developer Documentation

• API Reference - Backend API and data models
  • Backend API - Creating custom backends
  • Data Models - Core data structures
  • Extending Guide - Extension patterns
• Backend Integration Docs - Source-specific backend documentation
• Contributing Guide - Development setup and guidelines
• Coding Standards - Code quality requirements

Paper and Citation

This tool and its methodology are described in the following academic paper:

Aletheia-Probe: A Tool for Automated Journal Assessment Available at: https://arxiv.org/abs/2601.10431

To cite this work, please use the following BibTeX entry:

@misc{florath2026aletheiaprobetoolautomatedjournal,
      title={Aletheia-Probe: A Tool for Automated Journal Assessment},
      author={Andreas Florath},
      year={2026},
      eprint={2601.10431},
      archivePrefix={arXiv},
      primaryClass={cs.DL},
      url={https://arxiv.org/abs/2601.10431},
}

Funding Acknowledgment

This work was funded by the Federal Ministry of Research, Technology and Space (BMFTR) in Germany under the grant number 16KIS2251 of the SUSTAINET-guardian project. The views expressed are those of the author.

License

MIT License - see LICENSE file for details.