fraudurl: fast, offline phishing URL detector

A phishing URL checker for Python: check any URL, or a whole CSV of links, for phishing, with a calibrated probability and plain-English reasons. Free, open source, one Python file.
Website: amruth112.github.io/fraudurl-detector

Quick start · How it works · Accuracy · Usage · Limitations · Plain-English guide · Report · Model card

What it does

Give fraudurl a CSV of links. It returns the same CSV with three things added to every row:

• a verdict: FRAUD, LEGITIMATE, or REVIEW when the address alone is not conclusive;
• a calibrated probability;
• the three plain-English reasons behind the verdict.

It judges the URL itself, using 83 clues such as a brand in the wrong place, login or payment words, free-hosting addresses, risky domain endings, http:// and random-looking names. It never opens the website.

• One file, no dependencies. The whole tool, models included, is one ~450 KB Python file that runs on the Python 3.9+ standard library.
• Fast. It checks about 1,000 URLs a second on one CPU core and 2,600–3,200 a second on four, with flat memory (about 230 MB on 4 cores).
• Explains itself. The reasons are the model's own per-feature contributions, not a separate heuristic.
• Says when it is unsure. Ambiguous URLs go to REVIEW for a person instead of being guessed.
• Private by default. Offline mode never touches the network. The optional lookups ask DNS and the domain registry about the domain, never the website.

Quick start

Option 1: the single file.

1. Download fraudurl_standalone.py. Every release also attaches it with a SHA256 checksum.
2. Run:

python fraudurl_standalone.py my_urls.csv          # writes my_urls.fraudurl.csv

Option 2: install the package.

pip install fraudurl
fraudurl my_urls.csv
fraudurl --url "http://paypal.com.secure-login.test/webscr/login.php" --quiet   # one URL -> JSON

Both give identical results. CI checks this on every change, on Linux, macOS and Windows.

How it works

1. Read the URL the way a browser does. It splits off tricks such as paypal.com. in the subdomain and uses the Public Suffix List to find the domain someone actually bought, secure-login.test in the example.
2. Measure 83 clues. These cover structure, character mix, randomness, redirect tricks, brand and login words, free-hosting platforms, and the phishing history of the domain ending.
3. Score. A 400-tree gradient-boosted model, trained with LightGBM and exported to JSON, runs in about 60 lines of plain Python. Its score is calibrated into a probability, and two cut-offs turn that into a verdict.
4. Explain. Each tree's contribution is credited to the clue it used, and the three biggest become the reasons.
5. Optionally look closer. For unsure URLs, DNS records and the domain's registration age feed a second model that corrects the score.

The step-by-step version, with a worked example, is in HOW_IT_WORKS.md.

How accurate is it?

Every number here is measured on URLs from domains the model never saw in training: held-out test splits of its three training datasets, plus one dataset (Ariyadasa 2021) never used for training at all, 2020 to 2026.

• Good at: clearing ordinary URLs, catching classic phishing patterns, and handing the ambiguous rest to a person.
• Weaker on unfamiliar data: older or very different data is harder.
• Low-prevalence traffic: when phishing is rare in your traffic, many FRAUD rows will be false alarms, so use --base-rate.
• Slightly optimistic: the final set-up was chosen partly by looking at these test results.
• More detail: MODEL_CARD.md and the full engineering report, which includes a comparison with the Laya language model, 94–583× slower per URL here.

Usage

fraudurl input.csv                         # offline mode, auto-detects the URL column
fraudurl input.csv -o results.csv
fraudurl input.csv --url-column "Website"  # pick the column explicitly
fraudurl input.csv --enrich-review         # DNS + registration lookups, only for REVIEW rows (recommended)
fraudurl input.csv --enrich                # DNS + registration lookups for every URL (slowest)
fraudurl input.csv --block-list block.txt --allow-list allow.txt   # your own decisions win
fraudurl input.csv --base-rate 0.02        # re-weight probabilities for ~2% expected fraud
fraudurl --url https://example.com/login --enrich-review   # one URL -> one JSON line on screen
fraudurl input.csv --format json           # JSON Lines file (input.fraudurl.jsonl)
fraudurl input.csv -o -                    # write the CSV to the screen (stdout)
fraudurl input.csv --workers 1             # worker processes (default: up to 4)
fraudurl input.csv --enrich-review --cache-dir ./my_cache   # where lookups are cached (default ./.fraudurl_cache)

python fraudurl_standalone.py … accepts exactly the same options.

{"url": "...", "fraud_verdict": "REVIEW", "fraud_probability": 0.218, "verdict_confidence": 0.782,
 "top_reasons": ["..."], "registrable_domain": "...", "analysis_mode": "enriched (URL + DNS + RDAP)",
 "lookup_status": "dns=ok; rdap=ok", "error": null,
 "stages": {"offline_check": {"verdict": "REVIEW", "probability": 0.299},
            "lookups": {"dns": {"ipv4": ["..."], "nameservers": ["..."], "mail_servers": ["..."], "ttl_seconds": 3321},
                        "registration": {"registered": "1995-04-27T04:00:00Z", "expires": "...",
                                         "registrar": "...", "domain_age_days": 11477}}}}

python fraudurl_standalone.py examples/sample_urls.csv

Limitations

• Distribution shift. Trained on one collection and tested on another, ROC-AUC fell to 0.85–0.96. If your URLs look very different from the training data, expect less.
• Bare homepages of popular sites mostly land in REVIEW: 76% of 1,000 Tranco top-10k homepages, and 3% were called FRAUD.
• Shared platforms cut both ways.
  • Phishing on Google Docs, Forms or Sites, or Weebly, can look legitimate. The Google Forms row (id 27) in examples/sample_urls.fraudurl.csv is called LEGITIMATE.
  • Honest sites on heavily abused free hosts (pages.dev, webflow.io) are often called FRAUD.
• Other misses. Hacked legitimate sites and carefully disguised URLs are hard to catch from the URL alone.
• Ageing. The model reflects 2020–2026 data and should be retrained periodically.

It is a fast first-line filter for triage, not a replacement for a full phishing-defence stack.

Safety and privacy

• Offline mode (the default) never touches the network. Nothing leaves your computer.
• --enrich / --enrich-review send some data out:
  • host names go to Cloudflare's DNS-over-HTTPS resolver (1.1.1.1);
  • registrable domains go to the RDAP server of the domain's registry (at most 2 requests per second per registry);
  • IANA's RDAP bootstrap list is downloaded at most once every 30 days;
  • the path and query of your URLs are never sent.
• Lookups never touch the site. They never connect to the URL's web server, download pages or submit anything.
• Lookup cache. Results are cached in ./.fraudurl_cache/ (DNS for 7 days, registry data for 28 days), and several runs can share one cache folder.
• Registry terms. Some registries' terms restrict high-volume automated queries, so check before very large --enrich runs.
• No keys or paid services. No API keys, accounts or paid services are used.
• Spreadsheet formulas. Only the eight added columns are protected against formula injection. Your original cells are copied unchanged, so open output made from untrusted feeds with formulas disabled.

FAQ

How do I check if a link is phishing? Run fraudurl --url "https://example.com/login" (installed package) or python fraudurl_standalone.py --url "https://example.com/login" (single file). It prints the verdict, the probability of phishing and the reasons, without opening the link.

How do I check a list of links for phishing? Put the links in a CSV or text file and run fraudurl links.csv (or python fraudurl_standalone.py links.csv). You get the same file back with a verdict, probability and reasons on every row.

Does it detect malware or other scam links? It is built and tested for phishing URLs: fake login, payment and account pages. It has not been measured on malware downloads or other kinds of malicious or scam URLs, so use a malware scanner or blocklist for those too.

Is it safe to check suspicious links with it? Yes. fraudurl reads the address text only. It never opens, downloads or submits anything at the URLs it checks. In offline mode nothing is sent anywhere; with the optional lookups, the host name goes to Cloudflare's DNS.

How is it different from Google Safe Browsing or VirusTotal? Those are online services that check URLs against threat lists and, for VirusTotal, many vendors' verdicts, backed by far more data than any local model. fraudurl runs entirely on your machine, needs no account, checks a million-URL CSV in minutes and explains each verdict. Because it judges the address itself, it can also give a verdict on a URL that is not on any list yet. It has not been compared with those services; use them together.

Is it free? Yes. It is MIT-licensed, with no API keys, accounts or paid services.

Documentation

Repository layout

Data, licences and credits

• The code and the trained models are released under the MIT License.
• The bundled Public Suffix List is MPL-2.0 (LICENSES/MPL-2.0.txt).
• The models were trained on the PhreshPhish (CC BY 4.0) and Hannousse (CC BY 4.0) datasets, plus 2026 URLs from PhishTank, the OpenPhish community feed, Hacker News links and Common Crawl.
• No training data is redistributed.

Full credits and citations are in NOTICE.md and CITATION.cff.

Contributing and security

Contributions are welcome; see CONTRIBUTING.md. Report security problems privately, as described in SECURITY.md. Please never paste live phishing links into issues; defang them (hxxps://evil[.]example).