codelang-detect
A fast, lightweight, regex-based programming language detector for Python.
Codelang-detect identifies the programming language of a given code snippet. It is designed from the ground up to be fast, accurate, and have zero external dependencies. It's the perfect tool for pre-processing code, routing files, or any application where you need a quick and reliable language check without pulling in heavy libraries.
Key Features
- ⚡️ Blazing Fast: Built on a system of weighted, compiled regular expressions. Performance is measured in microseconds.
- 🎯 Highly Accurate: Demonstrably more accurate than popular alternatives — both on a curated suite of real-world and tricky code snippets, and on independent third-party datasets (smola/language-dataset, CodeSearchNet) it was never tuned against.
- 📦 Zero Dependencies: Pure Python.
pip install codelang-detectis all you need. No heavyweight models, no external binaries. - 🔧 Simple API: A single function call:
detect(code). - 💻 CLI Included: Use it directly from your terminal or in shell scripts.
Why codelang-detect?
Many existing language detectors have significant trade-offs:
- Heavy ML Models (e.g.,
guesslang): Often have complex or outdated dependencies (like older TensorFlow versions) that make installation difficult. They are also significantly slower for single detections. - Comprehensive Tools (e.g.,
pygments): Excellent for syntax highlighting, but its primary goal isn't detection. As the benchmarks show, its guessing can be unreliable on complex snippets. - Platform-Specific Tools (e.g., GitHub's
linguist): The industry standard, but it's a Ruby Gem, making it difficult to integrate into a Python environment.
codelang-detect fills the gap for a "just right" solution: a lightweight, portable, and fast detector that delivers best-in-class accuracy.
Benchmark: Accuracy & Performance
The results speak for themselves. On a curated set of 60 code snippets designed to test real-world accuracy, codelang-detect is both significantly more accurate and faster than other popular, lightweight libraries.
| Library | Accuracy | Avg. Time / Sample (µs) | Dependencies |
|---|---|---|---|
codelang-detect (Ours) |
100% | ~1525 µs | None |
Pygments |
23.3% | ~4394 µs | None |
WhatsThatCode |
31.7% | ~10089 µs | None |
Benchmarks run on Python 3.13. Your results may vary.
As the results show, codelang-detect is not only the most accurate solution on this test suite but also ~3x faster than Pygments and ~7x faster than WhatsThatCode, all while maintaining zero dependencies.
Click to see detailed accuracy breakdown
--- Accuracy Benchmark ---
| Test Case | Expected | Codelang-Detect (Ours) | Pygments | WhatsThatCode |
--------------------------------------------------------------------------------------------------------------
| c_function_pointer | c | c ✅ | c ✅ | unknown ❌ |
| cbl_simple | cbl | cbl ✅ | componentpascal ❌ | unknown ❌ |
| cs_async_method | cs | cs ✅ | gdscript ❌ | cs ✅ |
| cs_full | cs | cs ✅ | gdscript ❌ | java ❌ |
| cs_lambda | cs | cs ✅ | scdoc ❌ | unknown ❌ |
| cs_linq_query | cs | cs ✅ | gdscript ❌ | unknown ❌ |
| cs_simple | cs | cs ✅ | unknown ❌ | java ❌ |
| cs_embedded_sql | cs | cs ✅ | objective-c ❌ | unknown ❌ |
| cs_linq | cs | cs ✅ | gdscript ❌ | unknown ❌ |
| css_scrollbar | css | css ✅ | cplint ❌ | cpp ❌ |
| dart_pojo | dart | dart ✅ | perl6 ❌ | js ❌ |
| go_http_server | go | go ✅ | py ❌ | go ✅ |
| groovy_basic | groovy | groovy ✅ | py ❌ | unknown ❌ |
| java_full | java | java ✅ | teratermmacro ❌ | unknown ❌ |
| java_simple | java | java ✅ | py ❌ | java ✅ |
| java_streams | java | java ✅ | py ❌ | unknown ❌ |
| java_pojo | java | java ✅ | carbon ❌ | unknown ❌ |
| js_arrow | js | js ✅ | gdscript ❌ | unknown ❌ |
| js_promise_fetch | js | js ✅ | gdscript ❌ | unknown ❌ |
| js_react_component | js | js ✅ | py ❌ | unknown ❌ |
| js_config | js | js ✅ | gdscript ❌ | unknown ❌ |
| js_es6 | js | js ✅ | py ❌ | js ✅ |
| json_dependencies | json | json ✅ | unknown ❌ | json ✅ |
| json_package | json | json ✅ | carbon ❌ | json ✅ |
| kt_coroutine | kt | kt ✅ | py ❌ | py ❌ |
| kt_data_class | kt | kt ✅ | ssp ❌ | unknown ❌ |
| kotlin_observer | kt | kt ✅ | py ❌ | unknown ❌ |
| php_router | php | php ✅ | javascript+php ❌ | unknown ❌ |
| py_async_http | py | py ✅ | py ✅ | unknown ❌ |
| py_class | py | py ✅ | perl6 ❌ | rb ❌ |
| py_helloworld | py | py ✅ | unknown ❌ | py ✅ |
| py_pandas | py | py ✅ | py ✅ | unknown ❌ |
| py_simple | py | py ✅ | py ✅ | unknown ❌ |
| py_service | py | py ✅ | py ✅ | py ✅ |
| py_sqlalchemy | py | py ✅ | py ✅ | py ✅ |
| py_fastapi_router | py | py ✅ | py ✅ | py ✅ |
| py_try_except | py | py ✅ | actionscript3 ❌ | java ❌ |
| py_unittest | py | py ✅ | tsql ❌ | py ✅ |
| rb_class | rb | rb ✅ | tsql ❌ | rb ✅ |
| rust_result | rs | rs ✅ | ecl ❌ | unknown ❌ |
| rs_library_code | rs | rs ✅ | carbon ❌ | unknown ❌ |
| scala_case_class | scala | scala ✅ | unknown ❌ | unknown ❌ |
| scala_future | scala | scala ✅ | py ❌ | unknown ❌ |
| scala_oop | scala | scala ✅ | tsql ❌ | unknown ❌ |
| sh_env_check | sh | sh ✅ | sh ✅ | sh ✅ |
| sh_shebang | sh | sh ✅ | sh ✅ | sh ✅ |
| sql_join | sql | sql ✅ | scdoc ❌ | unknown ❌ |
| sql_select | sql | sql ✅ | scdoc ❌ | unknown ❌ |
| swift_func | swift | swift ✅ | gdscript ❌ | unknown ❌ |
| swift_struct | swift | swift ✅ | gdscript ❌ | unknown ❌ |
| ts_interface | ts | ts ✅ | gdscript ❌ | unknown ❌ |
| ts_nestjs | ts | ts ✅ | py ❌ | unknown ❌ |
| yaml_dockercompose | yaml | yaml ✅ | scdoc ❌ | unknown ❌ |
| yaml_k8s | yaml | yaml ✅ | actionscript3 ❌ | unknown ❌ |
| plain_text | unknown | unknown ✅ | unknown ✅ | unknown ✅ |
| plain_text_doc | unknown | unknown ✅ | unknown ✅ | unknown ✅ |
| html_basic | html | html ✅ | html ✅ | html ✅ |
| html_form | html | html ✅ | xml ❌ | unknown ❌ |
| xml_simple | xml | xml ✅ | xml ✅ | xml ✅ |
| xml_maven | xml | xml ✅ | xml ✅ | xml ✅ |
--------------------------------------------------------------------------------------------------------------
--- Accuracy Summary ---
Codelang-Detect (Ours) : 60/60 correct (100.0%)
Pygments : 14/60 correct (23.3%)
WhatsThatCode : 19/60 correct (31.7%)
Note: Libraries like guesslang and enry were excluded from the final benchmark due to significant installation issues with modern Python versions and their respective dependencies.
External Benchmarks (independent datasets)
The 100% score above is on a curated suite we wrote ourselves, so we also ran
codelang-detect against two independent, real-world datasets it was never
tuned against, to see how the claim holds up outside the lab:
| Dataset | Samples | codelang-detect (Ours) |
Pygments | WhatsThatCode |
|---|---|---|---|---|
| smola/language-dataset (human-reviewed real GitHub files, 24 shared languages) | 453 | 78.8% (F1 0.768) | 14.8% (F1 0.113) | 14–21%* (F1 0.13–0.16) |
| CodeSearchNet (real GitHub functions, 6 languages) | 1,800 | 80.9% (F1 0.870) | 1.1% (F1 0.020) | 44–57%* (F1 0.44–0.60) |
* WhatsThatCode's "election" algorithm is non-deterministic between runs.
On real-world, independent data codelang-detect still beats both alternatives
by a wide margin — 5x Pygments, 1.5–3.5x WhatsThatCode — while staying an order
of magnitude faster than both on CodeSearchNet's short function-level samples.
It does not score 100% here, though: isolated functions and files without
strong idiomatic markers (e.g. a TypeScript file with no type annotations, or
a Groovy file whose only distinguishing content is in imports) are genuinely
ambiguous even to a human reader. Full methodology, per-language breakdowns,
and the regex fixes this exposed are in
benchmark/EXTERNAL_RESULTS.md.
Installation
pip install codelang-detect
Usage
As a Python Library
The API is dead simple. The detect function takes a string of code and returns the file extension of the detected language.
from codelang_detect import detect
# Example 1: Python
python_code = "class User:\n def __init__(self, name): self.name = name"
print(detect(python_code))
# Output: py
# Example 2: C#
csharp_code = "public class Person { public string Name { get; set; } }"
print(detect(csharp_code))
# Output: cs
# Example 3: Non-code
unknown_text = "This is just a regular sentence."
print(detect(unknown_text))
# Output: unknown
As a Command-Line Tool (CLI)
You can also use codelang-detect directly from your terminal to analyze files or stdin.
# Analyze a file
codelang-detect my_script.js
# Output: js
# Pipe content into the CLI
cat deployment.yaml | codelang-detect
# Output: yaml
Supported Languages
codelang-detect currently provides high-quality detection for the following languages, sorted by their returned extension:
- C (
c) - C++ (
cpp) - C# (
cs) - COBOL (
cbl) - CSS (
css) - Dart (
dart) - Go (
go) - Groovy (
groovy) - HTML (
html) - Java (
java) - JavaScript (
js) - JSON (
json) - Kotlin (
kt) - PHP (
php) - Python (
py) - R (
r) - Ruby (
rb) - Rust (
rs) - Scala (
scala) - Shell (
sh) - Solidity (
sol) - SQL (
sql) - Swift (
swift) - TypeScript (
ts) - XML (
xml) - YAML (
yaml)
How It Works
No magic here. codelang-detect uses a curated list of regular expressions for each language. Each regex is assigned a "weight" based on how uniquely it identifies a language.
For example:
- The pattern
async Task<is a very strong signal for C# and gets a high weight. - The keyword
defis a strong signal for Python but could also appear in Scala or Ruby, so it gets a moderate weight. - The keyword
classis a weak signal, as it appears in many languages, and requires more context to be useful.
The library runs all regexes against the input code, sums the weights for each language, and returns the language with the highest score. It's simple, transparent, and incredibly fast.
Running Tests
This project uses pytest for testing. To run the test suite, first install the development dependencies and then run pytest:
# Install development dependencies
pip install -r requirements-dev.txt
# Run the test suite
pytest
Contributing
Contributions are welcome and appreciated! This project was started to fill a gap, and community help is the best way to make it the definitive tool for this job.
Whether it's improving regexes, adding support for a new language, or fixing a bug, please feel free to:
- Open an issue to discuss the change.
- Fork the repository and submit a pull request.
When adding a language or fixing a misidentification, please add relevant code snippets to tests/test_data.json. This helps verify your changes and prevents future regressions. We follow a simple principle: if a human can't reliably distinguish a short snippet, the detector probably can't either, so focus on realistic test cases.
License
This project is licensed under the Apache 2.0 License - see the LICENSE file for details.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file codelang_detect-1.2.0.tar.gz.
File metadata
- Download URL: codelang_detect-1.2.0.tar.gz
- Upload date:
- Size: 21.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.13.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
be02917554bc1162ad446fccd805b9e3f7fb73f406a9c850b09aa30b2d35a347
|
|
| MD5 |
55b49ebcecf5e7110d87d824d6d7f866
|
|
| BLAKE2b-256 |
eb2e163be514135e35f389792a2ba6b8483a3ac064198a43892ff736daacce47
|
File details
Details for the file codelang_detect-1.2.0-py3-none-any.whl.
File metadata
- Download URL: codelang_detect-1.2.0-py3-none-any.whl
- Upload date:
- Size: 15.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/6.2.0 CPython/3.13.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
01328faa6a827495c001379f1146a0a331a215b80a4d12ad3810912c8b1a36c4
|
|
| MD5 |
7219527bb5992dd259d25d5cc09689c2
|
|
| BLAKE2b-256 |
4d69c826d9b75f5d668cedc805d2dca5911dce44a46f2369ebb487aefb7f4263
|
