ADB-MLKit
Read text on Android from Python, using Google's on-device ML Kit models over ADB.
ADB-MLKit consists of a Python SDK/CLI and a small, headless Android instrumentation helper. It is an independent community project, not an official Google product. No OCR cloud account or API key is required.
Tiếng Việt · API reference · Protocol · Security
Features
- Read a local image, encoded image bytes, an image already on Android, or a screenshot.
- XPath-to-region lookup with uiautomator2; recognized text always comes from OCR.
- Five bundled script models: Latin (including Vietnamese), Chinese, Japanese, Korean and Devanagari.
- Crop ROI, clockwise rotation, block/line/element text and geometry, engine-provided confidence where available.
- Multiple devices via explicit serial selection; sequential batch images.
- Cold/first invocation and repeated-same-image OCR timings, plus host transfer/startup/cleanup timings.
- Request-isolated temporary storage and best-effort cleanup; Base64-safe ADB input transport.
- Structured JSON output, typed Python results, unit tests and GitHub Actions build configuration.
Requirements
- Python 3.10+, Android platform-tools (
adb) on PATH, and an authorized USB/TCP device. - Android 6.0 / API 23 or newer.
This does not bypass Android screen-capture restrictions or private-storage permissions. Avoid enabling ADB on untrusted networks.
Quick start
1. Install
python -m pip install adb-mlkit
The package includes the version-matched Android helper APK, all five OCR script models, and XPath support (uiautomator2 and Pillow). End users do not need Java, Gradle, or Android Studio. ADB/platform-tools must be installed separately.
2. Connect device
Ensure your Android device is connected via USB or TCP with USB debugging enabled and authorized:
adb devices
You should see your device listed with state device.
3. Read text
Recognition commands and the Python API automatically install the helper APK on the selected device if it is missing. No separate installation step is needed:
adb-mlkit recognize --screenshot --language vi --json
An existing helper is not automatically reinstalled or updated. Use adb-mlkit install to install or update it manually, or adb-mlkit install path/to/custom.apk for a custom APK. Installation errors stop recognition; the SDK never uninstalls an existing package to resolve a conflict.
Usage
Read text from Android
adb-mlkit recognize --device-file "/sdcard/Download/example.png" --language vi --json
The Android shell must already be allowed to read the path. The host reads the encoded bytes over ADB and stages them in helper-private storage; this is not zero-copy device-only ingestion and does not require broad storage permissions.
Read local images, crop and other scripts
adb-mlkit recognize --file image.png --language vi --roi 60 100 1000 700 --runs 5
adb-mlkit recognize --file japanese.png --script japanese --json
adb-mlkit recognize --file rotated.jpg --rotation 90 --json --output result.json
adb-mlkit batch first.png second.png --language vi --json --output batch.json
adb-mlkit --serial DEVICE_SERIAL recognize --screenshot --script latin
Existing output files are protected unless --overwrite is given. Global options (--serial, --adb-path, --timeout) go before the subcommand. python -m adb_mlkit is equivalent to adb-mlkit.
Python API
from adb_mlkit import ADBMLKit
ocr = ADBMLKit(serial="DEVICE_SERIAL")
result = ocr.recognize_device_file("/sdcard/Download/example.png", language="vi", runs=3)
print(result.text)
print(result.timing) # Android decode/init/recognition durations
print(result.host_timing) # Host load/setup/transfer/instrumentation/result/cleanup/total
for block in result.blocks:
for line in block.lines:
print(line.text, line.bounds, line.confidence)
See API reference and examples for all entry points.
Device management
adb-mlkit devices # List attached devices
adb-mlkit info # Device model, API level, helper status
adb-mlkit install # Install or update the bundled helper
adb-mlkit languages # List supported scripts and language aliases
Languages versus scripts
--language vi selects the Latin model, not a Vietnamese-only model. It does not translate, force output into Vietnamese, or provide a recognition hint. The SDK maps a documented set of aliases to the five supported scripts; adb-mlkit languages lists them. The alias list is not Google's exhaustive language list. Unsupported scripts (for example Arabic or Thai) are not automatically recognized by this integration.
ML Kit may recognize mixed-script text supported by a selected model, but this project does not run all five recognizers automatically or promise arbitrary multilingual detection. For mixed documents, select and benchmark an appropriate model or process the image separately with multiple models.
Coordinate and timing semantics
ROI is [left, top, right, bottom] in the original unrotated source image, with exclusive right/bottom edges. Crop precedes rotation. Output geometry uses the cropped, rotated image coordinate system, not necessarily screen coordinates. Explicit rotation is used; normalize EXIF orientation yourself when needed.
runs=N recognizes one image N times using one recognizer in one invocation. The first run can include lazy model initialization; later values measure warm recognition. Each API call starts instrumentation again. Warm OCR time is not end-to-end latency. The host total includes image loading/capture, helper setup, transfers and cleanup. host_timing.setup_ms measures the package check and any automatic APK installation; the first call on a device without the helper therefore takes longer. No fixed speed or accuracy guarantee is made.
Development
From source
Clone the repository and build from source. This requires JDK 25, Android SDK (API 37), and the toolchain described in android/README.md.
git clone https://github.com/zidonghua-source/ADB-MLKit.git
cd ADB-MLKit
python -m pip install -e ".[dev]"
python -m unittest discover -s tests -v
# Build Android helper first
cd android && ./gradlew :app:assembleDebug && cd ..
python scripts/prepare_package.py
# Build and verify wheel
python -m build
python scripts/verify_wheel.py dist/adb_mlkit-0.1.0-py3-none-any.whl
A bare source checkout installed with pip install . without the staging step installs Python only; recognition will report a missing bundled APK. You can supply a custom APK explicitly or install from PyPI instead.
CI checks
Python unit tests mock ADB; they do not establish on-device accuracy. Build the Android helper separately. See CONTRIBUTING.md for synthetic-image device tests and publication notes. No device credentials or private images are included.
Limitations
- One instrumentation invocation at a time per device. Use one client per device; separate processes must coordinate themselves.
- No always-running HTTP daemon, streaming camera service, translation API or cloud fallback.
- Image input limit: 32 MiB; decoded dimensions: at most 32 million pixels. Large input can still be memory-intensive.
- Same-device screenshot and XPath lookup are sequential, not an atomic UI snapshot. Changing UI may move the target.
- Debug APK is a developer tool, not a hardened Play Store app. Model libraries make the all-script APK substantially larger than a Latin-only helper.
- ML Kit models/libraries have their own terms. This repository provides integration source, not source code for Google's recognition models.
License and upstream
Project code: MIT. See third-party notices.
Metadata
Release files for adb-mlkit 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| adb_mlkit-0.1.2.tar.gz | 24.3 MB | With provenance | Details for adb_mlkit-0.1.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| adb_mlkit-0.1.2-py3-none-any.whl | Python 3 | none | any | Details for adb_mlkit-0.1.2-py3-none-any.whl |
Total release size: 48.6 MB
Release files / adb_mlkit-0.1.2.tar.gz
| Download URL | adb_mlkit-0.1.2.tar.gz |
|---|---|
| Size | 24.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
11e0312932de02be6c29cf73f4ccd495f4f1a42ca323ebe15e0d39eaac661fa0
|
|
BLAKE2b-256 checksum How to use checksums |
e9eca3cf28996513eae5d94bb7f1e2cab3716bdad8fdbca9b5e8d557326ef4f9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logRelease files / adb_mlkit-0.1.2-py3-none-any.whl
| Download URL | adb_mlkit-0.1.2-py3-none-any.whl |
|---|---|
| Size | 24.3 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
26139276d3f7d23253efa9ec13c2ddd7a257b488916a80b28bc872d64d2336b5
|
|
BLAKE2b-256 checksum How to use checksums |
16527876ec11b44e04506c146c1ed8a883c312b103c637c9a8c7a34ef95f2985
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logSecurity for adb-mlkit 0.1.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for adb_mlkit-0.1.2.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.