Skip to main content

ADB-MLKit

Read text on Android from Python, using Google's on-device ML Kit models over ADB.

ADB-MLKit consists of a Python SDK/CLI and a small, headless Android instrumentation helper. It is an independent community project, not an official Google product. No OCR cloud account or API key is required.

Tiếng Việt · API reference · Protocol · Security

Features

  • Read a local image, encoded image bytes, an image already on Android, or a screenshot.
  • XPath-to-region lookup with uiautomator2; recognized text always comes from OCR.
  • Five bundled script models: Latin (including Vietnamese), Chinese, Japanese, Korean and Devanagari.
  • Crop ROI, clockwise rotation, block/line/element text and geometry, engine-provided confidence where available.
  • Multiple devices via explicit serial selection; sequential batch images.
  • Cold/first invocation and repeated-same-image OCR timings, plus host transfer/startup/cleanup timings.
  • Request-isolated temporary storage and best-effort cleanup; Base64-safe ADB input transport.
  • Structured JSON output, typed Python results, unit tests and GitHub Actions build configuration.

Requirements

  • Python 3.10+, Android platform-tools (adb) on PATH, and an authorized USB/TCP device.
  • Android 6.0 / API 23 or newer.

This does not bypass Android screen-capture restrictions or private-storage permissions. Avoid enabling ADB on untrusted networks.

Quick start

1. Install

python -m pip install adb-mlkit

The package includes the version-matched Android helper APK, all five OCR script models, and XPath support (uiautomator2 and Pillow). End users do not need Java, Gradle, or Android Studio. ADB/platform-tools must be installed separately.

2. Connect device

Ensure your Android device is connected via USB or TCP with USB debugging enabled and authorized:

adb devices

You should see your device listed with state device.

3. Read text

Recognition commands and the Python API automatically install the helper APK on the selected device if it is missing. No separate installation step is needed:

adb-mlkit recognize --screenshot --language vi --json

An existing helper is not automatically reinstalled or updated. Use adb-mlkit install to install or update it manually, or adb-mlkit install path/to/custom.apk for a custom APK. Installation errors stop recognition; the SDK never uninstalls an existing package to resolve a conflict.

Usage

Read text from Android

adb-mlkit recognize --device-file "/sdcard/Download/example.png" --language vi --json

The Android shell must already be allowed to read the path. The host reads the encoded bytes over ADB and stages them in helper-private storage; this is not zero-copy device-only ingestion and does not require broad storage permissions.

Read local images, crop and other scripts

adb-mlkit recognize --file image.png --language vi --roi 60 100 1000 700 --runs 5
adb-mlkit recognize --file japanese.png --script japanese --json
adb-mlkit recognize --file rotated.jpg --rotation 90 --json --output result.json
adb-mlkit batch first.png second.png --language vi --json --output batch.json
adb-mlkit --serial DEVICE_SERIAL recognize --screenshot --script latin

Existing output files are protected unless --overwrite is given. Global options (--serial, --adb-path, --timeout) go before the subcommand. python -m adb_mlkit is equivalent to adb-mlkit.

Python API

from adb_mlkit import ADBMLKit

ocr = ADBMLKit(serial="DEVICE_SERIAL")
result = ocr.recognize_device_file("/sdcard/Download/example.png", language="vi", runs=3)
print(result.text)
print(result.timing)       # Android decode/init/recognition durations
print(result.host_timing)  # Host load/setup/transfer/instrumentation/result/cleanup/total

for block in result.blocks:
    for line in block.lines:
        print(line.text, line.bounds, line.confidence)

See API reference and examples for all entry points.

Device management

adb-mlkit devices           # List attached devices
adb-mlkit info              # Device model, API level, helper status
adb-mlkit install           # Install or update the bundled helper
adb-mlkit languages         # List supported scripts and language aliases

Languages versus scripts

--language vi selects the Latin model, not a Vietnamese-only model. It does not translate, force output into Vietnamese, or provide a recognition hint. The SDK maps a documented set of aliases to the five supported scripts; adb-mlkit languages lists them. The alias list is not Google's exhaustive language list. Unsupported scripts (for example Arabic or Thai) are not automatically recognized by this integration.

ML Kit may recognize mixed-script text supported by a selected model, but this project does not run all five recognizers automatically or promise arbitrary multilingual detection. For mixed documents, select and benchmark an appropriate model or process the image separately with multiple models.

Coordinate and timing semantics

ROI is [left, top, right, bottom] in the original unrotated source image, with exclusive right/bottom edges. Crop precedes rotation. Output geometry uses the cropped, rotated image coordinate system, not necessarily screen coordinates. Explicit rotation is used; normalize EXIF orientation yourself when needed.

runs=N recognizes one image N times using one recognizer in one invocation. The first run can include lazy model initialization; later values measure warm recognition. Each API call starts instrumentation again. Warm OCR time is not end-to-end latency. The host total includes image loading/capture, helper setup, transfers and cleanup. host_timing.setup_ms measures the package check and any automatic APK installation; the first call on a device without the helper therefore takes longer. No fixed speed or accuracy guarantee is made.

Development

From source

Clone the repository and build from source. This requires JDK 25, Android SDK (API 37), and the toolchain described in android/README.md.

git clone https://github.com/zidonghua-source/ADB-MLKit.git
cd ADB-MLKit

python -m pip install -e ".[dev]"
python -m unittest discover -s tests -v

# Build Android helper first
cd android && ./gradlew :app:assembleDebug && cd ..
python scripts/prepare_package.py

# Build and verify wheel
python -m build
python scripts/verify_wheel.py dist/adb_mlkit-0.1.0-py3-none-any.whl

A bare source checkout installed with pip install . without the staging step installs Python only; recognition will report a missing bundled APK. You can supply a custom APK explicitly or install from PyPI instead.

CI checks

Python unit tests mock ADB; they do not establish on-device accuracy. Build the Android helper separately. See CONTRIBUTING.md for synthetic-image device tests and publication notes. No device credentials or private images are included.

Limitations

  • One instrumentation invocation at a time per device. Use one client per device; separate processes must coordinate themselves.
  • No always-running HTTP daemon, streaming camera service, translation API or cloud fallback.
  • Image input limit: 32 MiB; decoded dimensions: at most 32 million pixels. Large input can still be memory-intensive.
  • Same-device screenshot and XPath lookup are sequential, not an atomic UI snapshot. Changing UI may move the target.
  • Debug APK is a developer tool, not a hardened Play Store app. Model libraries make the all-script APK substantially larger than a Latin-only helper.
  • ML Kit models/libraries have their own terms. This repository provides integration source, not source code for Google's recognition models.

License and upstream

Project code: MIT. See third-party notices.

Metadata

Release files for adb-mlkit 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for adb-mlkit 0.1.2
File Size Uploaded Provenance Actions
adb_mlkit-0.1.2.tar.gz 24.3 MB With provenance Details for adb_mlkit-0.1.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for adb-mlkit 0.1.2
File Interpreter ABI Platform Actions
adb_mlkit-0.1.2-py3-none-any.whl Python 3 none any Details for adb_mlkit-0.1.2-py3-none-any.whl

Total release size: 48.6 MB

Release files / adb_mlkit-0.1.2.tar.gz

Download URL adb_mlkit-0.1.2.tar.gz
Size 24.3 MB
Tags Source
SHA-256 checksum
How to use checksums
11e0312932de02be6c29cf73f4ccd495f4f1a42ca323ebe15e0d39eaac661fa0
BLAKE2b-256 checksum
How to use checksums
e9eca3cf28996513eae5d94bb7f1e2cab3716bdad8fdbca9b5e8d557326ef4f9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.

Transparency log

Release files / adb_mlkit-0.1.2-py3-none-any.whl

Download URL adb_mlkit-0.1.2-py3-none-any.whl
Size 24.3 MB
Tags Python 3
SHA-256 checksum
How to use checksums
26139276d3f7d23253efa9ec13c2ddd7a257b488916a80b28bc872d64d2336b5
BLAKE2b-256 checksum
How to use checksums
16527876ec11b44e04506c146c1ed8a883c312b103c637c9a8c7a34ef95f2985
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.

Transparency log

Security for adb-mlkit 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for adb_mlkit-0.1.2.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.2 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page