Skip to main content

ADB-MLKit

Read text on Android from Python, using Google's on-device ML Kit models over ADB.

ADB-MLKit consists of a Python SDK/CLI and a small, headless Android instrumentation helper. It is an independent community project, not an official Google product. No OCR cloud account or API key is required.

Tiếng Việt · API reference · Protocol · Security

Features

  • Read a local image, encoded image bytes, an image already on Android, or a screenshot.
  • Optional XPath-to-region lookup with uiautomator2; recognized text always comes from OCR.
  • Five bundled script models: Latin (including Vietnamese), Chinese, Japanese, Korean and Devanagari.
  • Crop ROI, clockwise rotation, block/line/element text and geometry, engine-provided confidence where available.
  • Multiple devices via explicit serial selection; sequential batch images.
  • Cold/first invocation and repeated-same-image OCR timings, plus host transfer/startup/cleanup timings.
  • Request-isolated temporary storage and best-effort cleanup; Base64-safe ADB input transport.
  • Structured JSON output, typed Python results, unit tests and GitHub Actions build configuration.

Requirements

  • Python 3.10+, Android platform-tools (adb) on PATH, and an authorized USB/TCP device.
  • Android 6.0 / API 23 or newer.
  • A distribution wheel includes the version-matched Android helper APK. End users do not need Java/Gradle/Android Studio.
  • ADB/platform-tools remains a separate prerequisite; pip never installs an APK on your phone automatically.
  • Optional XPath support: install the [ui] extra.

This does not bypass Android screen-capture restrictions or private-storage permissions. Avoid enabling ADB on untrusted networks.

Quick start

Install the built wheel (available locally in dist/; publication is a separate maintainer step):

python -m pip install ./dist/adb_mlkit-0.1.0-py3-none-any.whl
adb-mlkit devices

After the maintainer publishes this project to PyPI, installation by name becomes:

python -m pip install adb-mlkit
python -m pip install "adb-mlkit[ui]"

These package-index commands require an actual release; preparing this repository does not publish one. See PyPI publishing.

Run OCR directly. Recognition commands and Python recognition methods automatically install the bundled, checksum-verified helper on the selected phone if it is missing:

adb-mlkit recognize --screenshot --language vi --runs 3 --json
adb-mlkit info

An existing helper is not automatically reinstalled or updated. Use adb-mlkit install to install/update it manually, or adb-mlkit install path/to/custom.apk for a custom APK. Installation errors stop recognition; the SDK never uninstalls an existing package to resolve a conflict.

The new helper package is io.github.adbmlkit.helper, separate from the earlier com.example.mlkitocrtest prototype. The prototype APK cannot serve this protocol.

Images on Android

adb-mlkit recognize --device-file "/sdcard/Download/example.png" --language vi --json

The Android shell must already be allowed to read the path. The host reads the encoded bytes over ADB and stages them in helper-private storage; this is not zero-copy device-only ingestion and does not require broad storage permissions.

Local images, crop and other scripts

adb-mlkit recognize --file image.png --language vi --roi 60 100 1000 700 --runs 5
adb-mlkit recognize --file japanese.png --script japanese --json
adb-mlkit recognize --file rotated.jpg --rotation 90 --json --output result.json
adb-mlkit batch first.png second.png --language vi --json --output batch.json
adb-mlkit --serial DEVICE_SERIAL recognize --screenshot --script latin

Existing output files are protected unless --overwrite is given. Global options (--serial, --adb-path, --timeout) go before the subcommand. python -m adb_mlkit is equivalent to adb-mlkit.

Python API

from adb_mlkit import ADBMLKit

ocr = ADBMLKit(serial="DEVICE_SERIAL")
result = ocr.recognize_device_file("/sdcard/Download/example.png", language="vi", runs=3)
print(result.text)
print(result.timing)       # Android decode/init/recognition durations
print(result.host_timing)  # Host load/setup/transfer/instrumentation/result/cleanup/total

for block in result.blocks:
    for line in block.lines:
        print(line.text, line.bounds, line.confidence)

See API reference and examples for all entry points.

Languages versus scripts

--language vi selects the Latin model, not a Vietnamese-only model. It does not translate, force output into Vietnamese, or provide a recognition hint. The SDK maps a documented set of aliases to the five supported scripts; adb-mlkit languages lists them. The alias list is not Google's exhaustive language list. Unsupported scripts (for example Arabic or Thai) are not automatically recognized by this integration.

ML Kit may recognize mixed-script text supported by a selected model, but this project does not run all five recognizers automatically or promise arbitrary multilingual detection. For mixed documents, select and benchmark an appropriate model or process the image separately with multiple models.

Coordinate and timing semantics

ROI is [left, top, right, bottom] in the original unrotated source image, with exclusive right/bottom edges. Crop precedes rotation. Output geometry uses the cropped, rotated image coordinate system, not necessarily screen coordinates. Explicit rotation is used; normalize EXIF orientation yourself when needed.

runs=N recognizes one image N times using one recognizer in one invocation. The first run can include lazy model initialization; later values measure warm recognition. Each API call starts instrumentation again. Warm OCR time is not end-to-end latency. The host total includes image loading/capture, helper setup, transfers and cleanup. host_timing.setup_ms measures the package check and any automatic APK installation; the first call on a device without the helper therefore takes longer. No fixed speed or accuracy guarantee is made.

Development and verification

python -m pip install -e ".[dev]"
python -m unittest discover -s tests -v
# Build Android first: see android/README.md
python scripts/prepare_package.py
python -m build
python scripts/verify_wheel.py dist/adb_mlkit-0.1.0-py3-none-any.whl

Python unit tests mock ADB; they do not establish on-device accuracy. Build the Android helper separately. See CONTRIBUTING.md for synthetic-image device tests and publication notes. No device credentials or private images are included.

Limitations

  • One instrumentation invocation at a time per device. Use one client per device; separate processes must coordinate themselves.
  • No always-running HTTP daemon, streaming camera service, translation API or cloud fallback.
  • Image input limit: 32 MiB; decoded dimensions: at most 32 million pixels. Large input can still be memory-intensive.
  • Same-device screenshot and XPath lookup are sequential, not an atomic UI snapshot. Changing UI may move the target.
  • Debug APK is a developer tool, not a hardened Play Store app. Model libraries make the all-script APK substantially larger than a Latin-only helper.
  • ML Kit models/libraries have their own terms. This repository provides integration source, not source code for Google's recognition models.

License and upstream

Project code: MIT. See third-party notices.

Metadata

Release files for adb-mlkit 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for adb-mlkit 0.1.0
File Size Uploaded Provenance Actions
adb_mlkit-0.1.0.tar.gz 24.3 MB With provenance Details for adb_mlkit-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for adb-mlkit 0.1.0
File Interpreter ABI Platform Actions
adb_mlkit-0.1.0-py3-none-any.whl Python 3 none any Details for adb_mlkit-0.1.0-py3-none-any.whl

Total release size: 48.6 MB

Release files / adb_mlkit-0.1.0.tar.gz

Download URL adb_mlkit-0.1.0.tar.gz
Size 24.3 MB
Tags Source
SHA-256 checksum
How to use checksums
e29968b26cd73ee946ad530f8bae77426b0d66d04015cffb5351cd039e3a0346
BLAKE2b-256 checksum
How to use checksums
7d0b5777186e8784164c0919b7c2556edf4fa3de843d42f2d6c74b07ac3c3e84
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.

Transparency log

Release files / adb_mlkit-0.1.0-py3-none-any.whl

Download URL adb_mlkit-0.1.0-py3-none-any.whl
Size 24.3 MB
Tags Python 3
SHA-256 checksum
How to use checksums
34b341640bf856654fb360a9aebaea6fa0bccc8734715f5e000dd34c73f36bc4
BLAKE2b-256 checksum
How to use checksums
4a8bace09760cbe9f1ebcc2d939aada9632c2fdc855e4b9d7cd22237e9b32b06
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.

Transparency log

Security for adb-mlkit 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for adb_mlkit-0.1.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Sep 11, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page