onsense 0.3.1

Turn your Android phone into a camera and sensor source for PC AI agents (onSense MCP broker)

onsense — Phone Camera & Sensors for PC AI Agents

onsense is the PC-side MCP broker for onSense. Install the Android app on your phone, run uvx onsense pair on your PC, and your AI client (Claude, Cursor, or any MCP-compatible tool) can see through your phone's camera and read its sensors — all over local Wi-Fi, with no cloud relay.

---

Architecture

Android phone (onSense app)           PC (this package)           AI client
  HTTP provider :8080        ←→    stdio MCP broker          Claude / Cursor / …
  camera frames, photos,           onsense serve               natural-language or
  sensors, QR pairing              onsense clip :8770          /eye tool calls

• The phone runs an HTTP provider on port 8080. It exposes camera frames, recent photos, and sensor readings, all gated by a pairing token.
• The PC package (onsense) is a stdio MCP broker. It translates MCP tool calls from your AI client into HTTP requests to the phone.
• Discovery uses mDNS (_onsense._tcp.local.). If the phone's IP changes, the broker rediscovers it automatically — no manual reconfiguration needed.
• An optional clip bridge daemon on port 8770 lets the phone push camera frames, files, and clipboard content to the PC (and, if enabled, lets the phone pull from the PC clipboard).

---

Requirements

• Python 3.10+ and uv (provides uvx)
• The onSense Android app installed on a phone on the same local Wi-Fi network as your PC
• An MCP-compatible AI client such as Claude Code or Cursor

---

Quick Start

1. Pair your phone

Run this once. The PC prints a QR code; scan it from the onSense app ("Scan PC QR"). The phone pushes its address and token to the PC, which registers the MCP server automatically.

uvx onsense pair

After pairing, restart your AI client once so it picks up the new onsense MCP server.

2. Use it

Ask your AI client naturally:

"Take a photo of what's in front of my phone." "What are the current sensor readings?" "Show me the last 5 photos on my phone."

The MCP tools are called automatically. You can also invoke them directly as /eye or whichever slash command your client supports.

3. Diagnose problems

uvx onsense doctor           # checks Python, uv, MCP, Claude registration, mDNS, phone reachability
uvx onsense doctor --base http://192.168.1.5:8080 --token YOUR_PAIRING_TOKEN

---

Subcommands

Command	What it does
uvx onsense pair	Display a QR code on the PC; phone scans it and pushes {base, token}; registers MCP automatically
uvx onsense pair "onsense://pair?base=...&token=..."	Parse a pairing URI directly (phone-displayed QR → manual copy)
uvx onsense pair --img screenshot.png	Decode a pairing QR from a screenshot file (requires opencv)
uvx onsense serve	Run the MCP server (stdio). Also starts the clip daemon automatically unless --no-clip is passed
uvx onsense serve --no-clip	Run the MCP server only, without starting the clip daemon
uvx onsense clip	Run the clip bridge daemon standalone (port 8770)
uvx onsense clip --allow-pull	Enable phone→pull: phone can GET /clip to retrieve the PC clipboard
uvx onsense clip --set-clipboard	Auto-inject received content into the PC OS clipboard (off by default)
uvx onsense doctor	Diagnose installation, connectivity, and phone reachability

---

Other MCP clients (Claude Desktop, Cursor, Codex)

uvx onsense pair auto-registers the server with Claude Code (via claude mcp add). The MCP server itself is standard stdio MCP, so any MCP-compatible client works once configured — only the auto-registration is Claude Code-specific. For other clients, run uvx onsense pair once to obtain your phone's PHONE_BASE / PHONE_TOKEN (also saved to ~/.onsense/pair.json), then add the server manually:

Claude Desktop / Cursor — add to the client's MCP config (claude_desktop_config.json or .cursor/mcp.json):

{
  "mcpServers": {
    "onsense": {
      "command": "uvx",
      "args": ["onsense", "serve"],
      "env": { "PHONE_BASE": "http://<phone-ip>:8080", "PHONE_TOKEN": "<token>" }
    }
  }
}

Codex — add to ~/.codex/config.toml:

[mcp_servers.onsense]
command = "uvx"
args = ["onsense", "serve"]
env = { PHONE_BASE = "http://<phone-ip>:8080", PHONE_TOKEN = "<token>" }

Restart the client once after adding the server.

---

MCP Tools

These tools are exposed to your AI client after pairing:

Tool	Description
get_live_frame()	Capture the current camera frame from the phone (returns JPEG image)
read_sensors()	Return phone sensor readings as JSON: battery level/charging state, ambient light (lux), accelerometer (x/y/z)
recent_photos(limit=10)	List recent photos on the phone: id, name, date_added, size, width, height
get_photo(id, max_width=1024)	Fetch a specific photo by id (downscaled to max_width); use ids from recent_photos

If the phone's IP changes, the broker retries using mDNS autodiscovery before surfacing an error.

---

Clip Bridge (Phone ↔ PC File & Clipboard)

onsense serve automatically starts a clip daemon on port 8770. You can also run it standalone with onsense clip.

Phone → PC push (POST /clip)

The onSense Android app can push content to the PC:

• Images are saved to disk as latest.jpg (in ONSENSE_CLIP_DIR, default <tempdir>/onsense/).
• Text files are saved to disk.
• Other files (video, PDF, etc.) are saved to disk by filename.
• If --set-clipboard is active, images and text are also injected into the PC OS clipboard so you can paste with Ctrl+V immediately.

By default, --set-clipboard is off — files are saved to disk but the clipboard is not touched.

PC → Phone pull (GET /clip)

Off by default. Enable with:

uvx onsense clip --allow-pull

When enabled, the phone can GET /clip to retrieve the current PC clipboard content (copied files first, then images, then text). Returns 204 if the clipboard is empty.

Ports and environment variables

Variable	Default	Purpose
PHONE_BASE	(from pairing)	Phone HTTP base URL, e.g. http://192.168.1.5:8080
PHONE_TOKEN	(from pairing)	Pairing auth token
ONSENSE_CLIP_DIR	<tempdir>/onsense	Directory where pushed files are saved
ONSENSE_CLIP_MAX_MB	200	Maximum incoming file size in MB (0 = unlimited)
ONSENSE_CLIP_ALLOW_PULL	0	Set to 1 to enable PC→phone pull without the CLI flag
ONSENSE_CLIP_SET_CLIPBOARD	0	Set to 1 to auto-inject into OS clipboard without the CLI flag
ONSENSE_TEST_FRAME	(unset)	Local JPEG path returned by get_live_frame when phone is unreachable

---

Security

Local network only. The PC-side services bind to all interfaces but reject connections from non-private IP addresses at the application layer. They are not intended to be exposed to the internet.

HMAC request signing. The pairing token is never sent in cleartext. Every authenticated request is signed with HMAC-SHA256 over METHOD\npath\ntimestamp\nnonce\nalgorithm, keyed by a signing key derived from the pairing token via HKDF-SHA256 (a key separate from the encryption key), and carries X-Ts / X-Nonce / X-Sig / X-Enc headers. Servers verify in constant time, reject timestamps outside a ±300 s window, and reject reused nonces — so a sniffed request cannot be replayed and the token cannot be stolen off the wire. New Android installs generate a 128-bit random token, stored on the phone and in ~/.onsense/pair.json (chmod 600) after pairing. (The /health endpoint is unauthenticated.) See PROTOCOL.md for the exact wire format.

Pull and clipboard injection are off by default. GET /clip (phone pulls PC clipboard) and OS clipboard auto-injection (phone push → Ctrl+V) are disabled unless you explicitly pass --allow-pull or --set-clipboard.

Encrypted bodies (AES-256-GCM). Sensitive payloads — camera frames, photos, sensor data, files, and clipboard content — are encrypted end-to-end with AES-256-GCM. The key is derived from the pairing token via HKDF-SHA256 (a key separate from the signing key); each message uses a fresh 96-bit nonce, and the GCM tag authenticates the body and binds it to its request (so tampering or response substitution is rejected). A passive sniffer on the same Wi-Fi sees only ciphertext. Low-sensitivity metadata stays plaintext: the open /version and /health endpoints, HTTP error responses, and the saved-file path returned after a push. (Note: large transfers currently buffer the body in memory while encrypting/decrypting — chunked streaming for very large files is a future step. Transport-level TLS is also a possible future hardening.)

File size cap. Incoming pushes are rejected if they exceed ONSENSE_CLIP_MAX_MB (default 200 MB). Set to 0 to remove the cap.

---

License

MIT. See LICENSE.

---

한국어 요약

onsense는 Android 폰(onSense 앱)을 PC AI 에이전트의 카메라·센서로 연결하는 PC측 MCP 브로커입니다.

• uvx onsense pair — 폰과 한 번 페어링하면 Claude·Cursor에 MCP 서버가 자동 등록됩니다.
• uvx onsense doctor — 연결·설치 문제를 진단합니다.
• uvx onsense serve — AI 클라이언트가 호출하는 MCP 서버(stdio)를 실행합니다. 동시에 클립 브리지 데몬(:8770)도 자동 기동합니다.
• uvx onsense clip — 폰↔PC 파일·클립보드 브리지를 단독으로 실행합니다.

폰과 PC는 같은 로컬 Wi-Fi에 있어야 하며, 개발자 운영 클라우드 서버는 없습니다. 통신은 현재 로컬 HTTP(비암호화)이므로 신뢰할 수 있는 네트워크에서 사용하세요.