Skip to main content

sparrow-comm

Receivers for SPARROW field data. sparrow-comm accepts camera-trap images, audio and device telemetry over four channels and hands each received file to your storage code:

Channel How data arrives Entry point
Device API SPARROW field units push over HTTPS (/v1/onboarding, /v1/uploads/*, /v1/metrics/*, /v1/settings/*) make_device_router
Reconyx Reconyx cloud forwards base64 images (/v1/reconyx/login, /v1/reconyx/images) make_reconyx_router
E-mail Cellular cameras e-mail photos to a mailbox; the poller reads it over IMAP channels.imap.poll_once
FTPS Cameras upload over explicit FTP-over-TLS; the poller ingests finished files channels.ftps.make_server, channels.ftps.poll_once

The request and response formats match Sparrow Studio's receivers, so field devices and cameras configured for Studio work unchanged.

GPS tracking and device firmware delivery are not part of this release.

Install

pip install "sparrow-comm[all]"     # everything
pip install sparrow-comm            # device API, Reconyx and IMAP only
pip install "sparrow-comm[ftps]"    # adds the FTPS server (pyftpdlib, pyOpenSSL)
pip install "sparrow-comm[serve]"   # adds uvicorn for `sparrow-comm serve api`

Python 3.12 or newer.

Run it standalone

The bundled SQLite Store keeps devices, keys and credentials in one database file and writes received files under a data directory (<data>/<project>/<kind>/<date>/<sha256 prefix>__<name>).

export SPARROW_COMM_DB=./sparrow-comm.db SPARROW_COMM_DATA=./data
export SPARROW_COMM_SECRET_KEY=$(sparrow-comm secret-key)   # encrypts stored IMAP passwords
export SPARROW_COMM_JWT_SECRET=change-me                     # signs Reconyx tokens

sparrow-comm keys issue --project 1 --email you@example.org  # one-time device onboarding key
sparrow-comm reconyx issue-key --project 1                   # Reconyx id/token pair
sparrow-comm ftps set-password --project 1                   # FTPS user proj_1
sparrow-comm imap add --project 1 --address cam@example.org  # reads SPARROW_COMM_IMAP_PASSWORD

sparrow-comm serve api --port 8000
sparrow-comm serve ftps --cert cert.pem --key key.pem --port 21
sparrow-comm poll imap --every 60
sparrow-comm poll ftps --every 30

sparrow-comm items --project 1      # received items as JSON lines

Run sparrow-comm <command> --help for every option. Settings can also come from environment variables: SPARROW_COMM_DB, SPARROW_COMM_DATA, SPARROW_COMM_LOG_LEVEL, IMAP_HOST, IMAP_PORT, FTPS_STAGING_DIR, FTPS_PORT, FTPS_BIND_ADDR, FTPS_PASSIVE_PORTS, FTPS_MASQUERADE_ADDR, CERT_FILE, KEY_FILE.

The FTPS server refuses plaintext logins and, unless --no-tls-data is given, plaintext data connections. Each project logs in as proj_<id> and is confined to its own staging folder. The poller ingests files that have not changed for 5 seconds and deletes each one only after it is stored (or already present).

Use it inside your own application

Every channel talks to storage through three small interfaces in sparrow_comm.hooks. Implement them over your own database and pass your objects in; nothing else in the package touches storage.

Hook Used by Methods
Sink all channels seen(item), commit(item) -> CommitResult
DeviceRegistry device API device_by_key_hash, onboarding_key_by_hash, issuer_can_manage, consume_onboarding_key, device_settings, record_metrics
ConfigSource IMAP, FTPS, Reconyx imap_accounts, set_imap_last_uid, ftps_projects, ftps_credential, record_ftps_usage, reconyx_key, mark_reconyx_key_used, reconyx_key_authorized

Each received file reaches Sink.commit as an Item: kind (image, video, audio), channel, project id, the bytes and their SHA-256, a safe file name, and whatever the channel knows (device, camera name, capture time as naive UTC, detections, channel-specific extra). commit should be idempotent on (project_id, kind, sha256) and return CommitResult(id, created). Raising ValueError from commit turns into an HTTP 400 with your message on the device API.

from fastapi import FastAPI
from sparrow_comm.device_api import make_device_router
from sparrow_comm.reconyx import make_reconyx_router

app = FastAPI()
app.include_router(make_device_router(registry=my_registry, sink=my_sink))
app.include_router(make_reconyx_router(config=my_config, sink=my_sink, jwt_secret=SECRET))
from sparrow_comm.channels import ftps, imap

imap.poll_once(my_sink, my_config, host="imap.gmail.com")
ftps.poll_once(my_sink, my_config, "/srv/staging/ftps")

sparrow_comm.store.Store implements all three hooks and is a working reference.

Local test stack

docker/compose.yml runs the API, the FTPS server, both pollers and a GreenMail mail server. The simulator acts as a device, a Reconyx forwarder, an FTPS camera and an e-mail camera, then checks that each payload was stored exactly once.

docker compose -f docker/compose.yml up -d --build
docker compose -f docker/compose.yml --profile sim run --rm sim
docker compose -f docker/compose.yml down -v

The secrets in that file are fixed development values.

Comparing with Sparrow Studio

With a Sparrow Studio checkout, docker/parity.compose.yml runs Studio's device and Reconyx server next to sparrow-comm. docker/parity.py sends both the same 48 requests and compares status codes and response bodies. Values generated per server, such as ids, tokens and timestamps, are masked. The script exits 1 on any unexpected difference. It lists two inputs on which Studio returns HTTP 500 as known Studio defects.

export STUDIO_DIR=/path/to/sparrow-studio
docker compose -f docker/parity.compose.yml up -d --build --wait
docker compose -f docker/parity.compose.yml --profile parity run --rm --build parity
docker compose -f docker/parity.compose.yml down -v

Development

uv sync
uv run pytest
uv run ruff check . && uv run ruff format --check .

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit Contributor License Agreements.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

Metadata

Release files for sparrow-comm 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sparrow-comm 0.1.0
File Size Uploaded Provenance Actions
sparrow_comm-0.1.0.tar.gz 22.7 kB Without provenance Details for sparrow_comm-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sparrow-comm 0.1.0
File Interpreter ABI Platform Actions
sparrow_comm-0.1.0-py3-none-any.whl Python 3 none any Details for sparrow_comm-0.1.0-py3-none-any.whl

Total release size: 52.1 kB

Release files / sparrow_comm-0.1.0.tar.gz

Download URL sparrow_comm-0.1.0.tar.gz
Size 22.7 kB
Tags Source
SHA-256 checksum
How to use checksums
9249659b5f4099f9e209a3557e772d6ea454ebd45fc8c59eafd549dc034361b8
BLAKE2b-256 checksum
How to use checksums
fd14b31a7572cf42c9fb98924ee0e8e4710b7df4c1697db769589c33de346d2d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.2

Release files / sparrow_comm-0.1.0-py3-none-any.whl

Download URL sparrow_comm-0.1.0-py3-none-any.whl
Size 29.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
582dcd5b05b4fb3cd877b96a62316990e8375c0ca4928f0db0e37367c4cf441f
BLAKE2b-256 checksum
How to use checksums
dab28d00c2fa3613c7cfc2ce229f634a115ed1f1794723b97d293e4f8e150b44
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.2

Security for sparrow-comm 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page