PhotonRT
PhotonRT is a CPU-first native inference runtime for the Photon image-captioning model and its MobileCLIP-S1 image encoder.
Current pipeline
image (preprocessed CHW float32)
|
v
MobileCLIP-S1 native CPU encoder
|
512-D
|
v
Photon projector
|
8 x 256
|
v
Photon decoder (6 layers)
|
v
caption
The Photon decoder is implemented in C++ with a KV-cache generation path. MobileCLIP-S1 is implemented as a reparameterized FastViT/MCI-1 CPU graph. The upstream MobileCLIP repository defines S1 as a 256-pixel image model with a 512-D embedding and MCI-1 image encoder; see the official Apple implementation for model details.
Build (Windows + Visual Studio generator)
cmake -S . -B build
cmake --build build --config Release --parallel
ctest --test-dir build -C Release --output-on-failure
Convert Photon checkpoint
python tools/convert_photon.py \
--checkpoint "C:/path/to/nano_ep55.pt" \
--tokenizer "C:/path/to/mobilecap_tokenizer.json" \
--model-out photon.f32.photon \
--tokenizer-out photon.tokenizer
Convert MobileCLIP-S1
Use Apple's official MobileCLIP Python package for conversion:
python tools/convert_mobileclip.py \
--checkpoint "C:/path/to/mobileclip_s1.pt" \
--output mobileclip-s1.f32.mclip
The converter writes reparameterized inference weights and does not bundle the original Apple checkpoint.
Prepare an exact-reference MobileCLIP input
python tools/preprocess_mobileclip.py \
--image "C:/path/to/image.jpg" \
--output image_chw.f32
Run native MobileCLIP
./build/Release/mobileclip-embedding.exe \
mobileclip-s1.f32.mclip \
image_chw.f32
Compare native MobileCLIP against PyTorch
python tools/compare_mobileclip.py \
--checkpoint "C:/path/to/mobileclip_s1.pt" \
--image-tensor image_chw.f32 \
--native-embedding mobileclip_embedding.f32
Run end-to-end C++ image captioning
./build/Release/image-caption.exe \
mobileclip-s1.f32.mclip \
photon.f32.photon \
photon.tokenizer \
image_chw.f32
Important release note
PhotonRT can distribute its own runtime code under its chosen open-source license. Apple's MobileCLIP source code is MIT, while the pretrained MobileCLIP model weights are governed by Apple's model terms; do not bundle Apple's weights into a release without checking those terms.
Status
- Photon C++ CPU reference runtime: working
- real Photon checkpoint conversion: working
- Photon generation: working
- native MobileCLIP-S1 CPU graph: implemented
- PyTorch/native MobileCLIP parity: next validation target
- optimized SIMD kernels: next
- quantization: next
- CUDA kernels: next
- JPEG/PNG native image decoding: next
- Python bindings: next
Metadata
Release files for photonrt 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| photonrt-0.2.1.tar.gz | 6.4 MB | Without provenance | Details for photonrt-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| photonrt-0.2.1-cp310-cp310-win_amd64.whl | CPython 3.10 | CPython 3.10 | Windows x86-64 | Details for photonrt-0.2.1-cp310-cp310-win_amd64.whl |
Total release size: 12.6 MB
Release files / photonrt-0.2.1.tar.gz
| Download URL | photonrt-0.2.1.tar.gz |
|---|---|
| Size | 6.4 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
49f694d3b3e19fd9343ad34e0df8fdc42f173e5e5c7a1a7dda0f2b1752f545f7
|
|
BLAKE2b-256 checksum How to use checksums |
7c1b6f249d568b4afaf57b208344faf0a56b378aca5ada87a988441a3285daf3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.10.20
|
Release files / photonrt-0.2.1-cp310-cp310-win_amd64.whl
| Download URL | photonrt-0.2.1-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 6.2 MB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
18a450f652e4c2eec045af4e69e0f2f26648fc11d21a7652032aec0c26aa98a4
|
|
BLAKE2b-256 checksum How to use checksums |
56730776e76a32fc55db9c8e4bb58642cb9cbcd349d788dc1d92c98a04c6b30a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.10.20
|
Security for photonrt 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.