Skip to main content

Numerically verified simulation of transformer attention via optical wave interference in holographic crystals

Project description

ATOM

Angular-Multiplexed Transformer Optical Model

What if a neural network's weights didn't exist as floating points in memory but in holograms ?

This repo is a numerically verified simulation showing that AI weights (which are floating points), encoded as phase structure inside a holographic crystal, can perform transformer attention exactly through wave interference. Not approximately. ALgebraically equal to. Verified in code you can run in one command.

The math is proved. The hardware doesn't exist yet. That's the point of open sourcing this.


What this actually is

A standard transformer spends most of its energy moving weights from memory to compute and back. Every forward pass, every token, billions of numbers shuffling across a bus.

This project asks: what if the weights were the computer?

A photorefractive crystal stores holograms through its entire volume using Bragg angle selectivity, write a pattern at a specific angle, and only light arriving at that exact angle reads it back. Change the angle by a fraction of a degree and you're reading a completely different stored pattern. This is angular multiplexing: hundreds of independent weight matrices living in the same cubic centimetre of crystal, each addressed by angle.

The ATOM simulator models this in PyTorch. It proves that when query and key vectors are encoded as optical wave amplitudes and interfere inside such a crystal, the output is algebraically identical to scaled dot-product attention. The computation happens through physics — diffraction and interference — not digital arithmetic.


The capacity numbers (and what they actually mean)

A 1 cm³ crystal at reference parameters holds:

1,000 depth layers × 900 angular channels × 100M pixels/layer = 90 trillion addressable weight values

This is a geometric ceiling — the storage capacity of the medium before any data is written, like saying a hard drive holds 2 TB before downloading anything. Here's the honest range of what's usable:

Scenario Capacity Notes
Geometric ceiling 90T Pure math, no physical losses
50% error correction allocation 45T Half reserved for redundancy
Realistic (SNR + crosstalk degradation) 4.5T – 9T 5–10% of ceiling under real multiplexing conditions
NVIDIA H100 (for reference) 20–35B Measured hardware

The 5–10% realistic figure is where the engineering gets hard. It's also where most of the interesting open problems live.


What's proved vs what's projected

Claim Status
Optical interference = scaled dot-product attention ✓ Proved and verified to float precision
Angular Spectrum Method conserves energy ✓ < 2.3×10⁻⁷ relative error
Phase masks preserve intensity ✓ Verified
Field propagation is reversible ✓ Verified
Gradients flow through the full optical path ✓ Verified
90T geometric capacity per cm³ Geometric derivation — not experimentally validated
50–200 ps latency Physics derivation — not measured
~99% energy reduction Architecture projection — not measured
Inference accuracy on real tasks Not claimed. There is no noise model.

The last row is intentional. Accuracy claims require a noise model, a task benchmark, and hardware. None of those exist here yet.


Install and run

git clone https://github.com/mprahboamey/atom.git
cd atom
pip install -r requirements.txt

Run everything and see all validation results:

python scripts/run_all.py

Or run individual examples in order:

python examples/01_propagate_beam.py      # Gaussian beam through free space
python examples/02_train_phase_mask.py    # train a phase mask to focus light
python examples/03_optical_attention.py   # optical vs digital attention — exact match
python examples/04_validate_model.py      # all numerical checks with tolerances

Project layout

atom/                        ← core library
├── propagation.py           ← Angular Spectrum Method, Gaussian beam helpers
├── diffractive.py           ← trainable phase masks, stacked diffractive network
└── attention.py             ← interference-based attention scores

examples/                    ← one concept per script, run in order
docs/
├── model.md                 ← full mathematical derivation
└── benchmarks.md            ← every projection with assumptions stated
scripts/run_all.py           ← runs everything
tests/                       ← unit tests
results/                     ← validation output (generated at runtime)
figures/                     ← plots from simulation

Where this needs to go next

The math is done. The simulator works. What doesn't exist yet is everything physical — and that's what this repo is for. See CONTRIBUTING.md for specific open problems by domain.

The short version: building a real device requires solving noise, materials, readout, and system integration problems that this codebase deliberately does not claim to have solved. If you work in any of those areas, there is a concrete problem in CONTRIBUTING.md with your name on it.


References

  • Goodman, J. W. (2005). Introduction to Fourier Optics
  • Psaltis, D., Brady, D., & Wagner, K. (1988). Adaptive optical networks using photorefractive crystals. Applied Optics, 27(9), 1752–1759.
  • Psaltis, D., & Mok, F. (1995). Holographic memories. Scientific American, 273(5), 70–76.
  • Lin, X., et al. (2018). All-optical machine learning using diffractive deep neural networks. Science, 361(6406), 1004–1008.
  • Miller, D. A. B. (2017). Attojoule optoelectronics. Journal of Lightwave Technology, 35(3), 346–396.
  • Vaswani, A., et al. (2017). Attention is all you need. NeurIPS, 30.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

atom_optic-0.1.0.tar.gz (8.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

atom_optic-0.1.0-py3-none-any.whl (8.1 kB view details)

Uploaded Python 3

File details

Details for the file atom_optic-0.1.0.tar.gz.

File metadata

  • Download URL: atom_optic-0.1.0.tar.gz
  • Upload date:
  • Size: 8.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.6

File hashes

Hashes for atom_optic-0.1.0.tar.gz
Algorithm Hash digest
SHA256 9a3c6c15d36e3ec7d4de34fcf6f6532facbfce9d7bdd090ca97373fae00afbce
MD5 56ca448558cc35b1b0cdd3674556ff56
BLAKE2b-256 5b76e6e7f021ae2ee45fad3477bbd4662a1ad73a6534258603295477691027df

See more details on using hashes here.

File details

Details for the file atom_optic-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: atom_optic-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 8.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.6

File hashes

Hashes for atom_optic-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 2f98aca407b34c1b8593d8d827373db05db6981f78098099231650dd49cff37b
MD5 9f803fc402c6ca688c69fb06fba8cac1
BLAKE2b-256 ed83b941251e40748f94dbf2c1877d41aeecf3ab6fe858a36e1a76c3206be839

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page