digital-history — Space Memory System

Memory is reconstruction, not retrieval.

149,515 messages across WhatsApp, Telegram, Claude, ChatGPT, Gmail — plus GitHub commits, articles, legal records. All searchable. All reconstructable. Fully offline once the server is up.

---

Prerequisites

Two things must be running:

# 1. Ollama (local LLM — run once, stays up)
ollama serve

# 2. Query server (D1 semantic search — run once per session)
dh server
# or: cd ~/Documents/digital-history && python3.13 -m uvicorn query_server:app --host 127.0.0.1 --port 8000

The query server loads ~250K vectors from D1 on startup (~60 sec first time), then stays in memory. HF_TOKEN is loaded automatically from ~/.secret_tokens.

---

Quick commands — use dh

dh reconstruct "what was happening with amrita late 2024"   # full pipeline: CARM + tone + Ollama
dh reconstruct -i                                           # interactive mode
dh chat "what kind of projects do I keep building?"         # fine-tuned Phi-3 + SQLite search
dh query "amrita conversations"                             # raw D1 semantic search
dh server                                                   # start query server on :8000

Or via bazinga --kb-d1

bazinga --kb-sources                                                         # check D1 reachable
bazinga --kb "your query" --kb-d1                                           # raw D1 search
bazinga --kb "your query" --kb-d1 --summarize --local                       # LLM summary
bazinga --kb "your query" --kb-d1 --summarize --voice --local               # reconstruction in your voice
bazinga --kb "your query" --kb-d1 --summarize --voice --memory-model        # fine-tuned Phi-3

Good queries

dh reconstruct "what was I building in early 2025"
dh reconstruct "what was happening with amrita late 2024"
dh reconstruct "ZK proofs consciousness substrate"
dh reconstruct "darmiyan lambda-g work"
dh reconstruct "what was I going through mid 2024"
dh reconstruct "phi crystal fractal dimension"

---

How it works (the pipeline)

Your query
    │
    ▼
query_server (localhost:8000)
    └── Roberts-16D cosine search over 496K embeddings in D1
    └── Returns top-15 real messages with dates and sources
    │
    ▼
8-axis tonal scoring (deterministic, no LLM)
    └── assertion, concrete, self, present, warmth, certainty, action, technical
    └── Produces tonal target for generation
    │
    ▼
Generator (--local = Ollama llama3, --memory-model = fine-tuned Phi-3)
    └── Prompt = fragments + tonal target + reconstruction role
    └── "Speak in his voice. Not a summary. A memory."
    │
    ▼
Reconstructed memory in Space's voice

---

Data sources — what each command searches

Command	Data source	Notes
dh reconstruct	D1 digital-history only	149K messages, semantic search, clean
dh chat	local_copy.sqlite (keyword)	offline, same data, no embeddings
dh query	D1 digital-history only	raw semantic search
bazinga --kb-d1	D1 digital-history only	via query_server HTTP
bazinga --kb (no --kb-d1)	~/.bazinga index	Gmail exports, GDrive, Mac files — noisy
bazinga --ask / --chat	~/.bazinga RAG + cloud LLMs	general purpose, not memory-specific

Rule: for memory reconstruction, always use --kb-d1 or dh reconstruct. Never use plain bazinga --kb without --kb-d1 for personal memory queries — the ~/.bazinga index contains noisy local file scraps that are already in D1 at higher quality.

---

Lower-level interface — query.py directly

For raw search without bazinga (needs wrangler + Cloudflare auth):

cd ~/Documents/digital-history

python3 query.py "when did I talk about moving cities"
python3 query.py "amrita conversations about future" --k 30
python3 query.py "authentication bug" --source commit
python3 query.py "ZK proof ideas" --source message
python3 query.py -i    # interactive mode

---

What's in the database

Table	Rows	What
messages	149,515	WhatsApp, Telegram, Claude, ChatGPT, Gmail
entity_embeddings	496,665	384-dim semantic vectors
conversations	4,790	Thread groupings
github_commits	3,067	Every git commit
github_repos	79	Repos
authored_content	78	Articles written
transactions	63	Financial records
legal_records	30	Legal docs
people	10	Amrita, Aishwarya, Manasi, Mini, Asmita...

DB size: 1.23 GB on Cloudflare D1 (paid tier, 10GB limit)

Message sources

source_id	Source
1	Gmail
2	WhatsApp
3	Telegram
4	Claude
5	ChatGPT

---

Files

File	What
query.py	Direct D1 semantic search (Roberts-16D + 384D refine)
query_server.py	FastAPI wrapper — serves query.py over localhost:8000
chat.py	Prototype RAG + fine-tuned model (superseded by bazinga --kb-d1)
finetune.py	QLoRA training script (Phi-3-mini, Apple MPS)
build_dataset_v2.py	Dataset builder (pulls from D1, saves local SQLite)
local_copy.sqlite	Offline copy of all 149K messages (176MB, not in git)
finetune_dataset_v2.jsonl	Training dataset (1,133 weekly windows, not in git)
abhishek-memory-model/	Fine-tuned LoRA adapter (34MB, not in git)
PRD.md	Full product spec

---

Local vs Cloud

Operation	Internet needed?
bazinga --kb-d1 --voice --local	Yes — D1 embeddings fetched at server startup
After query_server loads	No — vectors in RAM, Ollama is local
--memory-model (fine-tuned Phi-3)	No — model is local
Re-training	No — dataset and model are local
Adding new data to D1	Yes

---

The fine-tuned model

• Base: Phi-3-mini-4k-instruct (Microsoft, 3.8B params)
• Adapter: QLoRA rank=16, 8.9M trainable params, 34MB on disk
• Trained on: 1,133 weekly windows — all messages, chronological, cross-source
• Training: ~7 hours on Apple MPS
• Use: bazinga --kb "..." --kb-d1 --summarize --voice --memory-model
• Note: loads the 7GB base model + 34MB adapter. Takes ~30 sec on first use.

---

Repo

bazinga integration: ~/github-repos-bitsabhi/bazinga-indeed/bazinga/kb.py + cli/_core.py GitHub: https://github.com/0x-auth/digital-history (private) bazinga-indeed: https://github.com/0x-auth/bazinga-indeed

---

φ = 1.618 · Genesis prime: 137 · Memory is reconstruction, not retrieval