sitelen-emoji 1.1.0

toki pona to sitelen emoji lookup and translation helpers

sitelen-emoji-truth

A pinned, versioned “source of truth” profile for toki pona → sitelen emoji, with reproducible book-ready visuals.

Canonical frozen mapping for toki pona → sitelen emoji.

License: MIT. See LICENSE.

Goal: one stable “source of truth” for production (translator, books), with reproducible visuals.

Browse the mapping: https://markoblogo.github.io/sitelen-emoji-truth/

Free toki pona Reader’s Kit: https://toki.abvx.xyz/kit

---

Use the data

• Browser: open the GitHub Pages viewer.
• JSON: pin a tag and fetch profiles/default-stable.v1.json.
• Python/Node: use the local package helpers while npm/PyPI publishing is prepared.
• Profile variants: use profiles/minimal.v1.json for core words only or profiles/extended.v1.json for community/upstream entries.

What is frozen vs generated

• Frozen (source of truth):

  • profiles/default-stable.v1.json — pinned mapping intended for integrations and publishing.
  • profiles/minimal.v1.json — core 120 toki pona words only.
  • profiles/extended.v1.json — broader community/upstream mapping.
  • profiles/schema.json — JSON Schema for validating profile files.

• Generated (for comparison / upstream tracking):

  • dist/default-stable.json — produced by tools/build_default_stable.py from upstream sources.
  • Use tools/diff_profiles.py to see what changed vs frozen.

---

Pinned profile URL (recommended for integrations)

Pin to a git tag (recommended) and fetch the frozen profile via raw.githubusercontent.com.

Example:

https://raw.githubusercontent.com/markoblogo/sitelen-emoji-truth/v1.0.0/profiles/default-stable.v1.json

Replace v1.0.0 with the tag you want to pin to.

Why pin: your translator/book pipeline should not change output unless you intentionally update the pinned version.

---

Translator integration (runtime behavior)

Recommended approach:

1. Fetch the pinned frozen JSON on startup (by tag URL above).
2. Parse JSON and keep it in memory (optionally cache to disk/redis).
3. Resolve aliases (e.g. ali → ale) and map word → entries[word].

Do not auto-update from main or “latest” without a version bump/tag change.

---

Dev setup

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-dev.txt
python -m pytest -q

---

Build (generate dist)

python tools/build_default_stable.py

---

Documentation

• Mapping reference
• Coverage report
• Design principles
• Publishing setup
• Changelog

Generated docs should be regenerated after profile changes:

python3 -m tools.export_mapping_md
python3 -m tools.coverage_report

---

Examples

python3 examples/python-load-profile.py
node examples/node-load-profile.js

Local package helpers

Python:

from sitelen_emoji import lookup, translate

lookup("toki")          # "🗣️"
translate("jan pona")  # "👤 👍"

Node:

const { lookup, translate } = require("./packages/js");

lookup("toki");          // "🗣️"
translate("jan pona");  // "👤 👍"

---

Books pipeline

1) Convert toki pona text → sitelen emoji tokens

Input: .txt or .md with toki pona text

Output: a text file where tokens are mapped to emoji (spaces preserved, newlines preserved)

python3 -m tools.convert_tp_text --in book_tp.txt --out book_se.txt

Options:

• --no-dot to keep . as text (otherwise mapped to _punct_period)
• --no-colon to keep : as text (otherwise mapped to _punct_colon)

2) Visual-stable build (HTML + optional PDF)

This renders emoji as Twemoji PNG so visuals are consistent across Kindle/apps/devices.

Fetch Twemoji assets (once per machine/version):

python3 -m tools.fetch_twemoji_assets

Build visual HTML (copies only used PNGs into the output folder):

./scripts/visual_build.sh book_se.txt out/visual
open out/visual/index.html

Optional PDF (requires Google Chrome installed):

./scripts/visual_build.sh --fetch --pdf book_se.txt out/visual
open out/visual/book.pdf

---

Updating upstream safely (without breaking published output)

1. Regenerate dist/ from upstream:

python tools/build_default_stable.py

2. Compare frozen vs new generated:

python3 -m tools.diff_profiles

3. If you intentionally want a new frozen version, create a new file under profiles/

   (e.g. profiles/default-stable.v2.json), update tests if needed, then tag a new release.

---

Releasing a pinned version

git tag -a v1.0.0 -m "Pinned frozen profile"
git push origin v1.0.0

Then consumers can pin to:

https://raw.githubusercontent.com/markoblogo/sitelen-emoji-truth/v1.0.0/profiles/default-stable.v1.json

---

License & attribution

• Repository code and profiles: MIT License (see LICENSE).
• Twemoji graphics are not included in this repository and are fetched separately. If you publish outputs that embed Twemoji PNG, include attribution (Twemoji is CC BY 4.0).

Want to read more toki pona?

Free beginner-friendly Reader’s Kit (PDF): https://toki.abvx.xyz/kit