Unicode Tools (uchr)

A powerful command-line tool for searching and exploring Unicode characters, emoji sequences, and character properties.

🚀 Features

• Search by name or meaning: Find characters by whole words of their Unicode name or, for CJK ideographs, their English meaning
• Related emoji: Search also lists emoji with a similar meaning (ghost → 👺 goblin, 👹 ogre), ranked by shared CLDR keywords
• Search by code: Look up characters by code point or range
• Search by character: Reverse lookup from character to details
• Search by block: Explore characters within Unicode blocks
• Emoji support: Full support for emoji sequences and ZWJ sequences
• CJK meanings: CJK ideographs are shown with their English meaning (Unihan kDefinition) in every search
• Multiple Unicode versions: Fetch any published version and switch between them
• Flexible output: Multiple output formats for different use cases

📖 Table of Contents

• Installation
• Quick Start
• Usage Examples
• Command Reference
• Database Management
• Text Normalization (deprecated)
• Advanced Examples
• Data Sources
• Development
• Contributing
• License

🛠 Installation

Install from PyPI (Recommended)

pip install uchr
# or, as an isolated command-line tool with uv
uv tool install uchr

Install from source

git clone https://github.com/mkyutani/uchr.git
cd uchr
uv tool install .

Initialize database

The database is created automatically on first use of uchr search, fetching the latest Unicode version. To fetch it explicitly instead:

uchr db update

This downloads the latest Unicode data and creates a local SQLite database at:

• Linux/macOS: ~/.local/share/uchr/unicode.db
• Root users: Automatically chooses between system (/var/lib/uchr/) or personal location

The database file is created with mode 0644: anyone who can reach it can search it, but only its owner can change it (db update/use/delete). For the system location this means root.

Multiple Unicode versions can coexist in the same database — see Database Management below.

⚡ Quick Start

# Search for ghost, followed by related emoji such as 👺
uchr search ghost

# Find characters in a code range
uchr search -c 1F47A-1F480

# Search by character
uchr search -x 👻

# Search within a Unicode block
uchr search -b "Emoticons"

📋 Usage Examples

Search by Name or Meaning

uchr search cat

猫 732B CAT
貓 8C93 CAT
🐈 1F408 CAT
😾 1F63E POUTING CAT FACE
🐯 1F42F TIGER FACE
...
鯴 9BF4 CAT FISH
鲺 9CBA CAT FISH
🐱 1F431 CAT FACE
🐅 1F405 TIGER
...
猞 731E A WILD CAT; 猞猁, A LYNX
...
챁 CC41 HANGUL SYLLABLE CAT
𐇬 101EC PHAISTOS DISC SIGN CAT
㣇 38C7 A KIND OF BEAST WITH LONG HAIR, OTHER NAME FOR PIG, FOX, WILD CAT, RACCOON

The expression must match whole words, of the character name or, for CJK ideographs, of their English meaning (Unihan kDefinition), which is shown in place of the name: cat finds 猫 (CAT) and HANGUL SYLLABLE CAT, but not INDICATOR or CATTLE. Search also lists related emoji, whose CLDR keywords overlap with those of the emoji the expression names (🐯 shares animal and cat with 🐈).

All results are ranked together by a score from 0 to 1, ties in code order:

• Word matches score by the share of the meaning (split into items at ; and ,) or else of the name that the expression takes up. An item counts half as much as the one before it, so 鬼 (GHOST; SPIRIT OF DEAD; …) scores 1 while 孽 (EVIL; SON OF CONCUBINE; GHOST) scores less. A partial match is lifted by 1 − (1 − score)², close to doubling it while an exact match stays at 1: 猫 (CAT) scores 1, 鯴 (CAT FISH) 0.61, 㣇 (…, WILD CAT, RACCOON) 0.09.
• Related emoji score by their cosine similarity of keywords, with keywords that many emoji share (like face) counting for little; 🐯 scores 0.64. An emoji that is both takes the higher score.

# Only closely related emoji (default threshold: 0.3)
uchr search ghost -t 0.4

# Exact name only, no related emoji
uchr search -s ghost

Related emoji need the keyword data that uchr db update fetches; until then, search lists the word matches only and prints a note. Only emoji have keywords. Emoji newer than the latest CLDR release, and flags and skin-tone variants (CLDR's derived annotations), have no keywords and are found by name alone.

Search by Code Range

uchr search -c 1F479-1F47B

👹 1F479 JAPANESE OGRE
👺 1F47A JAPANESE GOBLIN
👻 1F47B GHOST

Search by Character

uchr search -x 👻

👻 1F47B GHOST

CJK ideographs show their English meaning instead of their name in every search, and names derived from the code point are spelled out:

uchr search -x 猫
uchr search -c 17000

猫 732B CAT
𗀀 17000 TANGUT IDEOGRAPH-17000

Search by Unicode Block

uchr search -b "Misc_Pictographs"

Search Without Related Emoji

uchr search -d "pray for happiness"

祝 795D PRAY FOR HAPPINESS OR BLESSINGS

-d searches names and CJK meanings for whole words like the default search, best match first, but leaves out related emoji.

Output Formatting

# Simple format (characters only; multiple matches are concatenated)
uchr search goblin -f simple
👺䰨䰪👹👽👻👾🐲🧌👼🧚👿䰦

# UTF-8 format (UTF-8 bytes instead of code points)
uchr search -s "japanese goblin" -f utf8
👺 F09F91BA JAPANESE GOBLIN

# Custom delimiter
uchr search -s "japanese goblin" -D "|"
👺|1F47A|JAPANESE GOBLIN

🔧 Command Reference

uchr

Main command with subcommands for all Unicode operations.

uchr search

Search Unicode characters with various criteria.

uchr normalize (deprecated)

Unicode text normalization and conversion. Deprecated — see Text Normalization.

uchr db

Database management operations. Multiple Unicode versions can be stored in the database at once; current_version (set by update/use) is what search uses by default.

💾 Database Management

The database is created automatically the first time uchr search runs with no local database yet — it fetches the latest Unicode version. There's no separate "create" step to run manually.

Fetch a Version

# Fetch the latest version and make it current
uchr db update

# Fetch a specific version
uchr db update --version 16.0.0

A version that is already stored is not downloaded again; update just makes it current. The exception is a version stored as unreleased draft data, which is fetched again so it picks up changes (or the final release).

update also keeps the CLDR emoji keywords behind related emoji up to date: it looks up the latest CLDR release on GitHub and downloads its keywords only when that release isn't stored yet. The keywords are shared by all stored versions. A database created by an older uchr gets them on its next uchr db update. If the keywords can't be fetched, update warns and still succeeds, keeping any keywords stored before.

Switch Between Locally Stored Versions

uchr db use 16.0.0

List Versions

uchr db list

4.1.0
5.0.0
...
16.0.0 (local) [157667 chars]
17.0.0 (current) (latest) [162626 chars]
18.0.0 (draft)

Versions with no marker are published on unicode.org but not stored locally.

Search a Specific Version Without Switching

uchr search ghost --unicode-version 16.0.0

If that version isn't stored locally yet, this errors and tells you to run uchr db update --version 16.0.0 first — it won't silently download on a possibly-mistyped version number.

Remove Data

# Delete one version's data
uchr db delete 16.0.0

# Wipe the entire database file
uchr db delete --all

🔤 Text Normalization

Deprecated: uchr normalize will be removed in a future release. It uses the Unicode data built into Python (unicodedata.unidata_version, e.g. 15.0.0 on Python 3.12), not the uchr database, so it ignores uchr db versions. It prints a warning to stderr when used.

The uchr normalize command provides Unicode text normalization and conversion utilities.

Basic Normalization

# Normalize to NFC (default)
echo "café" | uchr normalize

# Normalize to NFD
echo "café" | uchr normalize --form nfd

# Normalize to NFKC
echo "file" | uchr normalize --form nfkc

# Normalize to NFKD
echo "file" | uchr normalize --form nfkd

Text Conversion

# Convert fullwidth to halfwidth
echo "ＨｅｌｌｏＷｏｒｌｄ" | uchr normalize --halfwidth

# Show detailed binary representation
echo "café" | uchr normalize --detail

# Compare all normalization forms
echo "café" | uchr normalize --compare

Interactive Mode

# Read from stdin (interactive)
uchr normalize
# Type text and press Ctrl+D to process

🌟 Advanced Examples

Finding Emoji Sequences

# National flags
uchr search -b "RGI_Emoji_Flag_Sequence"

# Family emoji with ZWJ sequences
uchr search family

Terminal Display vs. Browser/Application Support

Many terminals don't properly display complex emoji sequences, but the characters work correctly when copied to browsers or applications.

National Flag Example

When searching for flags, you might see separate letters in your terminal:

uchr search -b "RGI_Emoji_Flag_Sequence" | grep -i norway

🇳🇴 1F1F3 1F1F4 flag: Norway

Even though you see two separate letters (🇳🇴) in the terminal, when you copy and paste them into a browser or application like Twitter, they combine to display the Norwegian flag 🇳🇴.

ZWJ Sequence Example

The same applies to Zero Width Joiner (ZWJ) sequences. Complex emoji like family groups or professional emoji might not render correctly in terminals:

uchr search "polar bear"

In a terminal without proper font support:

But when pasted in Twitter or other applications:

💡 Tip: This is expected behavior. The Unicode data is correct, and the characters will work properly in applications that support modern emoji rendering.

Pipe Operations

# Get just the character
uchr search -s "japanese goblin" -f simple

# First match only
uchr search snow -1

# Custom format for scripting
uchr search ghost -D "," | cut -d',' -f1

Complex Searches

# CJK characters with specific meanings
uchr search -d "dragon"

# Characters in multiple blocks
uchr search -b "Mathematical" | head -10

📚 Data Sources

This tool fetches official Unicode data directly from unicode.org — the Unicode Character Database (ucdxml) and Emoji Sequences for whichever version you request with uchr db update. By default it tracks the latest released version; run uchr db list to see everything currently published.

Keywords for related emoji come from the English emoji annotations of the Unicode CLDR project (common/annotations/en.xml of the latest release in unicode-org/cldr).

🏗 Development

This project uses uv for dependency management and packaging.

Development Setup

# Clone the repository
git clone https://github.com/mkyutani/uchr.git
cd uchr

# Create .venv and install dependencies (from uv.lock)
uv sync

# Run tests
uv run pytest

# Format code
uv run ruff format

# Lint code
uv run ruff check

Building

uv build

Releasing

Releases are published by GitHub Actions (.github/workflows/release.yml). Bump version in pyproject.toml, merge it to main, then push a matching tag:

git tag -a vX.Y.Z -m "vX.Y.Z"
git push origin vX.Y.Z

The tag must equal v + the pyproject.toml version, or the build fails. The workflow then runs the tests, builds the sdist and wheel, publishes them to TestPyPI, installs the release back from TestPyPI as a smoke test, waits for approval on the pypi environment, publishes to PyPI, and creates a GitHub Release with the built files attached.

Uploads use Trusted Publishing, so no API tokens are stored in the repository. One-time setup:

1. PyPI: in the uchr project's Publishing settings, add a GitHub publisher — owner mkyutani, repository uchr, workflow release.yml, environment pypi.
2. TestPyPI: add the same publisher with environment testpypi (as a pending publisher if the project does not exist there yet).
3. GitHub: create the environments testpypi and pypi under Settings → Environments. Give pypi a required reviewer so the production upload waits for approval, and restrict both to v* tags.

Tests also run on every pull request and push to main (.github/workflows/ci.yml).

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes and add tests
4. Run the test suite: uv run pytest
5. Format your code: uv run ruff format
6. Lint your code: uv run ruff check
7. Commit your changes (git commit -m 'Add amazing feature')
8. Push to the branch (git push origin feature/amazing-feature)
9. Open a Pull Request

Code Quality

This project maintains high code quality standards:

• Type hints: All functions should include type annotations
• Testing: New features should include appropriate tests
• Documentation: Update README and docstrings for new features
• Code style: Follow Ruff formatting and linting rules

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Unicode Consortium for maintaining Unicode standards
• uv for Python packaging and project management
• Contributors and users of this project

📊 Project Status

• Current Version: 1.2.0
• Python Support: 3.11+
• Unicode Version: tracks the latest released version by default; multiple versions can coexist locally (see uchr db list)
• Package Name: uchr (on PyPI)
• Repository: mkyutani/uchr

For the latest updates and roadmap, see our GitHub Issues.