Piliwela

Fast Sinhala legacy-font to Unicode conversion for Python

Convert FM-family Sinhala text while preserving English content, PDF layout metadata, and PyMuPDF's page structure.

Why Piliwela?

Many older Sinhala documents use legacy fonts where Sinhala characters are stored as Latin/ASCII codes. Extracting text from those PDFs can produce unreadable strings such as:

Y%S ,xld

Piliwela converts that text into modern Unicode Sinhala:

ශ්‍රී ලංකා

It uses PDF font metadata when available and preserves English words in mixed-language content.

Features

Rust-powered conversion engine with Python bindings
FM-family legacy Sinhala to Unicode conversion
Automatic conversion using PDF font metadata
English and mixed-language text preservation
Single-span and batched-span APIs
PyMuPDF-compatible page dictionary conversion
Complete-document and page-streaming PDF APIs
Source font, bounding box, size, flags, block, and line preservation
Optional raw-text and conversion metadata
Per-document conversion reports
Python 3.9–3.13 support

Supported fonts

Piliwela 1.0 supports the FM legacy-font family, including fonts whose normalized names begin with FM and subset-prefixed PDF names such as:

FMAbhaya
FMSamantha
FMEmanee
BCDEEE+FMAbhaya

DL, Wijeya, and Kaputa mappings are not yet part of the supported 1.0 API.

Installation

For text and span conversion:

pip install piliwela

For PDF document conversion with PyMuPDF:

pip install "piliwela[pdf]"

Quick start

import piliwela

converted = piliwela.convert_auto_with_metadata(
    "Y%S ,xld",
    "FMAbhaya",
)

print(converted)

Output:

ශ්‍රී ලංකා

Convert plain text

Use convert_auto() when font metadata is unavailable:

import piliwela

converted = piliwela.convert_auto("Y%S ,xld")
print(converted)

For PDF files, prefer the metadata-aware APIs because embedded font names provide safer detection.

Convert one span with metadata

import piliwela

result = piliwela.convert_span(
    "Y%S ,xld",
    "FMAbhaya",
)

print(result.text)
print(result.raw_text)
print(result.font_name)
print(result.family)
print(result.changed)
print(result.detection_source)

Example result:

ශ්‍රී ලංකා
Y%S ,xld
FMAbhaya
FM
True
font_metadata

Convert multiple spans efficiently

The batch API performs one Python-to-Rust call for the complete group:

import piliwela

results = piliwela.convert_spans(
    [
        ("Y%S ,xld", "FMAbhaya"),
        ("English textbook", "Helvetica"),
    ]
)

for result in results:
    print(result.text)

Output:

ශ්‍රී ලංකා
English textbook

Convert a PyMuPDF page dictionary

If your application already opens the PDF with PyMuPDF, use convert_page_dict():

import pymupdf
import piliwela

with pymupdf.open("textbook.pdf") as document:
    page_data = document[0].get_text("dict")

converted_page = piliwela.convert_page_dict(page_data)

The returned value preserves PyMuPDF's structure:

page
└── blocks
    └── lines
        └── spans
            ├── text
            ├── font
            ├── size
            ├── flags
            └── bbox

Only span["text"] is replaced. The input dictionary is not modified unless in_place=True is supplied.

Preserve raw text and conversion metadata

converted_page = piliwela.convert_page_dict(
    page_data,
    preserve_raw=True,
    include_metadata=True,
)

Each text span then includes:

{
    "text": "ශ්‍රී ලංකා",
    "raw_text": "Y%S ,xld",
    "font": "FMAbhaya",
    "_piliwela": {
        "family": "FM",
        "changed": True,
        "detection_source": "font_metadata",
    },
}

Convert an entire PDF

import piliwela

result = piliwela.convert_pdf(
    "textbook.pdf",
    sort=True,
    include_images=False,
    preserve_raw=True,
    include_metadata=True,
)

for page in result.pages:
    for block in page["blocks"]:
        for line in block.get("lines", []):
            text = "".join(
                span["text"]
                for span in line.get("spans", [])
            )
            if text.strip():
                print(text)

convert_pdf() accepts:

A filesystem path
bytes
bytearray
memoryview
A binary file object

convert_document() is an alias of convert_pdf().

Stream large PDFs

Use iter_pdf() for large textbooks so only one converted page is held at a time:

import piliwela

for page in piliwela.iter_pdf(
    "large-textbook.pdf",
    sort=True,
):
    process(page)

Conversion report

convert_pdf() returns a ConvertedDocument containing converted pages and a report:

result = piliwela.convert_pdf("textbook.pdf")
print(result.report.to_dict())

Example:

{
    "pages": 120,
    "text_blocks": 3912,
    "lines": 18440,
    "spans": 22108,
    "converted_spans": 17902,
    "unchanged_spans": 4206,
    "family_counts": {
        "FM": 17902,
        "Unknown": 4206,
    },
}

Save converted text

import piliwela

result = piliwela.convert_pdf("textbook.pdf", sort=True)
pages = []

for page_number, page in enumerate(result.pages, start=1):
    lines = []

    for block in page["blocks"]:
        for line in block.get("lines", []):
            text = "".join(
                span["text"]
                for span in line.get("spans", [])
            )
            if text.strip():
                lines.append(text)

    pages.append(
        f"--- Page {page_number} ---\n" + "\n".join(lines)
    )

with open("converted_textbook.txt", "w", encoding="utf-8") as file:
    file.write("\n\n".join(pages))

API overview

API	Purpose
`convert_auto(text)`	Convert text without font metadata
`convert_auto_with_metadata(text, font_name)`	Convert text using a PDF font name
`convert_span(text, font_name)`	Return converted text with typed metadata
`convert_spans(spans)`	Convert multiple spans in one Rust batch
`convert_page_dict(page_data)`	Convert a PyMuPDF page dictionary
`convert_page(page_data)`	Return a converted page and report
`convert_pdf(source)`	Convert a complete PDF into structured pages
`convert_document(source)`	Alias of `convert_pdf()`
`iter_pdf(source)`	Stream converted page dictionaries
`detect(text)`	Detect a legacy family from text heuristics
`detect_from_metadata(font_name)`	Detect a family from a PDF font name
`version()`	Return the installed Piliwela version

Important behavior

Digital PDFs only

Piliwela converts text already present in a PDF's text layer. It does not perform OCR. Use an OCR engine first for image-only or scanned PDFs.

Structured output, not a rewritten PDF

convert_pdf() returns converted text in PyMuPDF-compatible page dictionaries. It does not visually replace glyphs or generate a newly typeset PDF.

Source geometry is preserved

Bounding boxes identify where the original legacy glyphs appeared. They are source PDF coordinates, not recalculated Unicode text dimensions.

Images

Images are excluded by default to reduce memory usage. Preserve PyMuPDF image blocks with:

result = piliwela.convert_pdf(
    "textbook.pdf",
    include_images=True,
)

Reading order

PDF internal text order may differ from visual reading order. Pass sort=True to request PyMuPDF's top-left to bottom-right sorting:

result = piliwela.convert_pdf("textbook.pdf", sort=True)

Error handling

import piliwela

try:
    result = piliwela.convert_pdf("textbook.pdf")
except piliwela.PDFDependencyError:
    print('Install PDF support with: pip install "piliwela[pdf]"')
except piliwela.DocumentConversionError as error:
    print(f"Invalid page structure: {error}")

Use in a document-ingestion service

Add the dependency:

piliwela[pdf]>=1.0.0,<2.0.0

If the service already has an open PyMuPDF page, avoid reopening the document:

page_data = page.get_text("dict")
converted_page = piliwela.convert_page_dict(
    page_data,
    in_place=True,
)

Development

git clone https://github.com/Naviya-C/piliwela.git
cd piliwela

python -m venv .venv
source .venv/bin/activate

pip install -U pip maturin
pip install ".[dev]"
maturin develop
pytest
cargo test --locked

Build a release wheel:

maturin build --release --out dist

Contributing

Contributions are welcome, particularly:

Verified mappings for additional Sinhala legacy-font families
Real-world FM-family regression examples
Mixed Sinhala-English conversion tests
PDF extraction edge cases
Documentation improvements

Please include tests and anonymized sample strings for conversion changes.

License

Piliwela is released under the MIT License.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

piliwela-1.0.0.tar.gz (53.6 kB view details)

Uploaded Aug 15, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

piliwela-1.0.0-cp39-abi3-win_amd64.whl (246.8 kB view details)

Uploaded Aug 15, 2026 CPython 3.9+Windows x86-64

piliwela-1.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (453.0 kB view details)

Uploaded Aug 15, 2026 CPython 3.9+manylinux: glibc 2.17+ x86-64

piliwela-1.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (495.9 kB view details)

Uploaded Aug 15, 2026 CPython 3.9+manylinux: glibc 2.17+ ARM64

piliwela-1.0.0-cp39-abi3-macosx_11_0_arm64.whl (404.6 kB view details)

Uploaded Aug 15, 2026 CPython 3.9+macOS 11.0+ ARM64

piliwela-1.0.0-cp39-abi3-macosx_10_12_x86_64.whl (399.4 kB view details)

Uploaded Aug 15, 2026 CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file piliwela-1.0.0.tar.gz.

File metadata

Download URL: piliwela-1.0.0.tar.gz
Upload date: Aug 15, 2026
Size: 53.6 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for piliwela-1.0.0.tar.gz
Algorithm	Hash digest
SHA256	`8e8e4671bab17a6b3c7d4e3e38c668c7a7ccd847ff02a7a29e7db6392dbdc5d5`
MD5	`1e544bb05d0c6e3202696004a041a9d0`
BLAKE2b-256	`667fec046e36e58f9747e21f24c7ec4551c394d63ed661cf3a18e114047d509f`

See more details on using hashes here.

Provenance

The following attestation bundles were made for piliwela-1.0.0.tar.gz:

Publisher: publish.yml on Naviya-C/piliwela

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: piliwela-1.0.0.tar.gz
- Subject digest: 8e8e4671bab17a6b3c7d4e3e38c668c7a7ccd847ff02a7a29e7db6392dbdc5d5
- Sigstore transparency entry: 2475291001
- Sigstore integration time: Aug 15, 2026
Source repository:
- Permalink: Naviya-C/piliwela@669e1529396616f25034ab831b107cbdf7bc9357
- Branch / Tag: refs/heads/main
- Owner: https://github.com/Naviya-C
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@669e1529396616f25034ab831b107cbdf7bc9357
- Trigger Event: workflow_dispatch

File details

Details for the file piliwela-1.0.0-cp39-abi3-win_amd64.whl.

File metadata

Download URL: piliwela-1.0.0-cp39-abi3-win_amd64.whl
Upload date: Aug 15, 2026
Size: 246.8 kB
Tags: CPython 3.9+, Windows x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for piliwela-1.0.0-cp39-abi3-win_amd64.whl
Algorithm	Hash digest
SHA256	`b70e41337aadfcad4ead76bf77c8fccae94f764b2157c5a6277e0a82556d7ca1`
MD5	`8391797c85ea697c06afa780a2fe2260`
BLAKE2b-256	`6cc60a605893af6c9fa08683930659ff6229876641c451688a5fc404b21c8f9f`

See more details on using hashes here.

Provenance

The following attestation bundles were made for piliwela-1.0.0-cp39-abi3-win_amd64.whl:

Publisher: publish.yml on Naviya-C/piliwela

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: piliwela-1.0.0-cp39-abi3-win_amd64.whl
- Subject digest: b70e41337aadfcad4ead76bf77c8fccae94f764b2157c5a6277e0a82556d7ca1
- Sigstore transparency entry: 2475291168
- Sigstore integration time: Aug 15, 2026
Source repository:
- Permalink: Naviya-C/piliwela@669e1529396616f25034ab831b107cbdf7bc9357
- Branch / Tag: refs/heads/main
- Owner: https://github.com/Naviya-C
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@669e1529396616f25034ab831b107cbdf7bc9357
- Trigger Event: workflow_dispatch

File details

Details for the file piliwela-1.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

Download URL: piliwela-1.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Upload date: Aug 15, 2026
Size: 453.0 kB
Tags: CPython 3.9+, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for piliwela-1.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm	Hash digest
SHA256	`6a0da801596dccc573b144a9e8074f64e6a36d2fc92b0960ef36494bbe54aa74`
MD5	`d699d060ee70eb477a04f88dee0db79f`
BLAKE2b-256	`30f650121b84da2ba1b515a56735c9070d387c2b57ecc06582fe5aec0c2cd22b`

See more details on using hashes here.

Provenance

The following attestation bundles were made for piliwela-1.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish.yml on Naviya-C/piliwela

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: piliwela-1.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
- Subject digest: 6a0da801596dccc573b144a9e8074f64e6a36d2fc92b0960ef36494bbe54aa74
- Sigstore transparency entry: 2475291223
- Sigstore integration time: Aug 15, 2026
Source repository:
- Permalink: Naviya-C/piliwela@669e1529396616f25034ab831b107cbdf7bc9357
- Branch / Tag: refs/heads/main
- Owner: https://github.com/Naviya-C
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@669e1529396616f25034ab831b107cbdf7bc9357
- Trigger Event: workflow_dispatch

File details

Details for the file piliwela-1.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

Download URL: piliwela-1.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Upload date: Aug 15, 2026
Size: 495.9 kB
Tags: CPython 3.9+, manylinux: glibc 2.17+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for piliwela-1.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm	Hash digest
SHA256	`28628fa0336868db7a1d329e4996b598cbcadc8f7eb287f7bd40330f994dcd0e`
MD5	`006dbc370b4025c8eb3d3bd4423c9918`
BLAKE2b-256	`97d9ca688f7e68d48b976acf7dcfe88ad1b79233b468528b709c268e09455cf3`

See more details on using hashes here.

Provenance

The following attestation bundles were made for piliwela-1.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish.yml on Naviya-C/piliwela

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: piliwela-1.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- Subject digest: 28628fa0336868db7a1d329e4996b598cbcadc8f7eb287f7bd40330f994dcd0e
- Sigstore transparency entry: 2475291119
- Sigstore integration time: Aug 15, 2026
Source repository:
- Permalink: Naviya-C/piliwela@669e1529396616f25034ab831b107cbdf7bc9357
- Branch / Tag: refs/heads/main
- Owner: https://github.com/Naviya-C
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@669e1529396616f25034ab831b107cbdf7bc9357
- Trigger Event: workflow_dispatch

File details

Details for the file piliwela-1.0.0-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

Download URL: piliwela-1.0.0-cp39-abi3-macosx_11_0_arm64.whl
Upload date: Aug 15, 2026
Size: 404.6 kB
Tags: CPython 3.9+, macOS 11.0+ ARM64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for piliwela-1.0.0-cp39-abi3-macosx_11_0_arm64.whl
Algorithm	Hash digest
SHA256	`aa3e0c5b3db629b4fe2eec3968ff0b62e35529fabf167f1884269d63e7b9954c`
MD5	`f61563911c1d6a4fe4bcf32533ba6c1c`
BLAKE2b-256	`0bbabe2fad9cd3684fcdcb03df1b646875983d804734806a913ca8ad1523b979`

See more details on using hashes here.

Provenance

The following attestation bundles were made for piliwela-1.0.0-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: publish.yml on Naviya-C/piliwela

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: piliwela-1.0.0-cp39-abi3-macosx_11_0_arm64.whl
- Subject digest: aa3e0c5b3db629b4fe2eec3968ff0b62e35529fabf167f1884269d63e7b9954c
- Sigstore transparency entry: 2475291139
- Sigstore integration time: Aug 15, 2026
Source repository:
- Permalink: Naviya-C/piliwela@669e1529396616f25034ab831b107cbdf7bc9357
- Branch / Tag: refs/heads/main
- Owner: https://github.com/Naviya-C
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@669e1529396616f25034ab831b107cbdf7bc9357
- Trigger Event: workflow_dispatch

File details

Details for the file piliwela-1.0.0-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

Download URL: piliwela-1.0.0-cp39-abi3-macosx_10_12_x86_64.whl
Upload date: Aug 15, 2026
Size: 399.4 kB
Tags: CPython 3.9+, macOS 10.12+ x86-64
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for piliwela-1.0.0-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm	Hash digest
SHA256	`5689678f5e6577803765a388b2842124960e97f8c1aa19d2d3cdc0f7a7f66cca`
MD5	`bfebf942ccfb10c5be024160306484ec`
BLAKE2b-256	`fbb8bcd08b0b2b1bb178bb1499d0820db465c5e47dad312132c37b578158f637`

See more details on using hashes here.

Provenance

The following attestation bundles were made for piliwela-1.0.0-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: publish.yml on Naviya-C/piliwela

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: piliwela-1.0.0-cp39-abi3-macosx_10_12_x86_64.whl
- Subject digest: 5689678f5e6577803765a388b2842124960e97f8c1aa19d2d3cdc0f7a7f66cca
- Sigstore transparency entry: 2475291060
- Sigstore integration time: Aug 15, 2026
Source repository:
- Permalink: Naviya-C/piliwela@669e1529396616f25034ab831b107cbdf7bc9357
- Branch / Tag: refs/heads/main
- Owner: https://github.com/Naviya-C
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: publish.yml@669e1529396616f25034ab831b107cbdf7bc9357
- Trigger Event: workflow_dispatch

Release history Release notifications | RSS feed

This release

1.0.0

Aug 15, 2026

piliwela 1.0.0

Piliwela

Why Piliwela?

Features

Supported fonts

Installation

Quick start

Convert plain text

Convert one span with metadata

Convert multiple spans efficiently

Convert a PyMuPDF page dictionary

Preserve raw text and conversion metadata

Convert an entire PDF

Stream large PDFs

Conversion report

Save converted text

API overview

Important behavior

Digital PDFs only

Structured output, not a rewritten PDF

Source geometry is preserved

Images

Reading order

Error handling

Use in a document-ingestion service

Development

Contributing

License

Project links

Key dates

1 maintainer

Credits

GitHub Statistics

License expression

Requires

Provides Extra

Tags

Classifiers

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

File details

File metadata

File hashes

Provenance

Release history Release notifications | RSS feed