pykeedy 0.1.0

Tools for easy statistical analysis of the Voynich Manuscript

A Python library for easy statistical analysis of the Voynich Manuscript.

Installation

pip install pykeedy

Development:

git clone https://github.com/pwspen/pykeedy.git
cd pykeedy
pip install -e .

Features

• Easily-loadable VMS transliteration (EVA or CUVA alphabet)
• Filtering by every available property (Currier language, illustration type, locus type, etc)
• Naibbe encoder supporting arbitrary encoding tables + decoder implementing algorithm from paper
• Handful of comparison manuscripts in European languages + easily load and chuck em all into the same analysis
• Plotter functions for common data types / analyses (see examples)
• Functions to easily calculate, at either character or word level:
  • n-gram frequency rank (most common character, word, character pair, word pair...)
  • n-gram co-occurence (aka, pair attraction for n=2)
  • entropy
    • single (Shannon), pair, conditional
  • Position distributions (letter in word, word in line, letter in page, etc)
• Highly composable and extensible
• Automatic analysis: All of these plots were generated with a single function call
  • (see examples/full_analysis.py)

Usage

from pykeedy import VMS, LocusProp
from pykeedy.analysis import shannon_entropy, conditional_entropy
from pykeedy.utils import load_corpus, scatterplot
from pykeedy.crypt import naibbe_encrypt

vms = VMS.to_text()  # Single string
vms = VMS.to_lines() # List of line strings
vms = VMS.to_words() # List of word strings

# Processing options
vms = VMS.to_text(alphabet="cuva", normalize_gaps=False)

# Filter by any property, or many at once
voynich_b = VMS.filter(props=[LocusProp.CurrierLanguage.B]).to_text()

plains = load_corpus() # Loads set of comparison plaintexts 
                       # (Latin, German, Italian)

analyze = {}
for name, text in plains.items()
    analyze[name] = text
    # Encrypt plaintexts in method that best matches Voynich statistically
    analyze[name + '_naibbe_encrypt'] = naibbe_encrypt(text)
analyze['vms b'] = voynich_b # Add VMS

results = {name: (shannon_entropy(text), conditional_entropy(text))
                                    for name, text in analyze.items()}

scatterplot(results) # Saves to scatterplot.png

[!WARNING] This library is not yet stable. You may find bugs and updates may break things. Please report any bugs.

Documentation

The current documentation is 10+ thoroughly commented examples which you can find in the examples folder.

There are examples that plot each of the following in around 20 lines or less:

• Word frequency in Voynich A and B
• Character and conditional entropy for the VMS vs plaintext comparison manuscripts
• Word length distribution (token and type)
• Character and word co-occurence / pair attraction heatmaps
• Most common character & word pairs / triplets
• Character & word position distributions
• Entropies of VMS + plaintexts + Naibbe cipher encryptions of those plaintexts
• You can see all of the plots generated by examples in examples/results.md

Goals

Library goals:

1. Lower the barrier to entry for Voynich statistical analysis
   • There are some tools out there, but they're scattered, relatively basic, and far from user friendly, even for programmers. This library aims to be very capable while not sacrificing any usability.
2. Reproduce all the important statistical results in one place
   • There is a TON of analysis out there, but it's even more scattered, and a lot of it is just results with no reproduction available, which means it's difficult to tweak and experiment with and build on.

Issues & PRs welcome!

Voynich resources

• Voynich Forum
• Most complete source of Voynich information
  • Voynich transliteration
  • Page by page overview
  • Existing Voynich software
    • bitrans - performs plaintext substitutions
    • IVTT - CLI for filtering and removing metadata from IVTFF files
    • You should not need either of these if you are using this library
• Most detailed Voynich scan
• Scan browser
• Multispectral imaging (direct link)
• Naibbe cipher paper (July 2025)
  • This was the inspiration for this library. It's IMO a very big result and you should read it!
• IVTFF format explanation
• Voynich Unicode

Wishlist / todo

Advanced

• Automated topic analysis
• Measure of clustering strength for similar words

Research directions

• Auto-solving Naibbe ciphers
• Simple reference self-citation algorithm

General

• docstrings
• then generate real docs
• Find better corpus for comparison manuscripts
• More composable system for corpus
• Option to generate subplots
• More options for dealing with spaces and newlines in entropy, freq, cooccurence analysis
• STA alphabet support
• character and word pair attraction symmetry scalar
• Support for recognizing single glyphs in non-basic EVA
• Measure of Naibbe encoding ambiguity
• Voynich font alongside transliteration for plots
• basic testing suite
• support for bitrans rulefiles
• encoding scorer (ambiguity, reconstruction, simplicity, match to VMS)