nenya-mac

Intro

Nenya is a general purpose 0.5B active, 2.5B total parameter mixture-of-experts (MoE) base model. Nenya rivals Qwen3.5 2B on pretraining benchmarks with 500× lower training compute and 3.4× fewer active FLOPs/token.

This package is Nenya's Metal engine for Apple silicon. With it installed, nenya.generate() runs the model on the Mac's GPU, from 8-bit weights in a single 3.47 GB file hosted in ArsalanZ/Nenya-2.5B-A0.5B-Base-Metal-8bit.

Quick start

pip install "nenya[mac]"

import nenya
print(nenya.generate("The mitochondrion is the organelle responsible for", max_new_tokens=40))
print(nenya.generate("Question: What is the capital of Australia?\nAnswer:", max_new_tokens=8, temperature=0))

pip install "nenya[mac]" installs two packages: nenya, which provides nenya.generate(), and this engine, which it runs on. Installing nenya-mac on its own gives you only the engine (see "Direct use" below).

Benchmarks

Speed and memory on a Mac

One sequence at a time, in the Metal engine.

Prompt processing and time to first token (Apple M4). Prompts run through every layer at once, in chunks of 512 tokens:

Usage notes

• Download: the first call downloads the weight file (3.47 GB) from ArsalanZ/Nenya-2.5B-A0.5B-Base-Metal-8bit and caches it with huggingface_hub. The engine stays loaded between calls; nenya.unload() frees it.
• Sampling: the default is temperature 0.9 with top-k 20, the model's recommended setting for open-ended text. Use temperature=0 (greedy) for maths, code and short factual answers. top_p and min_p are supported too.
• Backend: nenya.generate() uses the engine automatically. A call that passes transformers' own generate() arguments (e.g. repetition_penalty) runs on transformers instead, with a warning. backend="mac" or backend="transformers" picks one.
• Requirements: a Mac with Apple silicon (M1 or later) and Python 3.12. macOS 11+. The wheel ships the compiled engine, so no Xcode tools are needed.

Direct use, without the text wrapper:

import nenya_mac
eng = nenya_mac.Engine(nenya_mac.default_weights())   # one decoding session (up to 16,384 tokens)
eng.append(prompt_ids)                                  # the prompt: batched through every layer
logits = eng.logits()                                   # next-token logits, numpy float32 [vocab]
new_ids = eng.generate([], 40, temperature=0.9, top_k=20, seed=1, stop=[100257])
eng.reset()

Your own fine-tune, saved in the release's format, runs on the engine after a conversion: pip install "nenya-mac[convert]", then python -m nenya_mac.convert <folder or repo id> my.oe, and set NENYA_MAC_WEIGHTS=my.oe. Generating from another repo without converted weights falls back to transformers.

The weight file

The file holds 8-bit matrices in groups of 32, a 4-bit screening copy of the output head with exact fp16 re-scoring of the candidates, the mHC boundaries folded into the output projections, and fp16 Engram tables.

License

Apache 2.0, like the model (LICENSE, NOTICE).