Skip to main content

Automated Japanese vocabulary mining from media with Anki integration

Project description

Anki Miner

PyPI version Python 3.11+ License: GPL v3 GitHub downloads GitHub stars Discord community

Turn native Japanese content into Anki vocabulary cards.

Please leave a ⭐ star if Anki Miner helped you - it helps others find it :).

Mining Demo

Anki Miner Showcase

⬇️ Full demo with sound (MP4)

Example cards

ホント いちゃいちゃ 代
⬇️ MP4 (sound) ⬇️ MP4 (sound) ⬇️ MP4 (sound)

Installation

Requirements

  • ffmpeg only if installing from pip/pipx, .deb, or from source.
  • alass (optional) for automatic subtitle retiming. Linux and Windows release builds bundle it. macOS users: brew install alass or place it on PATH.
  • Anki with the AnkiConnect add-on. In Anki: Tools -> Add-ons -> Get Add-ons, paste code 2055492159, restart.

Download

Grab the download for your platform from the latest release:

Platform Download
Windows AnkiMiner-*-Setup.exe
macOS (Apple Silicon / M1-M4) AnkiMiner-macOS-arm64.tar.gz
macOS (Intel) AnkiMiner-macOS-x86_64.tar.gz ¹
Linux (Debian/Ubuntu) anki-miner_*_amd64.deb
Linux (other) AnkiMiner-*-Linux-x86_64.AppImage

¹ The Intel macOS build excludes local Whisper speech-to-text (Subtitles->Generate) and AVIF animated screenshots - every other feature works. For full functionality on an Intel Mac, install via pip instead: pipx install "anki-miner[asr]".

macOS first-run (unsigned binary): macOS Gatekeeper will block the app because it is not notarised. Extract the archive first, then clear the quarantine flag on the extracted folder (clearing it on the .tar.gz does not carry over to the extracted files):

xattr -dr com.apple.quarantine AnkiMiner/

Windows first-run (SmartScreen): Windows SmartScreen may show "Windows protected your PC". Click More info, then Run anyway.

Windows Defender false positive: Defender may wrongly flag the unsigned .exe (it bundles yt-dlp/ffmpeg, common AV triggers). Restore it from Protection history or report it to Microsoft.

Install from PyPI (Python 3.11+)
pipx install anki-miner   # or: pip install anki-miner
Install from source
git clone https://github.com/0xzerolight/anki_miner.git
cd anki_miner
pip install -e .

For full development setup, see CONTRIBUTING.md.

Tabs

  • Video: mine video, in three sub-tabs. Single: one video/subtitle pair with word curation. Batch: a folder of episode/subtitle pairs for sequential processing — files are paired by episode number, so each folder / queue item should hold a single show (use Multi-Series Queue for mining multiple series at a time). YouTube: paste one or more URLs, then mine the queue.
  • Deck Builder: point at a folder of episode/subtitle pairs and mine the full series into one named deck. Ranked by frequency; pick how many to include (all, top N, or a coverage target) and preview before cards are created.
  • Audio: queue local audio + subtitle/transcript pairs (audiobooks, podcasts, radio, songs, lectures) and mine them audio-only; embedded cover art stands in for screenshots.
  • Reading: mine manga and novels instead of video. Point at a mokuro-processed manga volume (an image folder or .cbz with its sibling .mokuro file) or a novel (.epub, or Aozora/plain .txt); cards carry the page image or the book cover. Anki Miner reads mokuro's output and does no OCR itself. Word curation works as in the other tabs.
  • Analytics: history, series difficulty rankings, milestones, undo.
  • Tools: generate subtitles from speech with a local Whisper model (no GPU required; optional CUDA/VAD packs install in-app), retime an out-of-sync subtitle file to your video with alass, or condense a video/audio file down to dialogue-only audio using its subtitles.
  • Settings: Anki, Media, Dictionaries, Audio, Filtering, Frequency, Subtitles, YouTube, Themes. Saved to ~/.anki_miner/gui_config.json.

Other Features

  • Extensive filtering options (i+1 filter, frequency limits, word blacklist, subtitle regex filtering, wordset filtering, per-volume minimum word occurrence, and more).
  • Offline Yomitan dictionary import (definitions, pitch accent, frequency data) with priority ordering.
  • Multiple frequency lists chained together, each indexed separately and ordered by priority.
  • Expression (word-level) audio on cards from local audio packs, JapanesePod101, or Google Translate TTS (opt-in, chained).
  • Per-dictionary glossary styling, Yomitan-style — each dictionary's own scoped CSS is applied automatically, self-contained per card.
  • Subtitle timing preview with adjustable offset.
  • Animated screenshots (see example card gifs).
Built-in themes (29)
  • Ayu - Light, Mirage, Dark
  • Catppuccin - Latte (light); Frappé, Macchiato, Mocha (dark)
  • Dracula - Dracula, Alucard
  • Everforest - Light, Dark
  • GitHub - Light; Dark, Dark Dimmed
  • Gruvbox - Light Medium, Dark Medium
  • Kanagawa - Lotus (light), Wave (dark)
  • Rosé Pine - Dawn (light); Main, Moon (dark)
  • Solarized - Light, Dark
  • Standalone - Light, Dark, Sakura, Nord, One Dark, Tokyo Night

Theme licenses: LICENSE-THEMES.md. Want another theme added? Suggest in a GitHub Issue.

How It Works
  1. Read the subtitles and split Japanese into individual words.
  2. Filter to content words you don't already know.
  3. Grab a screenshot and audio clip from the video for each line.
  4. Look up definitions in your configured offline dictionaries, optionally falling back to Jisho online if enabled (slower, rate-limited).
  5. Send the finished cards to Anki.

Recommended Resources

Type Resource What you get Download Add via
Dictionary Jitendex JMdict successor; structured formatting, examples, tags Yomitan zip Add Dictionary…
Dictionary JMdict Plain glosses; smaller, faster to index Yomitan zip Add Dictionary…
Dictionary Bee's Character Dictionary Character names from your AniList/VNDB lists, with roles and descriptions Generated on site Add Dictionary…
Pitch Kanjium ~124k patterns; drop-in TSV, no import step TSV Dictionary -> Pitch Accent File
Pitch アクセント辞典v2 Richer NHK notation Drive Dictionary -> Pitch Accent File
Frequency JPDB v2.2 Kana All-round default for media Yomitan zip Filtering -> Frequency List File
Frequency BCCWJ SUW+LUW Balanced corpus; pairs well with news/novels Yomitan zip Filtering -> Frequency List File

Dictionaries are indexed once into ~/.anki_miner/dicts/ (drag to reorder the chain). The pitch and frequency pickers accept a raw CSV/TSV or a Yomitan zip, auto-converted to ~/.anki_miner/pitch_accent.csv / frequency.csv on Save. Bee's Character Dictionary builds a custom Yomitan dictionary from your AniList/VNDB media lists, so character names in the shows you mine resolve to real definitions; re-generate and re-import when your lists change.

Proper-noun filtering uses bundled name wordsets derived from JMnedict (JMdict/EDICT project, EDRDG, CC BY-SA 4.0).

Troubleshooting

Issue Solution
"Cannot connect to Anki" Start Anki and ensure AnkiConnect is installed.
"Deck not found" The deck is created automatically when mining starts; if you meant a different deck, update the name in Settings.
"Note type not found" Configure your note type's field names in Settings -> Anki.
"ffmpeg not found" Install ffmpeg and add it to PATH.
No definitions found Add a Yomitan dictionary in Settings -> Add Dictionary… (recommended), or enable the Jisho fallback (slower, rate-limited).
Audio is wrong language The tool tries Japanese audio tracks first, then falls back to the default.
Subtitles out of sync Use the subtitle offset control in the GUI (range ±300 seconds).
AV1 won't preview In-app AV1 preview needs a hardware AV1 decoder (RTX-30+/Tiger-Lake+). Without one, the pane shows an "AV1 can't be decoded for preview" notice. Mining is unaffected - screenshots are extracted by FFmpeg, not the preview.

Roadmap

List of ideas for future versions of Anki Miner. Not in priority order. Feature requests take precedence.

  • Suggest a feature - Open an issue.

  • Discuss the roadmap - Discussions.

  • Features:

    • UI language selection.
    • Local subtitle creation tab: Opt-in tab to locally generate subtitles.
    • Reading tab: Mine manga and books.
    • Media library: Expand Analytics tab to display local media library across all media forms.
  • Improvements:

    • Improved user onboarding: automatic recommended resource fetching, easier setup.
  • Researching/Under consideration:

    • Android port.
    • Jellyfin integration.
    • Jimaku integration.
  • Long-term:

    • Beyond Japanese: Mining other languages.

Contributing

Contributions of any kind are welcome. If you want to support the project, please share it with others who may benefit from it.

Bug reports and feature requests -> Issues. General questions and discussion -> Discussions or Discord.

Special Thanks

Sincere thanks to people who made exceptional contributions to the project:

StyraxBenzoin - Brilliant feature suggestions, new release testing, community building

See CONTRIBUTORS.md for everyone who has made any kind of contribution to the project.

License

GNU General Public License v3.0. See LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

anki_miner-2.8.1.tar.gz (4.0 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

anki_miner-2.8.1-py3-none-any.whl (4.1 MB view details)

Uploaded Python 3

File details

Details for the file anki_miner-2.8.1.tar.gz.

File metadata

  • Download URL: anki_miner-2.8.1.tar.gz
  • Upload date:
  • Size: 4.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for anki_miner-2.8.1.tar.gz
Algorithm Hash digest
SHA256 a52832a4ab476bd894d43900c12f98102b9b17f16f617a645f6c49a35de54067
MD5 fe938d19549c4e0a5ec697ecece8600a
BLAKE2b-256 5dacb331998e426b1013da57a6e78e38b1d4c1a84727773e9829e0c3443d6388

See more details on using hashes here.

Provenance

The following attestation bundles were made for anki_miner-2.8.1.tar.gz:

Publisher: publish.yml on 0xzerolight/anki_miner

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file anki_miner-2.8.1-py3-none-any.whl.

File metadata

  • Download URL: anki_miner-2.8.1-py3-none-any.whl
  • Upload date:
  • Size: 4.1 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for anki_miner-2.8.1-py3-none-any.whl
Algorithm Hash digest
SHA256 f0265de00bb4e34690bdf0222e8ac5cf1b41135bbd63e5e970a9f2e2417b3949
MD5 cebd90a4685b3455778cf613491cc234
BLAKE2b-256 67674e92fdb639c0356167a7ff978a5833a822d20b53bd2bde7f434931211829

See more details on using hashes here.

Provenance

The following attestation bundles were made for anki_miner-2.8.1-py3-none-any.whl:

Publisher: publish.yml on 0xzerolight/anki_miner

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page