six2one 0.2.0

Manifest-backed e621/e926 dataset fetching from tiny numeric commands.

six2one

Quick Start • Queries • Manifest • Commands

six2one is a small command-line fetcher for e621 and e926. Pass it the same tags you'd type into the site search; it compiles them into a query, downloads matching posts in API-friendly pages, and stores post JSON plus images in a manifest-backed local cache. The manifest.json records where each post's metadata and image variants live, so you can resume, dedupe, repair, inspect, and prune without managing state yourself.

The result is a CLI that stays pleasant for one-off searches and trustworthy for long-running collections: reference folders for a particular artist, archives of a saved search, the occasional small dataset.

Quick Start

Install six2one from PyPI:

$ python -m pip install six2one

That installs the 621 and 926 commands onto your PATH.

The following command searches e621 for fox solo rating:s and begins to download the posts returned:

$ 621 fox solo --safe

This will download into output/fox-solo-safe/ with the following layout:

output/fox-solo-safe/
  json/
  images/
    preview/
    sample/
    original/
  manifest.json

Alternatively, for the safe-only sister site, pass --site e926, or use the 926 binary, which applies that site default automatically:

$ 926 fox solo --safe

You may also use --dry-run to see the compiled query before downloading anything:

$ 621 fox solo --safe --dry-run
Compiled query: fox solo rating:s

For an isolated CLI install, use pipx install six2one. For an editable install from a local clone, use python -m pip install --user -e .. If 621 installs but your shell cannot find it, run python -m site --user-base; the binary lives under that path's bin/ directory, or Scripts\ on Windows.

How Queries Compile

Tags pass through to e621 unchanged. Artist tags, OR terms, exclusions, and rating get appended in a fixed order, and --dry-run shows exactly what comes out:

$ 621 fox solo \
    --author some_artist \
    --any cat,dog \
    --exclude chicken,watermark,comic \
    --safe \
    --dry-run

# Compiled query: fox solo some_artist ~cat ~dog -chicken -watermark -comic rating:s

That compiled string is what you'd type into the e621 search bar, which is also why six2one's parser stays thin. The flags are conveniences for patterns six2one already knows; everything else, including wildcards (cat*), single-dash negation (-comic), and grouped OR syntax (( ~cat ~dog )), passes through to e621's native search syntax. Unknown long options stay CLI errors rather than being silently forwarded, but anything that works in the search bar works here:

$ 621 "( ~cat ~tiger ~leopard ) ( ~dog ~wolf )" --safe
$ 621 fox african_wild_dog -chicken

To fetch more than the 320-post default, raise --limit or use --all. The downloader continues in API-sized pages until it reaches the requested count or the query is exhausted:

$ 621 fox solo --safe --limit 1000
$ 621 dragon solo --explicit --all --resume

The Manifest

Every output folder gets a manifest.json that records downloaded posts by numeric post ID, with relative paths to the post JSON and any cached image variants. It also records each compiled query, the site it came from, and the cursor needed to continue that search later. That record keeps the folder understandable across sessions: you can stop a fetch, come back later, and continue without reasoning from whatever happens to be on disk.

The same record also controls how six2one handles folders it has already touched. Existing manifests are reused as a cache across searches. If a post is already recorded and the requested image size exists, six2one skips the download. If the manifest says a JSON file or image should exist but it is missing, the next fetch repairs that entry before continuing.

$ 621 fox solo --safe              # use cached posts, then fetch up to the limit
$ 621 fox solo --safe --resume     # fetch the next limit's worth of posts
$ 621 dragon solo --explicit --all --resume

The continuation rules follow from that cache:

Situation	Behavior
Same query without --resume	Start the search from the latest results and stop at the requested limit, skipping cached posts.
Same query with --resume	Continue after the previous cursor and fetch the next requested count, or to the end with --all.
Different query in the same output	Add a new query state and reuse any already cached posts.
Same post, different image size	Download only the missing size and record it under the same post entry.
Manifest-listed files are missing	Refetch post JSON or image files to heal the cache.
Files exist that aren't in the manifest	Leave them alone; the manifest is the cache index.

You do not edit the manifest directly. It is deliberately path-oriented rather than a second copy of the post metadata; the canonical post data lives in json/<id>.json.

What's in the Output Folder

The layout is stable:

json/                 raw post JSON record per post
images/preview/       preview images
images/sample/        sample images
images/original/      original files
manifest.json

Post JSON is stored once per post, and image variants are stored by size. That keeps the output folder useful after the download is over: grep the JSON, re-derive captions from tags, rebuild a downstream index, or use show to merge the manifest entry, raw post JSON, and local file paths into one view of a post.

The --size flag controls which image variant to fetch, mapping directly to e621 and e926 post fields:

Size	e621/e926 post field
preview	post["preview"]
sample	post["sample"] (default)
original	post["file"]

If the chosen URL is missing for a post, six2one records a warning and still keeps any post JSON it was able to cache.

Commands

usage: 621 [fetch] [TAGS ...] [options]
       621 show POST_ID... [options]
       621 prune [output_dir]

fetch is the default subcommand, so 621 fox solo --safe and 621 fetch fox solo --safe are equivalent.

Login

$ 621 login nollafox YOUR_E621_API_KEY
$ 621 logout

login writes .six2one-login.json next to the installed or source six2one project root, and logout removes it. The location is tied to where six2one itself lives on disk, not the directory where you run 621. When present, those credentials are used for HTTP Basic auth and a username-specific User-Agent on API requests.

That login flow is one part of being a well-behaved e621 client. The other part is pacing: six2one uses an instance-owned rate limiter capped at 2 requests per second, and it pages through search results in chunks of at most 320 posts per API call, which is the e621 post-search maximum. Even --all fetches use that same pacing and pagination.

For the safe-only sister site, pass --site e926 to any fetch or show command, or use the 926 binary, which applies that site default automatically:

$ 621 fox solo --site e926 --all
$ 926 fox solo --all
$ 926 show 6394158 --fetch  # fetch from e926 if available there

Fetch

$ 621 [fetch] TAGS... [options]

Option	Meaning
TAGS	e621 tag query terms. Supports -tag, ~tag, wildcards, and grouped OR syntax.
-o, --out DIR	Output directory. Default: ./output/<query-slug>.
-n, --limit N	Number of posts to fetch. Default: 320.
--all	Fetch until the query is exhausted.
--safe / --questionable / --explicit	Shortcuts for --rating.
--rating RATING	One of safe, questionable, explicit, s, q, or e.
--author NAME, --artist NAME, --by NAME	Add an artist tag. Repeatable.
--any TAG	Add OR terms as ~TAG. Repeatable and comma-separated.
-x, --exclude TAG	Exclude tags as -TAG. Repeatable and comma-separated.
--site SITE	e621 or e926. Default: e621 for 621, e926 for 926.
--size MODE	preview, sample, or original. Default: sample.
--resume	Continue this query after its previous cursor, fetching the next requested count.
--dry-run	Print the compiled query and exit.
--validate-tags	Check concrete and wildcard tags against the tag API before fetching.

Hidden compatibility aliases still work: --continue for --resume, --or for --any, and --file for --size.

Show metadata

$ 621 show 6394158 

show searches recursively under ./output for six2one manifest.json files and returns a merged JSON view for matching posts. The metadata command is an alias. Each result combines the manifest post entry, json/<id>.json when present, a derived caption built from post tags, and filesystem-derived paths, existence flags, and file sizes.

By default, show reads only what is already on disk. Use --root to search a narrower folder, --all to show every manifest entry under that root, and --fetch to fetch remote post JSON only when an ID is missing locally:

$ 621 show --all --root output/fox-solo-safe

# fetch from e926 if available there
$ 926 show 6394158 --fetch  

The merged object is filterable with dotted paths. Repeated filters and comma-separated filters are both accepted:

$ 621 show 6394158 \
  --filter local.image.absolute_path \
  --filter caption.text \
  --filter post.file.url

$ 621 show 6394158 -f local.image.absolute_path,caption.text,post.file.url

Filtered keys flatten to their unique leaf names, so caption.text becomes text. If leaf names collide, six2one keeps enough path context to avoid overwriting values. Without filters, the full merged object is printed.

$ 621 show 6394158 -f local.image.absolute_path,caption.text,post.file.url --pretty

# {
#   "results": [
#     {
#       "absolute_path": "/Users/nollafox/output/fox-solo-safe/images/sample/000006394158.jpg",
#       "text": "anthro, ... english_text, hi_res, signature",
#       "url": "https://static1.e621.net/data/51/b5/51b5a2f0925e153c2890e37836024f77.jpg"
#     }
#   ]
# }

That makes it easy to export a local dataset index for scripts, audits, or LoRA training prep.

Prune

$ 621 prune [output_dir]

Prune scans the output folder for manifest entries whose recorded JSON or image files are incomplete. If a cached file is deleted by hand, 621 prune removes the remaining recorded companions for that post and updates the manifest so a later fetch can repair the cache cleanly. If the output directory does not exist yet, prune creates it and exits.

Development

Install with Poetry:

$ poetry install

Run the CLI, the tests, and the compile check from inside the Poetry environment:

$ poetry run 621 --help
$ poetry run 926 fox solo --dry-run
$ poetry run 621 show --help
$ poetry run python -m unittest discover -s tests
$ poetry run python -m compileall -q six2one

---

six2one - manifest-backed e621 and e926 fetching.

Crafted by Nolla Fox