Storix

Storage for Unix lovers.

One unix-flavored filesystem API over any storage — local disk, in-memory,
Azure Data Lake — sync & async, sandboxable, fully typed.

0.2.0 is a ground-up rework (hexagonal core, generated sync flavor, layers, capabilities). Migrating from 0.1.x? See the table in release-notes.md. Full documentation site: coming soon.

Install

uv add storix            # local filesystem + in-memory
uv add "storix[azure]"   # + Azure Data Lake Gen2 (HNS accounts)

Five minutes of storix

from storix import Storix
from storix.backends import LocalBackend

fs = Storix(LocalBackend('~/data'))     # '/' is anchored at ~/data

fs.mkdir('/docs')
fs.echo(b'hello, storix!', '/docs/readme.txt')
print(fs.cat('/docs/readme.txt'))       # b'hello, storix!'

fs.cd('/docs')                          # sessions have a cwd, like a shell
fs.touch('a.txt', 'b.txt')              # variadic, like real touch
print(fs.ls())                          # dotfiles hidden, like real ls
fs.mkdir('/archive')
fs.mv('a.txt', 'b.txt', '/archive')     # last argument is the destination
fs.rm('/archive', recursive=True)       # rm -r; rmdir is strictly empty dirs

Async is the same API under storix.aio - every operation awaitable:

from storix.aio import Storix
from storix.aio.backends import AzureBackend

async with Storix(AzureBackend('raw', account_name=..., credential=...)) as fs:
    await fs.echo(b'...', '/report.csv')
    print(await fs.url('/report.csv', expires_in=600))   # SAS link

(The sync flavor is generated from the async source - identical semantics, verified by one conformance suite running against every backend in both flavors.)

Configuration

get_storage() builds a session from the environment (see env.example):

STORIX_PROVIDER=azure
STORIX_AZURE_CONTAINER=raw
STORIX_AZURE_ACCOUNT_NAME=myaccount
STORIX_AZURE_CREDENTIAL=...

from storix import get_storage

fs = get_storage()                        # env-driven; defaults to ~/.storix
fs = get_storage('local', base='~/x')     # typed per-provider overrides

Sandboxing & scratch spaces

Layers are backends that wrap backends. SandboxLayer is chroot as middleware - escape-proof, with errors re-scoped so the real prefix never leaks to the sandboxed caller:

from storix import SandboxLayer, Storix, temporary
from storix.backends import LocalBackend

backend = LocalBackend('/srv/data')
fs = Storix(SandboxLayer(backend, root='/tenant-42'))   # escape-proof jail

with fs.scratch() as tmp:               # ephemeral workspace on fs's OWN
    tmp.echo(b'work', '/notes.txt')     # backend (any backend) - unique
                                        # subtree, deleted on exit

with fs.scratch(root='/agent-7') as tmp:  # pinned: created if missing,
    ...                                   # reused, PERSISTS on exit

with temporary() as fs:                 # local-only convenience: zero-config
    fs.echo(b'scratch', '/tmp.txt')     # mkdtemp on real disk, self-destructs

Write your own layers by subclassing LayerBase (override what you change, upgrade the capabilities you add) — see samples/layers/.

Capabilities

Backends advertise optional features; storix fails loudly instead of silently dropping arguments:

fs.echo(b'x', '/f.png', content_type='image/png')       # azure: stored
fs.set_metadata('/f.png', {'owner': 'me'}, merge=True)  # azure/memory
fs.url('/f.png', expires_in=600)                        # presigned (SAS)
fs.data_url('/f.png')                                   # any backend
# unsupported -> UnsupportedOperationError naming the missing capability

Portable capabilities via layers. Bundled layers backfill missing capabilities so one construction path works across providers. with_layer_missing infers the capability from the layer and skips it where the backend is already native:

from storix import DataUrlLayer, MetadataLayer

# url() everywhere: native SAS on azure, data: URLs on local
fs = get_storage('local').with_layer_missing(DataUrlLayer)
# custom metadata everywhere: native on azure, JSON sidecar on local
fs = fs.with_layer_missing(MetadataLayer)
# with_layer forwards typed kwargs to the layer, Starlette-style
# (serialize/deserialize are object<->bytes; orjson works, json.dumps
# does not - it returns str):
fs = fs.with_layer(MetadataLayer, serialize=orjson.dumps, deserialize=orjson.loads)

Switching 'local' to 'azure' needs no code change — the native capability wins and the layer becomes a no-op.

Caching

CacheLayer is a read-through cache — another layer. Cache the ops that repeat: metadata (stat/ls/exists, on by default), the expensive du walk, file reads, and presigned urls. Each op is True or a cache(...) spec; every write through the layer evicts what it touches.

from storix import CacheLayer, cache, get_storage

fs = get_storage('azure').with_layer(
    CacheLayer,
    du=cache(ttl=60),                 # opt in; du (the tree walk) is the big win
    read=cache(max_bytes=8 << 20),    # content, capped per file
)
fs.du('/big-tree')                    # first call: a full walk
fs.du('/big-tree')                    # then: instant, until you write

The store is pluggable — a cashews-shaped get/set/delete/delete_match protocol — so the in-memory default swaps for a cashews.Cache (Redis, disk) with no adapter. Keys are namespaced and keyed on the physical location, so sessions sharing one store never collide. Correctness assumes you are the only writer; pass ttl to bound staleness.

In the sx shell, one flag turns it on:

sx -p azure --cache                        # cache on: du/ls/stat/cat
sx -p azure --cache --sandbox /tenant-42   # compose with a jail
# in the REPL: `refresh` clears it; `provider` shows the stack + store

Backends

Third-party backends implement the ~13-method StorageBackend port (or subclass BackendBase for generic fallbacks) and hook in via register_backend().

License

Storix is licensed under the Apache 2.0 License