Skip to main content

Async local/remote (SSH) file ops in Rust with throughput tuning and Python bindings

Project description

async_server_ops 0.2.0 (tuned)

Async, cross-implementation file operations with throughput tuning. Local backend uses tokio::fs. Remote backend shells out to ssh/scp/rsync via tokio::process::Command.

New in 0.2 (tuning)

  • RemoteConfig.ssh_extra_args: Vec<String> — inject flags like -o IPQoS=throughput -c aes128-gcm@openssh.com -o ControlMaster=auto -o ControlPersist=60s
  • RemoteConfig.prefer_rsync: bool — use rsync -a for uploads/downloads (better for big/recursive or repeated transfers)
  • RemoteConfig.compression: Option<String>"zstd" (rsync >= 3.2) or "zlib"; omit for already-compressed data
  • ServerManager::upload_files_concurrent(pairs, concurrency) — accelerate many small files

Build (Rust)

cargo build

Build (Python via maturin)

pip install maturin
maturin develop -F python
# or
maturin build -F python --release

Python example

import asyncio
from async_server_ops import ServerManagerPy as ServerManager, PyRemoteConfig as RemoteConfig

async def main():
    cfg = RemoteConfig(
        host="example.com",
        user="ubuntu",
        key_path="~/.ssh/id_rsa",
        port=22,
        ssh_extra_args=["-o","IPQoS=throughput","-c","aes128-gcm@openssh.com","-o","ControlMaster=auto","-o","ControlPersist=60s"],
        prefer_rsync=True,
        compression="zstd",  # or "zlib" or None
    )
    remote = ServerManager.remote(cfg)
    await remote.create_dir_all("/tmp/bench")
    await remote.upload_file("big.bin", "/tmp/bench/big.bin")
    print("exists:", await remote.exists("/tmp/bench/big.bin"))
    # batch small files
    await remote.upload_files_concurrent([("a.txt","/tmp/bench/a.txt"),("b.txt","/tmp/bench/b.txt")], concurrency=8)

asyncio.run(main())

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

async_server_ops-0.2.2-cp38-abi3-win_amd64.whl (477.1 kB view details)

Uploaded CPython 3.8+Windows x86-64

File details

Details for the file async_server_ops-0.2.2-cp38-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for async_server_ops-0.2.2-cp38-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 3caffbd99ed634a501c8ea5403a29910626fa97fd7fce0af33a8011de7954186
MD5 8ffe1b19ff339e491ed1645f7ba85340
BLAKE2b-256 56a396814118283baf28ba30411088ea6f475672d78f358fb12aa2d41cbf557d

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page