Assorted tar related things, including a fast tar-based copy.

Latest release 20260531: Update because an IterableQueue no longer has a .open() method.

My most heavily used use for this is my cpdir script which does a high performance directory copy by piping 2 tars together. It runs this:

from cs.tarutils import traced_cpdir
sys.exit(traced_cpdir(*sys.argv[1:]))

Short summary:

• tar: Tar up the contents of srcpaths to output. Return the Popen object for the tar command.
• traced_cpdir: Copy a directory to a new place using piped tars with progress reporting. Return 0 if both tars succeed, nonzero otherwise.
• traced_untar: Read tar data from tarfd and extract. Return the tar exit code.

Module contents:

• tar(*srcpaths: List[str], chdirpath='.', output, tar_exe='tar', bcount=2048): Tar up the contents of srcpaths to output. Return the Popen object for the tar command.

  Parameters:

  • srcpaths: source filesystem paths
  • chdirpath: optional directory to which to chdir before accessing srcpaths
  • tar_exe: optional tar executable, default from TAR_EXE: tar
  • bcount: blocking factor in 512 byte unites, default from DEFAULT_BCOUNT: 2048

• traced_cpdir(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x10b0b7920>, **kw): Copy a directory to a new place using piped tars with progress reporting. Return 0 if both tars succeed, nonzero otherwise.

  Parameters:

  • srcdirpath: the source directory filesystem path
  • dstdirpath: the destination directory filesystem path, which must not already exist
  • label: optional label for the progress bar
  • tar_exe: optional tar executable, default from TAR_EXE: tar
  • bcount: blocking factor in 512 byte unites, default from DEFAULT_BCOUNT: 2048

• traced_untar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x10b0b7560>, **kw): Read tar data from tarfd and extract. Return the tar exit code.

  Parameters:

  • tarfd: the source tar data, suitable for subprocess.Popen's stdin parameter
  • chdirpath: optional directory to which to chdir before accessing srcpaths
  • label: optional label for the progress bar
  • tar_exe: optional tar executable, default from TAR_EXE: tar
  • bcount: blocking factor in 512 byte unites, default from DEFAULT_BCOUNT: 2048

Release Log

Release 20260531: Update because an IterableQueue no longer has a .open() method.

Release 20240318: Initial PyPI release with nice traced_cpdir() function.