Skip to main content

Compressed File Sequence String Module

Project description

PySeq

PySeq is a python module that finds groups of items that follow a naming convention containing a numerical sequence index (e.g. fileA.001.png, fileA.002.png, fileA.003.png...) and serializes them into a compressed sequence string representing the entire sequence (e.g. fileA.1-3.png). It should work regardless of where the numerical sequence index is embedded in the name. For examples, see basic usage below or http://rsgalloway.github.io/pyseq

Installation | Basic Usage | API Examples | Formatting | Command-Line Tools | Frame Patterns | Testing

Installation

The easiest way to install pyseq:

$ pip install -U pyseq

Environment

PySeq uses envstack to externalize settings and looks for a pyseq.env file to source environment variables:

$ pip install -U envstack
$ ./pyseq.env -r
PYSEQ_FRAME_PATTERN=\d+
PYSEQ_GLOBAL_FORMAT=%4l %h%p%t %R
PYSEQ_RANGE_SEP=, 
PYSEQ_STRICT_PAD=0

Distribution

If installing from source you can use distman to install PySeq using the provided dist.json file:

$ pip install -U distman
$ distman [-d]

Using distman will deploy the targets defined in the dist.json file to the root folder defined by ${DEPLOY_ROOT}:

Basic Usage

Using the "z1" file sequence example in the "tests" directory, we start by listing the directory contents using ls:

$ ls tests/files/z1*
tests/files/z1_001_v1.1.png  tests/files/z1_001_v1.4.png  tests/files/z1_002_v1.3.png   tests/files/z1_002_v2.11.png
tests/files/z1_001_v1.2.png  tests/files/z1_002_v1.1.png  tests/files/z1_002_v1.4.png   tests/files/z1_002_v2.12.png
tests/files/z1_001_v1.3.png  tests/files/z1_002_v1.2.png  tests/files/z1_002_v2.10.png  tests/files/z1_002_v2.9.png

Now we list the same directory contents using lss, which will find the sequences and display them in the default compressed format:

$ lss tests/files/z1*
   4 z1_001_v1.%d.png [1-4]
   4 z1_002_v1.%d.png [1-4]
   4 z1_002_v2.%d.png [9-12]

Recursivly walk a folder and find all the sequences:

$ lss -r tests
tests
├── test_pyseq.py
└── files
    ├── 012_vb_110_v001.1-10.png
    ├── 012_vb_110_v002.1-10.png
    ├── a.1-14.tga
    ├── alpha.txt
    ├── bnc01_TinkSO_tx_0_ty_0.101-105.tif
    ├── bnc01_TinkSO_tx_0_ty_1.101-105.tif
    ├── bnc01_TinkSO_tx_1_ty_0.101-105.tif
    ├── bnc01_TinkSO_tx_1_ty_1.101-105.tif
    ├── file.1-99.tif
    ├── file.info.03.rgb
    ├── file01.1-4.j2k
    ├── file01_40-43.rgb
    ├── file02_44-47.rgb
    ├── file1-4.03.rgb
    ├── fileA.1-3.jpg
    ├── fileA.1-3.png
    ├── file_02.tif
    ├── z1_001_v1.1-4.png
    ├── z1_002_v1.1-4.png
    └── z1_002_v2.9-12.png

Piping the output of find to lss, for example finding all the png sequences:

$ find ./tests/ -name *.png | lss
  10 012_vb_110_v001.%04d.png [1-10]
  10 012_vb_110_v002.%04d.png [1-10]
   3 fileA.%04d.png [1-3]
   4 z1_001_v1.%d.png [1-4]
   4 z1_002_v1.%d.png [1-4]
   4 z1_002_v2.%d.png [9-12]

Use the --format option to retain the relative path:

$ find tests/ -name "*.png" | lss -f "%D%h%r%t"
tests/files/012_vb_110_v001.1-10.png
tests/files/012_vb_110_v002.1-10.png
tests/files/fileA.1-3.png
tests/files/z1_001_v1.1-4.png
tests/files/z1_002_v1.1-4.png
tests/files/z1_002_v2.9-12.png

API Examples

Compression, or serialization, of lists of items:

>>> s = Sequence(['file.0001.jpg', 'file.0002.jpg', 'file.0003.jpg'])
>>> print(s)
file.1-3.jpg
>>> s.append('file.0006.jpg')
>>> print(s.format("%h%p%t %R"))
file.%04d.jpg [1-3, 6]

Uncompression, or deserialization, of compressed sequences strings:

>>> s = uncompress('./tests/012_vb_110_v001.%04d.png 1-1001', fmt='%h%p%t %r')
>>> len(s)
1001
>>> print(s.format('%04l %h%p%t %R'))
1001 012_vb_110_v001.%04d.png [1-1001]

Walk a directory tree and print disk usage for file sequences:

>>> for root, dirs, seqs in pyseq.walk(folder):
...     for seq in seqs:
...         print(seq.format("%h%r%t %H"))
012_vb_110_v001.1000-1321.exr   123.5G
012_vb_110_v002.1000-1163.exr    40.2G
012_vb_110_v003.1000-1027.exr    72.2G

Formatting

The following directives can be embedded in the format string.

Directive Meaning
%s sequence start
%e sequence end
%l sequence length
%f list of found files
%m list of missing files
%M explicit missing files [11-14,19-21]
%p padding, e.g. %06d
%r implied range, start-end
%R explicit broken range, [1-10, 15-20]
%d disk usage
%H disk usage (human readable)
%D parent directory
%h string preceding sequence number
%t string after the sequence number

Here are some examples using lss -f <format> and seq.format(..):

Using lss -f <format>:

$ lss tests/files/a*.tga -f "%h%r%t"
a.1-14.tga
$ lss tests/files/a*.tga -f "%l %h%r%t"
7 a.1-14.tga
$ lss tests/files/a*.tga -f "%l %h%r%t %M"
7 a.1-14.tga [4-9, 11]

In Python, using seq.format(..):

>>> s = pyseq.get_sequences("tests/files/a*.tga")[0]
>>> print(s.format("%h%r%t"))
a.1-14.tga
>>> print(s.format("%l %h%r%t"))
7 a.1-14.tga
>>> print(s.format("%l %h%r%t %M"))
7 a.1-14.tga [4-9, 11]

Command-Line Tools

PySeq comes with the following sequence-aware command-line tools:

Command Description Example Usage
lss List image sequences in a directory lss shots/
stree Display sequence-aware directory tree stree shots/
sfind Recursively find image sequences sfind assets/ -name "*.exr"
sdiff Compare two sequences sdiff A.%04d.exr B.%04d.exr
sstat Print detailed stats about a sequence sstat render.%04d.exr
scopy Copy a sequence to another directory scopy a.%04d.exr /tmp/output/
smove Move a sequence to another directory smove b.%04d.exr /tmp/archive/

Example commands:

# List sequences in a folder
$ lss tests/files

# Show directory structure with grouped sequences
$ stree tests/

# Find all .png sequences recursively
$ sfind ./tests -name "*.png"

# Compare two sequences and print diffs
$ sdiff comp_A.%04d.exr comp_B.%04d.exr

# Show stats for a sequence
$ sstat render.%04d.exr
$ sstat --json render.%04d.exr

# Copy a sequence and rename it
$ scopy input.%04d.exr output/ --rename scene01

# Move and renumber a sequence starting at frame 1001
$ smove old.%04d.exr archive/ --renumber 1001

Frame Patterns

The environment var ${PYSEQ_FRAME_PATTERN} can be used to define custom regex patterns for identifying frame numbers. For example if frames are always preceded with an _, you might use:

$ export PYSEQ_FRAME_PATTERN="_\d+"

Environment vars can be defined anywhere in your environment, or if using envstack add it to the pyseq.env file and make sure it's found in ${ENVPATH}:

$ export ENVPATH=/path/to/env/files

Examples of regex patterns can be found in the pyseq.env file:

# matches all numbers, the most flexible
PYSEQ_FRAME_PATTERN: \d+

# excludes version numbers, e.g. file_v001.1001.exr
PYSEQ_FRAME_PATTERN: ([^v\d])\d+

# frame numbers are dot-delimited, e.g. file.v1.1001.exr
PYSEQ_FRAME_PATTERN: \.\d+\.

# frame numbers start with an underscore, e.g. file_v1_1001.exr
PYSEQ_FRAME_PATTERN: _\d+

Testing

To run the unit tests, simply run pytest in a shell:

$ pytest tests/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyseq-0.9.0.tar.gz (27.6 kB view details)

Uploaded Source

File details

Details for the file pyseq-0.9.0.tar.gz.

File metadata

  • Download URL: pyseq-0.9.0.tar.gz
  • Upload date:
  • Size: 27.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/51.3.3 requests-toolbelt/0.9.1 tqdm/4.57.0 CPython/3.8.10

File hashes

Hashes for pyseq-0.9.0.tar.gz
Algorithm Hash digest
SHA256 44bc558267c70c591188c3e79c4e1d601c24bc7341508d205b3efe117383ae2a
MD5 2095fbaed4bf78fcdb62628615bf0cb2
BLAKE2b-256 a575b1c648f218c6e68517ba2af5ebb6a7ce71e4b98013acf648d5fb32eda6d0

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page