Skip to main content

aind-behavior-video-transformation

License Code Style semantic-release: angular Interrogate Coverage Python

This project is used to encode behavior videos and, depending on the user request, will compress those videos while ensuring that they are formatted for correct display across many devices. This may include common image preprocessing steps, such as gamma encoding, that are necessary for correct display, but have to be done post-hoc for our behavior videos.

Goals

This will attempt to compress videos so that the results:

  • Retain the majority of the detail of the input video
  • Take up as little space as possible for a target amount of visual detail
  • Are in a format that can be widely viewed across devices, and streamed by browsers
  • Have pixel data in a color space that most players can properly display

This video compression is often lossy, and the original videos are not kept, so this library will attempt to produce the highest-quality video for a target compression ratio. The speed of this compression is strictly secondary to the quality of the compression, as measured by the visual detail retained and the compression ratio. See this section for more details.

Additionally, this package should provide an easy to use interface that:

  • Presents users with a curated set of compression settings, which have been rigorously tested in terms of their visual quality using perception-based metrics like VMAF.
  • Allow users to also provide their own compression settings, if they have specific requirements

Non-goals

  • Sacrifice the visual fidelity of videos in order to decrease encoding time.

Brief benchmarks on video compression with CPU-based encoders and GPU-based encoders

A surprising fact is that video encoders implementing the same algorithm, but written for different compute resources do not have the same visual performance; for a given compression ratio, or similar settings, they do not retain the same amount of visual detail. This is also true for different presets of the same encoder and compute resource even if the other settings are identical. For example, the presets -preset fast and -preset veryslow of the encoder libx264 produce videos with the same compression ratio, but differing visual quality.

This can be seen in the plot below, where the GPU encoder and CPU encoders retain different amounts of visual detail, as assessed with visual perception-based metric VMAF. Also note the difference between presets for the same encoder and compute resource: CPU Fast and CPU Slow.

visual performance vs compress ratio

This figure shows that for compression ratios greater than 100, it often makes sense to take your time and use a slow preset of a CPU-based encoder to retain as much visual information for a given amount of compression.

While it may be tempting to select a faster preset, or faster compute resource like GPU for dramatic speedups shown below, doing will degrade the quality of the resulting video.

throughput vs compress ratio

Because the output of this package are permanent video artifacts, the compression is lossy, and the intent is to delete the original, taking the CPU time to produce the highest quality video possible might well be worth it.

Usage

  • The BehaviorVideoJob.run_job method in the transform_videos should be the primary method to call for processing video files.
  • On a merge to main, this package will be published as a docker image, which can easily be run on a SLURM cluster.

Running across multiple nodes

A job can be split across a SLURM array. Every task runs the same command and discovers the same list of files independently, then acts only on the share identified by its own partition_number, so no coordination between tasks is needed.

sbatch --array=1-4 --cpus-per-task=8 --wrap='
  python -m aind_behavior_video_transformation.etl --job-settings "{
    \"input_source\": \"/mnt/input_source\",
    \"output_directory\": \"/mnt/output_directory\",
    \"partition_number\": $SLURM_ARRAY_TASK_ID,
    \"num_partitions\": 4,
    \"ffmpeg_thread_cnt\": 8
  }"'

Notes:

  • partition_number is 1-based, so use --array=1-N, not --array=0-(N-1).
  • num_partitions must match the array size and be identical on every task.
  • Both default to 1, which processes every video in a single task.
  • Set ffmpeg_thread_cnt to match --cpus-per-task. It defaults to 0, which lets ffmpeg use every core on the physical node regardless of the allocation.

Videos and the non-video files that are symlinked alongside them are distributed separately, so each task receives a comparable share of the transcoding work rather than one task inheriting every video.

Deprecated settings

parallel_compression is deprecated. It is still accepted, so existing job settings continue to validate, but it no longer has any effect and will be removed in a future release. Compression now runs sequentially within a task, and parallelism comes from the scheduler running one task per partition. Use num_partitions and partition_number instead.

Docker build for local testing

In the same directory as the Dockerfile, run

docker build -t aind-behavior-video-transformation-local .

Now a docker container can be run. You may need to change the mount locations and file permissions:

docker run -v /home/local_videos/input_source:/mnt/input_source -v /home/local_videos/output_directory:/mnt/output_directory aind-behavior-video-transformation-local python -m aind_behavior_video_transformation.etl --job-settings '{"compression_requested": {"compression_enum": "gamma fix colorspace"}, "input_source": "/mnt/input_source", "output_directory": "/mnt/output_directory"}'

Development

To develop the code, run

pip install -e .[dev]

Contributing

Linters and testing

There are several libraries used to run linters, check documentation, and run tests.

  • Please test your changes using the coverage library, which will run the tests and log a coverage report:
coverage run -m unittest discover && coverage report
  • Use interrogate to check that modules, methods, etc. have been documented thoroughly:
interrogate .
  • Use flake8 to check that code is up to standards (no unused imports, etc.):
flake8 .
  • Use black to automatically format the code into PEP standards:
black .
  • Use isort to automatically sort import statements:
isort .

Pull requests

For internal members, please create a branch. For external members, please fork the repository and open a pull request from the fork. We'll primarily use Angular style for commit messages. Roughly, they should follow the pattern:

<type>(<scope>): <short summary>

where scope (optional) describes the packages affected by the code changes and type (mandatory) is one of:

  • build: Changes that affect build tools or external dependencies (example scopes: pyproject.toml, setup.py)
  • ci: Changes to our CI configuration files and scripts (examples: .github/workflows/ci.yml)
  • docs: Documentation only changes
  • feat: A new feature
  • fix: A bugfix
  • perf: A code change that improves performance
  • refactor: A code change that neither fixes a bug nor adds a feature
  • test: Adding missing tests or correcting existing tests

Semantic Release

The table below, from semantic release, shows which commit message gets you which release type when semantic-release runs (using the default configuration):

Commit message Release type
fix(pencil): stop graphite breaking when too much pressure applied Patch Fix Release, Default release
feat(pencil): add 'graphiteWidth' option Minor Feature Release
perf(pencil): remove graphiteWidth option

BREAKING CHANGE: The graphiteWidth option has been removed.
The default graphite width of 10mm is always used for performance reasons.
Major Breaking Release
(Note that the BREAKING CHANGE: token must be in the footer of the commit)

Documentation

To generate the rst files source files for documentation, run

sphinx-apidoc -o doc_template/source/ src

Then to create the documentation HTML files, run

sphinx-build -b html doc_template/source/ doc_template/build/html

More info on sphinx installation can be found here.

Metadata

Release files for aind-behavior-video-transformation 1.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for aind-behavior-video-transformation 1.0.1
File Size Uploaded Provenance Actions
aind_behavior_video_transformation-1.0.1.tar.gz 22.4 kB Without provenance Details for aind_behavior_video_transformation-1.0.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for aind-behavior-video-transformation 1.0.1
File Interpreter ABI Platform Actions
aind_behavior_video_transformation-1.0.1-py3-none-any.whl Python 3 none any Details for aind_behavior_video_transformation-1.0.1-py3-none-any.whl

Total release size: 37.1 kB

Release files / aind_behavior_video_transformation-1.0.1.tar.gz

Download URL aind_behavior_video_transformation-1.0.1.tar.gz
Size 22.4 kB
Tags Source
SHA-256 checksum
How to use checksums
e0dcc73451c343f8c3723c4b950a58a10374cac099da3db924a9d228bc691e48
BLAKE2b-256 checksum
How to use checksums
d0b7605e72cc0f2a0365b6693a29bd559964434cc19bde7218009103591906ab
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / aind_behavior_video_transformation-1.0.1-py3-none-any.whl

Download URL aind_behavior_video_transformation-1.0.1-py3-none-any.whl
Size 14.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ea31a775cbddb0108264db7bfd8554c8bc8cf9ae1c4aca63c84ae04656ee81d3
BLAKE2b-256 checksum
How to use checksums
fd58f55f42b796f3e02af1a09fae37688c9251115f34be58394a4487577ed771
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for aind-behavior-video-transformation 1.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page