Cross platform framework for splitting and parallel encoding of video
Project description
PyParallelEncode
Framework to split and encode videos
Easy, powerful, all-in-one encoding based on Av1an
Output to AV1 / VVC / HEVC / H264 / VP9 / VP8. Input from all formats supported by ffmpeg.
Encoders supported: AOM, RAV1E, SVT-AV1, VTM, x265, x264, SVT-VP9, VPX.
Python example with default parameters:
from pathlib import Path
from parallelencode import run
params = {"input": Path("input.mp4")}
run(params)
Parameters
input Path: Input file (default: None) Required
output_file Path: output file to create (default: (input file name)_(encoder).mkv)
Recommended to end with .mkv to avoid errors.
encoder str: Encoder to use
(`aom`,`rav1e`,`svt_av1`,`svt_vp9`,`vpx`,`x265`, `x264`,`vvc`)
(default: "aom")
video_params list[str]: Strongly recommended to use shlex.split to generate
from a string.
(If not set, default encoder parameters will be used.)
(Example: 'video_params': shlex.split("--rt -t 12 --kf-max-dist=240")
passes int: Number of passes for encoding. most encoders only support 1/2
(Default: AOMENC: 2, rav1e: 1, SVT-AV1: 1, SVT-VP9: 1,
VPX: 2, x265: 1, x264: 1, VVC:1)
workers int: Number of encoding workers. More means more instances of
encoder running at once. (default: half the system threads)
resume bool: If encode was stopped/quit resumes, redoing all partial chunks.
but skipping any chunks which have already been completed.
Does nothing if temp folder cannot be found. If found it automatically
skips scenedetection, audio encoding/copy, spliting, so it only works
after actual encoding has started. (default: False)
no_check bool: Skip checking numbers of frames for source and encoded chunks.
Needed if framerate changes are added in ffmpeg options. Not recommended
By default any differences in frames of encoded files will be reported
and will likely result in corrupted video. (default: False)
keep bool: If true, do not delete temporary folder after completion.
(default: False)
temp Path: path for temporally folders. (default: .temp)
mkvmerge bool: Use mkvmerge for concatenating instead of ffmpeg.
Recommended if concatenation fails. (default: False)
FFmpeg options
audio_params list[str]: FFmpeg audio args (Default: ['-c:a', 'copy'])
(Example: 'audio_params': shlex.split("-c:a libopus -b:a 64k")
ffmpeg list[str]: FFmpeg video params. Run on each chunk before encoding.
(Warning: Cropping doesn't work with Target VMAF mode)
(default: None)
(Example: 'ffmpeg': shlex.split("-vf scale=320:240")
pix_format str: pixel/bit format for piping. This should match your
input video (default: 'yuv420p')
(Example for 10 bit source: 'pix_format': 'yuv420p10le')
Ensure your encoder supports this format and knows about it.
Segmenting
split_method str: Method used for generating splits.
(`pyscene`, `aom_keyframes`, `file`, `none`) (default: 'pyscene')
`pyscene` - PyScenedetect, content based scenedetection
with threshold.
`aom_keyframes` - using stat file of 1 pass of aomenc encode
to get exact place where encoder will place new keyframes.
`file` - read from file specified by 'scenes'
(NOT recommended unless using aom encoder)
(Keep in mind that speed also depends on set aomenc parameters)
chunk_method str: How chunks are made for encoding.
('hybrid', 'select', 'vs_ffms2', 'vs_lsmash')
(default: 'hybrid') vs_ffms2 is probably best but also beta
threshold float: PySceneDetect threshold for scene detection. Larger
values make it less sensitive, splitting less (default: 35.)
scenes Path: Path to file with scenes timestamps. If existing file
specified and split_method is file, acts as alternative split
method.
Otherwise csv file will be generated with splits data.
(default: None)
extra_split int: Adding extra splits if frame distance beetween splits bigger than
given value. (default: None)
Example: imagine a 1000 frame scene, 'extra_split': 200
will add splits at 200,400,600,800.
Target VMAF
vmaf_target float: Vmaf value to target. Supports all except vvc.
Setting this will enable target vmaf mode.
Requires crf or q mode or whatever equivalent the
encoder has and an explicit "default" crf value.
min_q, max_q Min,Max Q values limits for Target VMAF
If not set by user, encoder default will be used.
vmaf bool: Calculate vmaf after encode is done. showing vmaf values for all frames,
mean, 1,25,75 percentile. Invokes "plotvmaffile" callback with results.
vmaf_path str: Custom path to libvmaf models. Does NOT need to be set if ffmpeg
default path works. Path should point to .pkl file in same folder as
.model file. (default: None)
vmaf_res str: Optional scaling for vmaf calculation. Should match model resolution
vmaf_v0.6.1.pkl is 1920x1080 (by default), vmaf_4k_v0.6.1.pkl is 3840x2160
(default: "1920x1080"). Will preserve original aspect ratio
vmaf_steps int: Number of probes for interpolation in vmaf calculations.
1 and 2 probes have special mechanisms to make them slightly less bad.
Optimal is 4-6 probes. Default: 4
vmaf_filter str: Filter used for vmaf calculation with ffmpeg. Uses filter_complex.
(default: None). (example: 'vmaf_filter': "crop=200:1000:0:0")
vmaf_rate int: Framerate for vmaf testing. Set to 0 to use original video framerate
or to any other number to save cpu cycles at the cost of some accuracy
(default: 4)
n_threads Limit number of threads that used for vmaf calculation
(default: None) - Default has no thread limit for vmaf calculations
Register callbacks with ease using the built in callbacks tool:
def run_on_newframes(frames):
print("We just transcoded an extra " + frames + " frames")
# TODO: add code here
from parallelencode import Callbacks
from parallelencode import run
c = Callbacks()
c.subscribe("newframes", run_on_newframes)
params = {"input": Path("input.mp4")}
run(params, run_on_log)
Main Features
Spliting video by scenes for parallel encoding because all encoders are currently not as good at multithreading as amd is at making threads, encoding is often limited to a few threads at the same time.
- PySceneDetect used for spliting video by scenes and running multiple encoders.
- Fastest way to transcode video into lossy formats.
- Target VMAF mode. Saves tons of bitrate while generating good looking video.
- Resuming encoding without loss of encoded progress.
- Easy to use.
- Automatic detection of the number of workers the host can handle.
- Building encoding queue with bigger files first, minimizing waiting for the last scene to encode.
- Supports audio transcoding through FFmpeg.
Install
-
Prerequisites:
- Install Python3
When installing under Windows, select the optionadd Python to PATH
in the installer - Install FFmpeg
- Install Python3
-
At least one of these encoders:
- Install AOMENC
- Install rav1e
- Install SVT-AV1
- Install SVT-VP9
- Install vpx VP9, VP8 encoding
- Install VTM VVC encoding test model
-
Optional:
-
With a package manager:
-
Manually:
- Clone Repo or Download from Releases
pip3 install -r requirements.txt
python setup.py install
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
parallelencode-0.1.tar.gz
(36.9 kB
view hashes)
Built Distribution
Close
Hashes for parallelencode-0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cfaf7d28cd8cafd5a30d4d8cf760ed00f0ad2ec847d6a81fbf8f97b37d9070c1 |
|
MD5 | 0ac6f5194413a15971ac7163ebb1a2ed |
|
BLAKE2b-256 | 8e6a200bb254fd27852bd8acd130f5be72adb4ba548920221f460f8915f9f7d5 |