batchmp 1.0

Command-line tools for batch media processing

Status: A rainy weekends project under occasional development :)

Requirements:

• Python 3.4.x or later

Install:

• from PyPI: $ pip install batchmp
• latest from source repository: $ pip install git+https://github.com/akpw/batch-mp-tools.git

Blogs:

• Practical BatchMP
• BatchMP Tools Tutorial
• The BatchMP Tools Project
• Parallel batch media processing with FFmpeg and Python

Description

Batch management of media files, from base properties such as file names through tags / artwork metadata to advanced operations on the media content.

The project is written in Python 3.4 and currently consists of three main command-line utilities. All three share the core concept of various transformations applied to generated stream of file system entries, and consequently they also share the same set of global options. A quick way to check on that is to run:

    $ renamer -h
    $ tagger -h
    $ bmfp -h

That will show the global options along with specific commands for each tool. Getting more info on the commands level can be done using a similar approach, e.g. to learn more about the renamer replace command:

    $ renamer replace -h

By default the tools always visualize targeted changes (whenever possible) before actual processing.

A little bit more details on each utility:

Renamer is a multi-platform batch rename tool. In addition to common operations such as regexp-based replace, adding text / dates, etc. it also supports advanced operations such as expandable template processing during replace, multi-level indexing across nested directories, flattening folders, and cleaning up non-media files. At its simplest, Renamer can be used to print out the content of current directory:

    $ renamer

Without command arguments, renamer uses print as the default command. This is also the case for both $ tagger and $ bmfp, with each of the tool showing info which is relevant to its intended purpose. A little bit more advanced, to see what's lurking at the 7th nested folder level:

    $ renamer print -sl 7

For multi-level indexing of all M4A files in all sub-directories of the current folder:

    $ renamer -r -in '*.m4a' index
    /Desktop/_test/Gould
      |-/1
        |- 01 Glenn Gould French Suite 1 In D, BWV812 1 Allemande.m4a
        |- 02 Glenn Gould French Suite 1 In D, BWV812 2 Courante.m4a
        |- 03 Glenn Gould French Suite 1 In D, BWV812 3 Sarabande.m4a
      |-/2
        |- 01 Bach, Johann Sebastian French Suite 5 In G Major, BWV816 1 Allemande.m4a
        |- 02 Bach, Johann Sebastian French Suite 5 In G Major, BWV816 2 Courante.m4a
        |- 03 Bach, Johann Sebastian French Suite 5 In G Major, BWV816 3 Sarabande.m4a
      |-/_Art
    6 files, 3 folders

    Proceed? [y/n]:

Sequential indexing is supported as well using the -sq switch. An important detail here, by default Renamer is visualizing the targeted changes and asking for permission to proceed before actually doing anything.

Tagger manages media metadata, such as tags and artwork. Setting those in selected media file over multiple nested directories now becomes a breeze, with just a few simple commands working uniformly over almost any practically imaginable audio / video media formats. While easy to use, Tagger supports advanced metadata manipulation such as regexp-based replace, expandable template processing, etc. For example, to set the title tag to respective file names followed by the values of track and tracktotal tags:

    $ tagger -r -in '*BWV816 1*' set --title '$filename, $track of $tracktotal'
    Targeted after processing:
    /Desktop/_test/Gould
      |-/1
      |-/2
        |- Bach, Johann Sebastian French Suite 5 In G Major, BWV816 1 Allemande.m4a
            Title: Bach, Johann Sebastian French Suite 5 In G Major, BWV816 1 Allemande, 1 of 26
            Album: Bach French Suites BWV 812-817 Vol. II; Glenn Gould
            Artist: Glenn Gould
            Album Artist: Glenn Gould
            Genre: Classical
            Composer: Johann Sebastian Bach (1685-1750)
            Year: 1994
            Track: 1/26
            Disk: 2/2
      |-/Art
    1 files, 3 folders

    Proceed? [y/n]: n

The commands above show some of the available global options: -r for recursion into nested folders and -in to select media files. In the example above just one file was selected (for the sake of output brevity), which also could be achived via using -f for the file source mode.

BMFP is all about efficient media content processing, such as conversion between various formats, normalizing sound volume, segmenting / fragmenting media files, denoising audio, detaching individual audio / video streams, etc. As processing media files can typically be resource consuming, BMFP is designed to take advantage of multi-core processors. By default, it automatically breaks up jobs into individual tasks that are then run as separate processes on available CPU cores. BMFP is built on top of FFmpeg, which needs to be installed and available in the command line. BMFP can be thought of as a batch FFmpeg runner, intended to make common uses of FFmpeg easy while not restricting its full power.

For example, to convert all files from the above example from M4A to FLAC:

    $ bmfp -r convert -la -tf FLAC

The -tf switch specifies the target format, while -la explicitly tells BMFP to do a lossless conversion.

To check on the result, lets's just use the tagger's ability to print media files info:

    $ tagger -r -in '*BWV816 1*' print -st -ss
    /Users/AKPower/Desktop/_test/Gould
      |-/1
      |-/2
        |-  7.4MB Bach, Johann Sebastian French Suite 5 in G major, BWV816 1 Allemande.flac
            Title: French Suite 5 in G major, BWV 816 - 1 Allemande
            Album: Bach French Suites BWV 812-817 Vol. II; Glenn Gould
            Artist: Glenn Gould
            Album Artist: Glenn Gould
            Genre: Classical
            Composer: Johann Sebastian Bach (1685-1750)
            Year: 1994
            Track: 1/26
            Disk: 2/2
            Format: FLAC
            Duration: 0:01:48, Bit rate: 548kb/s, Sample rate: 44100Hz, Bit depth: 16
        |-/_backup_15Mar25_094341
          |-  7.9MB Bach, Johann Sebastian French Suite 5 in G major, BWV816 1 Allemande.m4a
              Title: French Suite 5 in G major, BWV 816 - 1 Allemande
              Album: Bach French Suites BWV 812-817 Vol. II; Glenn Gould
              Artist: Glenn Gould
              Album Artist: Glenn Gould
              Genre: Classical
              Composer: Johann Sebastian Bach (1685-1750)
              Year: 1994
              Track: 1/26
              Disk: 2/2
              Format: ALAC
              Duration: 0:01:48, Bit rate: 579kb/s, Sample rate: 44100Hz, Bit depth: 16
      |-/Art
    2 files, 4 folders
    Total size: 15.2MB

From a brief glance, all looks OK. BMFP used FFmpeg to do the actual conversion, while taking care of all other things like preserving tags / artwork, etc.

I will follow up with more examples and common use-cases in future blogs.

Brief Description of CLI Commands (use -h to expand on details for individual commands)

renamer

Batch renaming of files and directories
  . source directory or source file modes
  . visualises original / targeted folders structure before actual processing
  . supports recursion, can optionally stop at specified end_level
  . supports flattening folders beyond specified target_level
  . supports include / exclude patterns (Unix style)
  . display sorting:
      .. by size/date, ascending/descending
  . action commands:
      .. print      Prints source directory
      .. flatten    Flatten all folders below target level, moving the
                    files up at the target level. By default, deletes all empty flattened folders
      .. index      Adds index to files and directories names
      .. replace    RegExp-based replace in files and directories names. Supports expandable templates, 
                    such as $dirname, $pardirname, $atime, $ctime, etc. For media files, also can process 
                    tag-based templates such as $title, $album, $artist, $albumartist, $genre, $year, $track, 
                    etc.
      .. add_date   Adds date to files and directories names
      .. add_text   Adds text to files and directories names
      .. remove     Removes n characters from files and directories names
      .. capitalize Capitalizes words in files / directories names
      .. delete     Delete selected files and directories

Usage: renamer [-h] [-d DIR] [-f FILE] [Global Options] {Commands} [Commands Options]
Global Options:
  Input source mode:
    [-d, --dir]                 Source directory (default is the current directory)
    [-f, --file]                File to process

  Recursion mode:
    [-r, --recursive]           Recurse into nested folders
    [-el, --end-level]          End level for recursion into nested folders

  Filter files or folders:
    [-in, --include]            Include: Unix-style name patterns separated by ';'
    [-ex, --exclude]            Exclude: Unix-style name patterns separated by ';'
                                  (excludes hidden files by default)
    [-fd, --filter-dirs]        Enable  Include/Exclude patterns on directories
    [-af, --all-files]          Disable Include/Exclude patterns on files
                                  (shows hidden files excluded by default)
  Miscellaneous:
    [-s, --sort]{na|nd|sa|sd}   Sort order for files / folders (name | date, asc | desc)
    [-ni, nested-indent]        Indent for printing nested directories
    [-q, --quiet]               Do not visualise changes / show messages during processing

  Commands:
    {print, index, add_date, add_text, remove, replace, capitalize, flatten, delete, version, info}
    $ renamer {command} -h  #run this for detailed help on individual commands

tagger

Supported formats:

'MP3', 'MP4', 'M4A', M4V', 'AIFF', 'ASF', 'QuickTime / MOV', 'FLAC', 'MonkeysAudio', 'Musepack', 'Ogg FLAC', 'Ogg Speex', 'Ogg Theora', 'Ogg Vorbis', 'True Audio', 'WavPack', 'OptimFROG'

Support via FFmpeg: 'AVI', 'FLV', 'MKV', 'MKA'

Batch management of media files metadata (tags & artwork)
  . source directory / source file modes
  . visualises original / targeted files metadata structure
  . supports recursion, can optionally stop at specified end_level
  . include / exclude patterns (Unix style)
  . display sorting:
      .. by size/date, ascending/descending
  . action commands:
      .. print      Prints media info
      .. set        Sets tags in media files, including artwork, e.g:
                            $ tagger set --album 'The Album' -art '~/Desktop/art.jpg'
                        Supports expandable templates. To specify a template value,
                        use the long tag field name preceded by $:
                            $ tagger set --title '$title, $track of $tracktotal'
                        In addition to tag fields templates, file system names are also supported:
                            $ tagger set --title '$filename' --album '$dirname' --artist '$pardirname'...
      .. copy       Copies tags from a specified media file
      .. index      Indexes Track / Track Total tags
      .. remove     Removes tags from media files
      .. replace    RegExp-based replace in specified tags
                      e.g., to remove the first three characters in title:
                            $ tagger replace -tf 'title' -fs '^[\s\S]{0,3}' -rs ''
      .. capitalize Capitalizes words in specified tags
      .. detauch    Extracts artwork

Usage: tagger [-h] [-d DIR] [-f FILE] [Global Options] {Commands} [Commands Options]
Global Options:
  Input source mode:
    [-d, --dir]                 Source directory (default is the current directory)
    [-f, --file]                File to process

  Recursion mode:
    [-r, --recursive]           Recurse into nested folders
    [-el, --end-level]          End level for recursion into nested folders

  Filter files or folders:
    [-in, --include]            Include: Unix-style name patterns separated by ';'
    [-ex, --exclude]            Exclude: Unix-style name patterns separated by ';'
                                  (excludes hidden files by default)
    [-fd, --filter-dirs]        Enable  Include/Exclude patterns on directories
    [-af, --all-files]          Disable Include/Exclude patterns on files
                                  (shows hidden files excluded by default)
  Miscellaneous:
    [-s, --sort]{na|nd|sa|sd}   Sort order for files / folders (name | date, asc | desc)
    [-ni, nested-indent]        Indent for printing nested directories
    [-q, --quiet]               Do not visualise changes / show messages during processing

  Commands:
    {print, set, copy, index, remove, replace, capitalize, detauch, version, info}
    $ tagger {command} -h #run this for detailed help on individual commands

bmfp (requires FFmpeg)

Batch processing of media files
  . Uses multiprocessing to utilize available CPU cores
  . source directory / source file modes
  . recursion to specified end_level
  . include / exclude patterns (Unix style)
  . action commands:
      .. print          Prints media files
      .. convert        Converts media to specified format
                            For example, to convert all files in current directory
                                $ bmfp convert -la -tf FLAC
      .. normalize      Nomalizes sound volume in media files
                            Peak normalization supported, RMS normalizations TBD
      .. fragment       Extract a media file fragment
      .. segment        Splits media files into segments
                            For example, to split media files in segments of 45 mins:
                                $ bmfp segment -d 45:00
      .. silencesplit   Splits media files into segments via detecting specified silence
                                $ bmfp silencesplit
      .. cuesplit       Splits media files into parts with specified output format,
                        according to their respective cue sheets
                            For example, to split all cue files in the current directory
                                $ bmfp cuesplit -tf mp3
      .. denoise        Reduces background audio noise in media files

      .. adjust volume  TDB: Adjust audio volume
      .. speed up       TDB: Uses Time Stretching to increase audio / video speed
      .. slow down      TDB: Uses Time Stretching to increase audio / video speed

Usage: bmfp [-h] [-d DIR] [-f FILE] [GLobal Options] {Commands}[Commands Options]
  Input source mode:
    [-d, --dir]                 Source directory (default is the current directory)
    [-f, --file]                File to process

  Recursion mode:
    [-r, --recursive]           Recurse into nested folders
    [-el, --end-level]          End level for recursion into nested folders

  Filter files or folders:
    [-in, --include]            Include: Unix-style name patterns separated by ';'
    [-ex, --exclude]            Exclude: Unix-style name patterns separated by ';'
                                  (excludes hidden files by default)
    [-fd, --filter-dirs]        Enable  Include/Exclude patterns on directories
    [-af, --all-files]          Disable Include/Exclude patterns on files
                                  (shows hidden files excluded by default)

    Target output Directory     Target output directory. When omitted, will be
    [-td, --target-dir]         automatically created at the parent level of
                                the input source. For recursive processing,
                                the processed files directory structure there
                                will be the same as for the original files.
  FFmpeg General Output Options:
    [-ma, --map-all]            Force including all streams from the input file
    [-cc, --copy-codecs]        Copy streams codecs without re-encoding
    [-vn, --no-video]           Exclude video streams from the output
    [-an, --no-audio]           Exclude audio streams from the output
    [-sn, --no-subs]            Exclude subtitles streams from the output
    [-fo, --ffmpeg-options]     Additional FFmpeg options

  FFmpeg Commands Execution:
    [-q, --quiet]               Do not visualise changes / show messages during processing
    [-se, --serial-exec]        Run all task's commands in a single process

  Commands:
    {print, convert, normalize, fragment, segment, silencesplit, cuesplit, denoise, version, info}
    $ bmfp {command} -h  #run this for detailed help on individual commands

Installing Development version

• Clone the repo, then run: $ python -m pip install .

Running Tests

• Run via: $ python setup.py test