Snakefmt

This repository provides formatting for Snakemake files. It follows the design and specifications of Black.

Table of Contents

• Install
  • PyPi
  • Conda
  • Containers
  • Local
• Example File
• Usage
  • Basic Usage
  • Full Usage
  • Shell Block Formatting
  • Directive Sorting
    • Custom directive order
  • Format Directives
  • Configuration
• Integration
  • Editor Integration
  • Version Control Integration
  • GitHub Actions
• Plug Us
  • Markdown
  • ReStructuredText
• Changes
• Contributing
• Cite

Install

PyPi

pip install snakefmt

Conda

conda install -c bioconda snakefmt

Containers

As snakefmt has a Conda recipe, there is a matching image built for each version by Biocontainers.

In the following examples, all tags (<tag>) can be found here.

Docker

$ docker run -it "quay.io/biocontainers/snakefmt:<tag>" snakefmt --help

Singularity

$ singularity exec "docker://quay.io/biocontainers/snakefmt:<tag>" snakefmt --help

Local

These instructions include installing uv.

# install uv
curl -LsSf https://astral.sh/uv/install.sh | sh

git clone https://github.com/snakemake/snakefmt && cd snakefmt
# install snakefmt in a new environment
make install
# you can now run snakefmt with
uv run snakefmt --help

Example File

Input

from snakemake.utils import min_version
min_version("5.14.0")
configfile: "config.yaml" # snakemake keywords are treated like classes i.e. 2 newlines
SAMPLES = ['s1', 's2'] # strings are normalised
CONDITIONS = ["a", "b", "longlonglonglonglonglonglonglonglonglonglonglonglonglonglonglong"] # long lines are wrapped
include: "rules/foo.smk" # 2 newlines

rule all:
    input: "data/results.txt" # newlines after keywords enforced and trailing comma

rule gets_separated_by_two_newlines:
    input:
        files = expand("long/string/to/data/files/gets_broken_by_black/{sample}.{condition}",sample=SAMPLES, condition=CONDITIONS)
if True:
    rule can_be_inside_python_code:
        input: "parameters", "get_indented"
        threads: 4 # Numeric params stay unindented
        params: key_val = "PEP8_formatted"
        run:

                print("weirdly_spaced_string_gets_respaced")

Output

from snakemake.utils import min_version

min_version("5.14.0")


configfile: "config.yaml" # snakemake keywords are treated like classes i.e. 2 newlines


SAMPLES = ["s1", "s2"] # strings are normalised
CONDITIONS = [
    "a",
    "b",
    "longlonglonglonglonglonglonglonglonglonglonglonglonglonglonglong",
]  # long lines are wrapped


include: "rules/foo.smk" # 2 newlines


rule all:
    input:
        "data/results.txt", # newlines after keywords enforced and trailing comma


rule gets_separated_by_two_newlines:
    input:
        files=expand(
            "long/string/to/data/files/gets_broken_by_black/{sample}.{condition}",
            sample=SAMPLES,
            condition=CONDITIONS,
        ),


if True:

    rule can_be_inside_python_code:
        input:
            "parameters",
            "get_indented",
        threads: 4 # Numeric params stay unindented
        params:
            key_val="PEP8_formatted",
        run:
            print("weirdly_spaced_string_gets_respaced")

Usage

Basic Usage

Format a single Snakefile.

snakefmt Snakefile

Format all Snakefiles within a directory.

snakefmt workflows/

Format a file but write the output to stdout.

snakefmt - < Snakefile

Full Usage

$ snakefmt --help
Usage: snakefmt [OPTIONS] [SRC]...

  The uncompromising Snakemake code formatter.

  SRC specifies directories and files to format. Directories will be searched
  for file names that conform to the include/exclude patterns provided.

  Files are modified in-place by default; use diff, check, or  `snakefmt - <
  Snakefile` to avoid this.

Options:
  -l, --line-length INT           Lines longer than INT will be wrapped.
                                  [default: 88]
  -s, --sort / -S, --no-sort      Sort directives in rules and modules.
                                  [default: sort]
  -o, --directive-order NAMES     Comma-separated order of all the directives
                                  of rules, checkpoints and `use rule` blocks.
                                  Action directives (run, shell, script, ...)
                                  are not listed as they always come last. In
                                  the config file this is an array of strings.
                                  Use --print-directive-order to see the
                                  default order.
  -m, --module-directive-order NAMES
                                  Comma-separated order of all the directives
                                  of modules. In the config file this is an
                                  array of strings.
  -p, --print-directive-order     Print the directive orders in effect, as
                                  lines for the config file and as command
                                  line options, then exit.
  -f, --format-shell / -F, --no-format-shell
                                  Format shell directives using shfmt.
                                  [default: format-shell]
  --check                         Don't write the files back, just return the
                                  status. Return code 0 means nothing would
                                  change. Return code 1 means some files would
                                  be reformatted. Return code 123 means there
                                  was an error.
  -d, --diff                      Don't write the files back, just output a
                                  diff for each file to stdout.
  --compact-diff                  Same as --diff but only shows lines that
                                  would change plus a few lines of context.
  --include PATTERN               A regular expression that matches files and
                                  directories that should be included on
                                  recursive searches.  An empty value means
                                  all files are included regardless of the
                                  name.  Use forward slashes for directories
                                  on all platforms (Windows, too).  Exclusions
                                  are calculated first, inclusions later.
                                  [default: (\.smk$|^Snakefile)]
  --exclude PATTERN               A regular expression that matches files and
                                  directories that should be excluded on
                                  recursive searches.  An empty value means no
                                  paths are excluded. Use forward slashes for
                                  directories on all platforms (Windows, too).
                                  Exclusions are calculated first, inclusions
                                  later.  [default: (\.snakemake/|\.eggs/|\.gi
                                  t/|\.hg/|\.mypy_cache/|\.nox/|\.tox/|\.venv/
                                  |\.svn/|_build/|buck-
                                  out/|/build/|/dist/|\.template/)]
  -c, --config PATH               Read configuration from PATH. By default,
                                  will try to read from `./pyproject.toml`
  -h, --help                      Show this message and exit.
  -V, --version                   Show the version and exit.
  -v, --verbose                   Turns on debug-level logger.

Directive Sorting

By default, snakefmt sorts rule and module directives (like input, output, shell, etc.) into a consistent order. This makes rules easier to read and allows for quicker cross-referencing between inputs, outputs, and the resources used by the execution command.

You can disable this feature using the --no-sort flag.

Custom directive order

If you prefer a different order, you can set your own. There are two orders, because rules and modules have different directives:

• -o/--directive-order (directive_order in the config file) is the order of the directives in rule, checkpoint and use rule ... with: blocks.
• -m/--module-directive-order (module_directive_order in the config file) is the order of the directives in module blocks.

An order has to list every directive of its block type exactly once, so the order is never ambiguous. The easiest way to write one is to start from the orders in effect, which -p/--print-directive-order prints as lines you can paste into your pyproject.toml, and as command line options. It exits without formatting anything:

$ snakefmt --print-directive-order
# pyproject.toml, under [tool.snakefmt]:
directive_order = ["name", "default_target", "input", "output", "log", "benchmark", "pathvars", "wildcard_constraints", "cache", "priority", "retries", "group", "localrule", "handover", "shadow", "conda", "container", "singularity", "containerized", "envmodules", "threads", "resources", "params", "message"]
module_directive_order = ["name", "pathvars", "snakefile", "config", "skip_validation", "prefix", "replace_prefix", "meta_wrapper"]

# command line:
--directive-order name,default_target,input,output,log,benchmark,pathvars,wildcard_constraints,cache,priority,retries,group,localrule,handover,shadow,conda,container,singularity,containerized,envmodules,threads,resources,params,message --module-directive-order name,pathvars,snakefile,config,skip_validation,prefix,replace_prefix,meta_wrapper

Paste the lines you want into your pyproject.toml and move the directives around. For example, to sort snakefile and config first in modules:

[tool.snakefmt]
module_directive_order = ["snakefile", "config", "name", "pathvars", "skip_validation", "prefix", "replace_prefix", "meta_wrapper"]

or on the command line:

snakefmt --module-directive-order snakefile,config,name,pathvars,skip_validation,prefix,replace_prefix,meta_wrapper workflow/

-p prints the orders that are in effect after reading your configuration and command line options, so you can check them.

• Action directives (run, shell, script, notebook, wrapper, cwl and template_engine) are not part of the rule order. Snakemake allows only one action per rule and no directive may follow it, so actions always come last.
• snakefmt stops before formatting anything if an order has a directive that doesn't exist for its block type (for example snakefile in the rule order, or input in the module order), a directive listed more than once, an action directive, or if it is incomplete. For an incomplete order, the error lists the missing directives and shows a complete order, with your directives first and the missing ones after them in the default order, in the same two forms as above so you can check it and paste it.
• Directives inside a # fmt: off[sort] region keep their original order.
• With --no-sort (or sort_directives = false) nothing is sorted, so custom orders are ignored and snakefmt logs a warning. -p still prints the orders, with the same kind of warning.

Shell Block Formatting

By default, snakefmt formats the body of shell: directives using shfmt. This keeps shell snippets in your Snakefiles formatted consistently and avoids cosmetic diffs triggering unnecessary Snakemake re-runs.

Example

Before:

rule align:
    input:
        "reads.fq",
    output:
        "aligned.bam",
    threads: 4
    shell:
        """
        bwa mem -t {threads} ref.fa {input} | samtools sort -o {output} -
        if [ -s {output} ]
        then
        echo "done"
        else
        echo "empty"
        exit 1
        fi
        """

After:

rule align:
    input:
        "reads.fq",
    output:
        "aligned.bam",
    threads: 4
    shell:
        """
        bwa mem -t {threads} ref.fa {input} | samtools sort -o {output} -
        if [ -s {output} ]; then
            echo "done"
        else
            echo "empty"
            exit 1
        fi
        """

Disabling

You can disable shell formatting on the command line with --no-format-shell (-F), or in pyproject.toml:

[tool.snakefmt]
format_shell = false

shfmt is invoked with -i 4 -ci -bn (four-space indentation, indented switch cases, binary operators may start a line).

rule example:
    shell:
        """
        awk '{{print $1}}' {input} > {output}
        """

shell:
    """
    python <<!EOF!
    \nif True:
        print("hello")
    \n!EOF!
    """

Format Directives

snakefmt supports comment directives to control formatting behaviour for specific regions of code. Directives should appear as standalone comment lines, an inline occurrence (e.g. input: # fmt: off) is treated as a plain comment and has no effect. All directives are scope-local: only the region they select is affected, while code before and after follows normal snakefmt formatting and spacing rules (equivalent to replacing the directive with a plain comment line).

# fmt: off / # fmt: on

Disables all formatting for the region between the two directives. Both directives must appear at the same indentation level; a # fmt: on at a deeper indent than the matching # fmt: off has no effect.

rule a:
    input:
        "a.txt",


# fmt: off
rule b:
  input: "b.txt"
  output:
          "c.txt"
# fmt: on


rule c:
    input:
        "d.txt",

Note: inside run: blocks and other Python contexts, # fmt: off / # fmt: on is passed through to Black, which handles it natively.

# fmt: off[sort]
rule keep_my_order:
    output:
        "result.txt",
    input:
        "source.txt",
    shell:
        "cp {input} {output}"
# fmt: on[sort]

rule formatted:
    input:
        "a.txt",
    output:
        "b.txt",


# fmt: off[next]
rule unformatted:
  input: "a.txt"
  output: "b.txt"


rule also_formatted:
    input:
        "a.txt",

Configuration

snakefmt is able to read project-specific default values for its command line options from a pyproject.toml file. In addition, it will also load any black configurations you have in the same file.

By default, snakefmt will search in the parent directories of the formatted file(s) for a file called pyproject.toml and use any configuration there. If your configuration file is located somewhere else or called something different, specify it using --config.

Any options you pass on the command line will take precedence over default values in the configuration file.

Example

pyproject.toml

[tool.snakefmt]
line_length = 90
include = '\.smk$|^Snakefile|\.py$'
sort_directives = true   # sort rule directives into a consistent order (default: true)
format_shell = true      # format shell: blocks with shfmt (default: true)

# snakefmt passes these options on to black
[tool.black]
skip_string_normalization = true

In this example we increase the --line-length value and also include python (*.py) files for formatting - this effectively runs black on them. snakefmt will also pass on the [tool.black] settings, internally, to black.

Integration

Editor Integration

For instructions on how to integrate snakefmt into your editor of choice, refer to docs/editor_integration.md

Version Control Integration

snakefmt supports pre-commit, a framework for managing git pre-commit hooks. Using this framework you can run snakefmt whenever you commit a Snakefile or *.smk file. Pre-commit automatically creates an isolated virtual environment with snakefmt and will stop the commit if snakefmt would modify the file. You then review, stage, and re-commit these changes. Pre-commit is especially useful if you don't have access to a CI/CD system like GitHub actions.

To do so, create the file .pre-commit-config.yaml in the root of your project directory with the following:

repos:
  - repo: https://github.com/snakemake/snakefmt
    rev: v1.1.0 # Replace by any tag/version ≥v0.6.0 : https://github.com/snakemake/snakefmt/releases
    hooks:
      - id: snakefmt

Then install pre-commit and initialize the pre-commit hooks by running pre-commit install (Note you need to run this step once per clone of your repository). Additional pre-commit hooks can be found here.

GitHub Actions

GitHub Actions in combination with super-linter allows you to automatically run snakefmt on all Snakefiles in your repository e.g. whenever you push a new commit.

---
name: Lint

on: # yamllint disable-line rule:truthy
  push: null
  pull_request: null

permissions: {}

jobs:
  build:
    name: Lint
    runs-on: ubuntu-latest

    permissions:
      contents: read
      packages: read
      # To report GitHub Actions status checks
      statuses: write
    steps:
      - name: Checkout code
        uses: actions/checkout@v5
        with:
          # super-linter needs the full git history to get the
          # list of files that changed across commits
          fetch-depth: 0
          persist-credentials: false

      - name: Lint Code Base
        uses: super-linter/super-linter@v8.2.1
        env:
          VALIDATE_ALL_CODEBASE: false
          DEFAULT_BRANCH: main
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

          VALIDATE_SNAKEMAKE_SNAKEFMT: true

[tool.black]
skip_string_normalization = true

For more information check the super-linter readme.

Alternatively, MegaLinter is another linter aggregator that runs snakefmt out of the box on any CI system - see its snakefmt page for setup.

Plug Us

If you can't get enough of badges, then feel free to show others you're using snakefmt in your project.

[![Code style: snakefmt](https://img.shields.io/badge/code%20style-snakefmt-000000.svg)](https://github.com/snakemake/snakefmt)

.. image:: https://img.shields.io/badge/code%20style-snakefmt-000000.svg
    :target: https://github.com/snakemake/snakefmt

Changes

See CHANGELOG.md.

Contributing

See CONTRIBUTING.md.

Cite

@article{snakemake2021,
  doi = {10.12688/f1000research.29032.2},
  url = {https://doi.org/10.12688/f1000research.29032.2},
  year = {2021},
  month = apr,
  publisher = {F1000 Research Ltd},
  volume = {10},
  pages = {33},
  author = {Felix M\"{o}lder and Kim Philipp Jablonski and Brice Letcher and Michael B. Hall and Christopher H. Tomkins-Tinch and Vanessa Sochat and Jan Forster and Soohyun Lee and Sven O. Twardziok and Alexander Kanitz and Andreas Wilm and Manuel Holtgrewe and Sven Rahmann and Sven Nahnsen and Johannes K\"{o}ster},
  title = {Sustainable data analysis with Snakemake},
  journal = {F1000Research}
}