Skip to main content

Python code snippets for markdown files, e.g READMEs, from actual (testable) code.

Project description

Snipinator

Audience: Developers Platform: Linux

🏠Home  •  🎇Features  •  🔨Installation  •  🚜Usage  •  💻CLI  •  💡Examples

🤖Jinja2 API  •  ✅Requirements  •  🚸Gotchas

Top language GitHub License PyPI - Version Python Version

CLI to embed (testable) snippets from your codebase into your README


Status Stable Unstable
Master Build and Test since tagged last commit
Develop Build and Test since tagged since tagged last commit
Demo

❔ What

What it does: Snipinator lets you take a EXAMPLE.md template and include snippets from your (working and tested) codebase.

Turn this (./snipinator/examples/EXAMPLE.md.jinja2):

# A README

Here is a code snippet:

<!--{{ pysnippet(path='snipinator/examples/code.py', symbol='MyClass', backtickify='py', decomentify='nl') }}-->

Note that `code.py` has a test:
{{path('./snipinator/examples/code_test.py', link='md')}}.

Into this (./snipinator/examples/EXAMPLE.generated.md):

<!--

WARNING: This file is auto-generated by snipinator. Do not edit directly.
SOURCE: `snipinator/examples/EXAMPLE.md.jinja2`.

-->
# A README

Here is a code snippet:

<!---->
```py
class MyClass:
  """This is a global class"""

  def __init__(self, name):
    self.name = name

  def MyClassMethod(self):
    """This is a method of MyClass"""
    print(self.name)
```
<!---->

Note that `code.py` has a test:
[./snipinator/examples/code_test.py](./snipinator/examples/code_test.py).

🎇 Features

  • 📦✅🪄 Supports anything Jinja2 supports.
  • 🥇🐍📜 First-class support for python source code.
    • Can include python function signatures, docstrings, entire function source code, classes.
  • ✂🌐🗂️ Snip from any source code language.
    • Put delimiter markers into the code (e.g # START_SNIPPET, # END_TEMPLATE), and use snippet().
  • 🥇🔖📜 First-class support for Markdown templates (with backtickify, decomentify).
  • 📦🐚🖨️ Can include shell output.
    • Supports ANSI colors :heart: :green_heart: :blue_heart: with SVG output :camera:.
  • ⚙️🔗🗃️ More robust references/links to local files using path().

🔨 Installation

# Install from pypi (https://pypi.org/project/snipinator/)
pip install snipinator

# Install from git (https://github.com/realazthat/snipinator)
pip install git+https://github.com/realazthat/snipinator.git@v1.4.1

🚜 Usage

Example template README: (./snipinator/examples/EXAMPLE.md.jinja2):

# A README

Here is a code snippet:

<!--{{ pysnippet(path='snipinator/examples/code.py', symbol='MyClass', backtickify='py', decomentify='nl') }}-->

Note that `code.py` has a test:
{{path('./snipinator/examples/code_test.py', link='md')}}.

Generating the README:

$ python -m snipinator.cli -t snipinator/examples/EXAMPLE.md.jinja2
<!--

WARNING: This file is auto-generated by snipinator. Do not edit directly.
SOURCE: `snipinator/examples/EXAMPLE.md.jinja2`.

-->
# A README

Here is a code snippet:

<!---->
```py
class MyClass:
  """This is a global class"""

  def __init__(self, name):
    self.name = name

  def MyClassMethod(self):
    """This is a method of MyClass"""
    print(self.name)
```
<!---->

Note that `code.py` has a test:
[./snipinator/examples/code_test.py](./snipinator/examples/code_test.py).

💻 Command Line Options

Output of `python -m snipinator.cli --help`

💡 Examples

🤖 Jinja2 API

The regular Jinja2 v3 template syntax is supported. For more information, see Template Designer Documentation.

Additional (Jinja2) functions made available:

🐍✂ pysnippet

Used several times in ./snipinator/examples/LONG-EXAMPLE.md.jinja2.

Documentation:

def pysnippet(path: str,
              symbol: Optional[str],
              *,
              escape: bool = False,
              indent: Union[str, int, None] = None,
              indented: Union[str, int, None] = None,
              backtickify: Union[bool, str] = False,
              decomentify: Union[bool, Literal['nl']] = False,
              _ctx: _Context) -> Union[str, markupsafe.Markup]:
  """Return a python snippet, allowing you to specify a class or function.

  Args:
      path (str): The path to the file.
      symbol (Optional[str]): The symbol to extract. If None, the entire file is
        returned. Defaults to None.
      escape (bool, optional): Should use HTML entities escaping? Defaults to
        False.
      indent (Union[str, int, None], optional): Should indent? By how much, or
        with what prefix? Defaults to None.
      indented (Union[str, int, None], optional): Indents every line except the
        first. By how much, or with what prefix? Defaults to None.
      backtickify (Union[bool, str], optional): Should surround with backticks?
        With what language? Defaults to False.
      decomentify (Union[bool, Literal['nl']]): Assuming that you will be using
        HTML comments around this call, setting this to true will add
        correspondingcomments to uncomment the output. This allows you to have
        the Jinja2 call unmolested by markdown formatters, because they will be
        inside of a comment section. "nl" adds additional newlines after the
        newline delimiters. Defaults to False.
      _ctx (_Context): This is used by the system and is not available as an
        argument.

  Returns:
      Union[str, markupsafe.Markup]: The snippet.
  """

🐍📖 pysignature

Used several times in ./README.md.jinja2.

Documentation:

def pysignature(path: str,
                symbol: str,
                *,
                escape: bool = False,
                indent: Union[str, int, None] = None,
                indented: Union[str, int, None] = None,
                backtickify: Union[bool, str] = False,
                decomentify: Union[bool, Literal['nl']] = False,
                _ctx: _Context) -> str:
  """Return the signature of a class or function in a python file.

  Returns the {class,function} signature and the docstring.

  Args:
      path (str): The path to the file.
      symbol (str): The symbol to extract.
      escape (bool, optional): Should use HTML entities escaping? Defaults to
        False.
      indent (Union[str, int, None], optional): Should indent? By how much, or
        with what prefix? Defaults to None.
      indented (Union[str, int, None], optional): Indents every line except the
        first. By how much, or with what prefix? Defaults to None.
      backtickify (Union[bool, str], optional): Should surround with backticks?
        With what language? Defaults to False.
      decomentify (Union[bool, Literal['nl']]): Assuming that you will be using
        HTML comments around this call, setting this to true will add
        correspondingcomments to uncomment the output. This allows you to have
        the Jinja2 call unmolested by markdown formatters, because they will be
        inside of a comment section. "nl" adds additional newlines after the
        newline delimiters. Defaults to False.
      _ctx (_Context): This is used by the system and is not available as an
        argument.

  Returns:
      str: The signature and docstring.
  """

✂ rawsnippet

Used several times in ./README.md.jinja2.

Documentation:

def rawsnippet(path: str,
               *,
               escape: bool = False,
               indent: Union[str, int, None] = None,
               indented: Union[str, int, None] = None,
               backtickify: Union[bool, str] = False,
               decomentify: Union[bool, Literal['nl']] = False,
               _ctx: _Context) -> Union[str, markupsafe.Markup]:
  """Return an entire file as a snippet.

  Args:
      path (str): The path to the file.
      escape (bool, optional): Should use HTML entities escaping? Defaults to
        False.
      indent (Union[str, int, None], optional): Should indent? By how much, or
        with what prefix? Defaults to None.
      indented (Union[str, int, None], optional): Indents every line except the
        first. By how much, or with what prefix? Defaults to None.
      backtickify (Union[bool, str], optional): Should surround with backticks?
        With what language? Defaults to False.
      decomentify (Union[bool, Literal['nl']]): Assuming that you will be using
        HTML comments around this call, setting this to true will add
        correspondingcomments to uncomment the output. This allows you to have
        the Jinja2 call unmolested by markdown formatters, because they will be
        inside of a comment section. "nl" adds additional newlines after the
        newline delimiters. Defaults to False.
      _ctx (_Context): This is used by the system and is not available as an
        argument.

  Returns:
      Union[str, markupsafe.Markup]: The snippet.
  """

✂ snippet

Example in ./snipinator/examples/LONG-EXAMPLE.md.jinja2.

Documentation:

def snippet(path: str,
            start: str,
            end: str,
            *,
            escape: bool = False,
            indent: Union[str, int, None] = None,
            indented: Union[str, int, None] = None,
            backtickify: Union[bool, str] = False,
            decomentify: Union[bool, Literal['nl']] = False,
            _ctx: _Context) -> Union[str, markupsafe.Markup]:
  """Returns a _delimited_ snippet from a file.

  Does not return the delimiters themselves.

  Args:
      path (str): The path to the file.
      start (str): A string that indicates the start of the snippet.
      end (str): A string that indicates the end of the snippet.
      escape (bool, optional): Should use HTML entities escaping? Defaults to
        False.
      indent (Union[str, int, None], optional): Should indent? By how much, or
        with what prefix? Defaults to None.
      indented (Union[str, int, None], optional): Indents every line except the
        first. By how much, or with what prefix? Defaults to None.
      backtickify (Union[bool, str], optional): Should surround with backticks?
        With what language? Defaults to False.
      decomentify (Union[bool, Literal['nl']]): Assuming that you will be using
        HTML comments around this call, setting this to true will add
        correspondingcomments to uncomment the output. This allows you to have
        the Jinja2 call unmolested by markdown formatters, because they will be
        inside of a comment section. "nl" adds additional newlines after the
        newline delimiters. Defaults to False.
      _ctx (_Context): This is used by the system and is not available as an
        argument.

  Returns:
      Union[str, markupsafe.Markup]: The snippet.
  """

🐚 shell

Used several times in ./README.md.jinja2.

Documentation:

def shell(args: str,
          *,
          escape: bool = False,
          indent: Union[str, int, None] = None,
          indented: Union[str, int, None] = None,
          backtickify: Union[bool, str] = False,
          decomentify: Union[bool, Literal['nl']] = False,
          rich: Union[Literal['svg'], Literal['img+b64+svg'], Literal['raw'],
                      str] = 'raw',
          rich_alt: Optional[str] = None,
          rich_bg_color: Optional[str] = None,
          rich_term: Optional[str] = None,
          rich_rows: int = 24,
          rich_cols: int = 80,
          include_args: bool = True,
          _ctx: _Context) -> Union[str, markupsafe.Markup]:
  """Run a shell command and return the output.

  Use at your own risk, this can potentially introduce security vulnerabilities.
  Only use if you know what you are doing. Ensure that no untrusted input can
  be injected into the `args` parameter, or, into anything the command might
  access. If an adversary can control the `args` parameter, they can execute
  arbitrary commands on your system.

  Note: On persistent output colors:

  * I found that the environment variables TERM, COLORTERM and FORCE_COLOR,
    CLI_WIDTH, COLUMNS also influence the outputs for some applications.
  * Also various library versions used in various programs, e.g colorama,
    rich-argparse, Pygments might influence the output.
  * I had to pin all my python packages, and explicitly set TERM, COLORTERM and
    FORCE_COLOR, CLI_WIDTH, COLUMNS to get the output to be consistent across
    two different systems, both using Ubuntu, for a single program.

  Args:
      args (str): The command to run.
      escape (bool, optional): Should use HTML entities escaping? Defaults to
        False.
      indent (Union[str, int, None], optional): Should indent? By how much, or
        with what prefix? Defaults to None.
      indented (Union[str, int, None], optional): Indents every line except the
        first. By how much, or with what prefix? Defaults to None.
      backtickify (Union[bool, str], optional): Should surround with backticks?
        With what language? Defaults to False.
      decomentify (bool, optional): Assuming that you will be using HTML
        comments around this call, setting this to true will add corresponding
        uncomments to uncomment the output. This allows you to have the Jinja2
        call unmolested by markdown formatters, because they will be inside of
        a comment section. Defaults to False.
      rich (Union[Literal['svg'], Literal['img+b64+svg'], Literal['raw'], str], optional):
        Optionally outputs colored terminal output as an image.
        * If `rich` is a relative file path that ends with ".svg", the svg will
          be saved to that location and an img tag will be emitted. The path
          will be relative to the template file, which is specified on the
          command line. If the template is from stdin, the path will be relative
          to the current working directory (cwd) which is also specified on the
          command line.
        * If 'svg' a raw svg tag will be dumped into the markdown with the
          colored terminal output. Note that your markdown renderer may not
          support this.
        * If 'img+svg' a base64 encoded image will be dumped into the markdown
          with the colored terminal output.
        * If 'raw' the raw (uncolored) terminal output will be dumped into the
          markdown directly.
        * Defaults to 'raw.
      rich_alt (Optional[str], optional): The alt text for the img tag. Defaults
        to None.
      rich_bg_color (Optional[str], optional): The background color for the
        terminal output. Valid colors include anything valid for SVG colors. See
        <https://developer.mozilla.org/en-US/docs/Web/CSS/color>. Defaults to
        None (fully transparent).
      rich_term: (Optional[str], optional): Sets the TERM env var. Defaults to
        None, which uses whatever the env vars already have.
      rich_rows (int, optional): The number of rows to use for the terminal
        output. Doesn't seem to have much effect. Defaults to 24.
      rich_cols (int, optional): The number of columns to use for the terminal
        output. Defaults to 80.
      include_args (bool, optional): Should include the command that was run in
        the output? Defaults to True.
      _ctx (_Context): This is used by the system and is not available as an
        argument.

  Returns:
      Union[str, markupsafe.Markup]: Returns the output of the command.
  """

🌀 path

Used several times in ./README.md.jinja2.

Documentation:

def path(path: str,
         *,
         escape: bool = False,
         indent: Union[str, int, None] = None,
         indented: Union[str, int, None] = None,
         backtickify: Union[bool, str] = False,
         decomentify: Union[bool, Literal['nl']] = False,
         link: Optional[Literal['md', 'html']] = None,
         text: Optional[str] = None,
         _ctx: _Context) -> Union[str, markupsafe.Markup]:
  """Verifies that `path` exists, and just returns `path`.

  Unfortunately, I don't know how to use this inside a link, because the
  formatters will destroy it, and it cannot be put into a code block (as the url
  section of a link in markdown does not allow code blocks).

  Args:
      path (str): The path to verify.
      escape (bool, optional): Should use HTML entities escaping? Defaults to
        False.
      indent (Union[str, int, None], optional): Should indent? By how much, or
        with what prefix? Defaults to None.
      indented (Union[str, int, None], optional): Indents every line except the
        first. By how much, or with what prefix? Defaults to None.
      backtickify (Union[bool, str], optional): Should surround with backticks?
        With what language? Defaults to False.
      decomentify (Union[bool, Literal['nl']]): Assuming that you will be using
        HTML comments around this call, setting this to true will add
        correspondingcomments to uncomment the output. This allows you to have
        the Jinja2 call unmolested by markdown formatters, because they will be
        inside of a comment section. "nl" adds additional newlines after the
        newline delimiters. Defaults to False.
      link (Optional[Literal['md', 'html']], optional): If specified, will
        return a markdown or html link to the path. Defaults to None.
      text (Optional[str], optional): If specified, will use this text as the
        return value instead of the path. If used with link, will return this
        text as the link text instead of the path. Defaults to None.
      _ctx (_Context): This is used by the system and is not available as an
        argument.

  Returns:
      Union[str, markupsafe.Markup]: Just returns the path. If the path doesn't exist,
        it will raise an error.
  """

✅ Requirements

  • Linux-like environment
    • Why: Uses pexpect.spawn().
  • Python 3.8+
    • Why: Some dev dependencies require Python 3.8+.

Tested Platforms

  • WSL2 Ubuntu 20.04, Python 3.8.0.
  • Ubuntu 20.04, Python 3.8.0, 3.9.0, 3.10.0, 3.11.0, 3.12.0, tested in GitHub Actions workflow (build-and-test.yml).

🚸 Gotchas and Limitations

  • Security: This tool is NOT designed to be used with untrusted input. It is designed to be used with your own codebase. Even when using your own input, be careful that your own code won't be doing anything that might inadvertently include untrusted input.
  • Be careful to escape {{ and }}, or {% and %} or anything jinja2 is sensitive to, in the templates. You'll have to escape it properly for jinja2, which involves using {% raw %} and {% endraw %} tags.
  • Recursion: Snipinator doesn't directly support recursive inclusion of generated content. You can generate the contents of one file first, and include that generated content into another template. This would mean that you have to worry about order of generation.
  • Embedded Backticks: If there are backticks in the included snippet, it might ruin the backticks you have in your markdown. This is why backtickify parameter exists in the API, so that Snipinator provides the backticks, and it will detect if there are backticks in the snippet and use a different number of backticks on the entire snippet. So if the snippet contains ```My Snippet```, Snipinator will use ````language ```My Snippet``` ```` and this is a method that Markdown uses to allow embedded backticks inside a code block.
  • Formatting: The {{ }} used to surround the snippet calls will unfortunately be formatted by a Markdown formatter and make the call invalid. Workarounds:
    • Decommentify: Put the snippet call inside a HTML comment, then use decommentify parameter. See ./snipinator/examples/LONG-EXAMPLE.md.jinja2 for examples.
    • prettier formatter is pretty good at leaving the Jinja2 calls alone, especially if you don't have any spaces. This especially helps for markdown "reference-style links" that have Jinja2 calls in them generating part of the URL, mdformat will URL encode the Jinja2 calls, and/or split them on spaces, which is not what we want. prettier will leave them alone.
    • For code blocks: If you embed the snippet call in a code block, it will not be formatted. However, because of Embedded Backticks gotcha, (see above), this is not recommended, unless you know for sure that there are no embedded backticks.
    • If your formatter supports a comment that disabled formatting, you can surround the snippet call with that comment.
  • Editing the wrong file: When you have a template and a generated file, it is easy to edit the wrong file. To combat this:
    • Snipinator provides a warning at the top of the generated file to remind you that it is auto-generated.
    • Snipinator will optionally chmod the file for you to make it read-only.
  • Newlines: This program assumes LF newlines. I don't know if it will work for anything else.
  • Combining backtickify and indent: Doesn't make much sense, but if you do it, it will run backtickify first, then indent everything including the backticks.

🤏 Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

🔑 License

This project is licensed under the MIT License - see the ./LICENSE.md file for details.

🙏 Thanks

Main libraries used in Snipinator are:

🤝 Related Projects

Not complete, and not necessarily up to date. Make a PR (contributions) to insert/modify.

Project Stars Last Update Language Platform Similarity X Obviousness
mdx-js/mdx 16.8k 2024/04/17 JS N/A ⭐⭐⭐⭐⭐
zakhenry/embedme 222 2023/11/08 JS N/A ⭐⭐⭐⭐⭐
cmacmackin/markdown-include 95 2023/02/07 Python N/A ⭐⭐⭐⭐⭐
SimonCropp/MarkdownSnippets 23 2024/04/23 .NET N/A ⭐⭐⭐⭐⭐
endocode/snippetextractor 4 2014/08/16 C++ N/A ⭐⭐⭐⭐⭐
polywrap/doc-snippets 3 2023/09/26 JS N/A ⭐⭐⭐⭐⭐
JulianCataldo/remark-embed 2 2022/09/22 JS N/A ⭐⭐⭐⭐⭐
xrd/oreilly-snippets 2 2015/10/15 Ruby N/A ⭐⭐⭐⭐⭐
DamonOehlman/injectcode 1 2021/08/01 JS N/A ⭐⭐⭐⭐⭐
electrovir/markdown-code-example-inserter 1 2024/02/19 JS N/A ⭐⭐⭐⭐⭐
andersfischernielsen/Simple-Embedded-Markdown-Code-Snippets 1 2021/02/12 JS N/A ⭐⭐⭐⭐⭐
ildar-shaimordanov/git-markdown-snippet 0 2021/09/14 Perl N/A ⭐⭐⭐⭐⭐
teyc/markdown-snippet 0 2024/01/22 Powershell N/A ⭐⭐⭐⭐⭐
marc-bouvier-graveyard/baldir_markdown 0 2020/06/15 Python N/A ⭐⭐⭐⭐⭐
dineshsonachalam/markdown-autodocs 176 2022/09/19 JS GH Action ⭐⭐⭐⭐
tokusumi/markdown-embed-code 28 2022/01/05 Python VSCode Extension ⭐⭐⭐⭐
sammndhr/gridsome-remark-embed-snippet 2 2021/06/14 JS Gridsome ⭐⭐⭐⭐
NativeScript/markdown-snippet-injector 4 2019/01/24 JS N/A ⭐⭐⭐⭐
fuxingloh/remark-code-import-replace 0 2022/12/21 JS Remark? ⭐⭐⭐⭐
szkiba/mdcode 15 2014/02/12 Go N/A ⭐⭐⭐
devincornell/pymddoc 0 2023/12/01 Python Python ⭐⭐⭐
shiftkey/scribble (docs) 40 2013/08/08 .NET N/A ⭐⭐
calebpeterson/jest-transformer-test-md 2 2020/08/21 JS Jest Tests ⭐⭐
tjstankus/commitate 0 2014/05/29 Ruby N/A
GitHub Docs: Creating a permanent link to a code snippet N/A N/A N/A N/A
javierfernandes/markdown-exercises 1 2017/05/01 JS N/A
gatsby-remark-embed-snippet N/A (55k) 2024/01/23 JS Gatsby
ARMmbed/snippet 6 2021/08/05 Python N/A ?
drewavis/markdowninclude 1 2024/04/06 JS VSCode Extension ?
romnn/embedme 0 2024/04/18 Go N/A ?

🫡 Contributions

Development environment: Linux-like

  • For running pre.sh (Linux-like environment).

    • From ./.github/dependencies.yml, which is used for the GH Action to do a fresh install of everything:

      bash: scripts.
      findutils: scripts.
      grep: tests.
      xxd: tests.
      git: scripts, tests.
      xxhash: scripts (changeguard).
      rsync: out-of-directory test.
      expect: for `unbuffer`, useful to grab and compare ansi color symbols.
      jq: dependency for [yq](https://github.com/kislyuk/yq), which is used to generate
        the README; the README generator needs to use `tomlq` (which is a part of `yq`)
        to query `pyproject.toml`.
      
    • Requires pyenv, or an exact matching version of python as in .python-version (which is currently 3.8.0).

    • act (to run the GH Action locally):

      • Requires nodejs.
      • Requires Go.
      • docker.
    • Generate animation:

      • docker

Commit Process

  1. (Optionally) Fork the develop branch.
  2. Stage your files: git add path/to/file.py.
  3. bash ./scripts/pre.sh, this will format, lint, and test the code.
  4. git status check if anything changed (generated ./README.md for example), if so, git add the changes, and go back to the previous step.
  5. git commit -m "...".
  6. Make a PR to develop (or push to develop if you have the rights).

🔄🚀 Release Process

These instructions are for maintainers of the project.

  1. In the develop branch, run bash ./scripts/pre.sh to ensure everything is in order.
  2. In the develop branch, bump the version in ./pyproject.toml, following semantic versioning principles. Also modify the last_unstable_release and last_stable_release in the [tool.snipinator-project-metadata] table as appropriate. Run bash ./scripts/pre.sh to ensure everything is in order.
  3. In the develop branch, commit these changes with a message like "Prepare release X.Y.Z". (See the contributions section above).
  4. Merge the develop branch into the master branch: git checkout master && git merge develop --no-ff.
  5. master branch: Tag the release: Create a git tag for the release with git tag -a vX.Y.Z -m "Version X.Y.Z".
  6. Publish to PyPI: Publish the release to PyPI with bash ./scripts/deploy-to-pypi.sh.
  7. Push to GitHub: Push the commit and tags to GitHub with git push && git push --tags.
  8. The --no-ff option adds a commit to the master branch for the merge, so refork the develop branch from the master branch: git checkout develop && git merge master.
  9. Push the develop branch to GitHub: git push origin develop.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

snipinator-1.4.1.tar.gz (59.6 kB view hashes)

Uploaded Source

Built Distribution

snipinator-1.4.1-py3-none-any.whl (28.9 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page