Profile manager of text processing pipelines: Pandoc filters, any text CLI filters. Atom+Markdown+Pandoc+Jupyter workflow, export to ipynb.
Project description
Pandoctools
Pandoctools is a combination of tools that help write reproducible markdown reports. They rely on Pandoc and Jupyter kernels.
Introduction articles:
- Best Python/Jupyter/PyCharm experience + report generation with Pandoc filters.
- Convenient and easily tweakable Atom+Markdown+Pandoc+Jupyter experience (can export to ipynb).
“Glueing” part of pandoctools is a profile manager of text processing pipelines. It stores short crossplatform bash scripts that define chain operations over text. They are mostly Pandoc filters but any CLI text filter is OK.
Update instructions
(Update instructions to v.1.4.2)
- Switch to bash profiles as batch profiles are no longer supported (and install bash if needed),
results=pandoc
was a misunderstanding. The right way to output Markdown is to use
from IPython.display import Markdown; Markdown('hello')
.- Import pandas, matplotlib and feather helpers from separate modules: matplotlibhelper, featherhelper, tabulatehelper,
- v1.4.2 is not backward compatible but profiles can be easily fixed. Uninstall Pandoctools before updating. Update your custom bash scripts as names and logic changed. References: Default_args, Default (profile), Default_pipe.
- Since v1.4.2 bash on Windows first rearched in the python environment, then in the $PATH, then
by path from config, then in the
%PROGRAMFILES%\Git
.
Contents
- Pandoctools
- Contents
- Notable parts of Pandoctools
- Examples
- Install
- Useful tips (reload imported modules in Hydrogen, R kernel, LyX)
- Alternatives to R Markdown (Markdown-based Literate Programming)
Notable parts of Pandoctools
- Pandoc, Jupyter, pandoc-crossref (dependence) - classical tools.
- Pandoctools CLI app: profile manager of text processing pipelines. It stores short bash scripts - called profiles - that define chain operations over text. They are mostly Pandoc filters but any CLI text filter is OK. Profiles can be used to convert any document of choise in the specified manner.
- Knitty (dependence): Knitty is a Pandoc filter and another CLI for Stitch/Knotr: reproducible report generation tool via Jupyter, Pandoc and Markdown. Insert python code (or other Jupyter kernel code) to the Markdown document and have code's results in the output document. Can even export to Jupyter ipynb notebooks. You can use ipynb-py-convert to convert .ipynb to .py to use with Knitty.
- SugarTeX (dependence): SugarTeX is a more readable LaTeX language extension and transcompiler to LaTeX.
- Pyppdf (dependence): Pyppeteer PDF. Prints html output to pdf via patched Pyppeteer.
- (optional) Tabulate Helper converts tabular data like Pandas dataframe to GitHub Flavored Markdown pipe table.
- (optional) Matplotlib Helper: custom helper to tune Matplotlib experience in Atom/Hydrogen and Pandoctools/Knitty.
- (optional) Feather Helper: concise interface to cache numpy arrays and pandas dataframes.
Pandoctools is a tool for converting markdown document. But we also need tools for writing markdown and deploying python/Jupyter code blocks.
And the best one for it is:
- Atom editor with plugins. It helps easily type Unicode, interactively run highlighed python/Jupyter code blocks and instantly see results (+ completions from the running Jupyter kernel), can convert basic pandoc markdown to html with live preview.
- Must have plugins: SugarTeX Completions, Unix Filter, Hydrogen, Markdown Preview Plus
Examples
Here are examples that demonstrate converting documents:
- from markdown
.md
with Jupyter python code blocks, SugarTeX math and cross-references toipynb
notebook. - from Hydrogen/python notebook
.py
with Atom/Hydrogen code cells, Knitty markdown incerts (again with SugarTeX math and cross-references) to.ipynb
notebook.
Examples are given for to .ipynb conversion but Pandoctools surely capable of conversion to .html, .pdf, .md.md or any Pandoc output format.
Extras:
- If you need to capture Matplotlib plots please see matplotlibhelper (the approach showed in examples there can be used with other plot libraries).
- If you need to autonumber sections see pandoc-crossref or this SE question
Install
Via conda
- (on Windows) Install 64-bit Git together with Bash,
- Install 64-bit Miniconda3 (≥3.6),
- (on Windows) Creating "pandoctools" conda environment:
call activate root conda update conda conda create -n pandoctools -c defaults -c conda-forge "pandoctools>=1.4.4" call activate pandoctools pandoctools-ready
- (on Unix) Creating "pandoctools" conda environment:
source activate root conda update conda conda create -n pandoctools -c defaults -c conda-forge "pandoctools>=1.4.4" source activate pandoctools pandoctools-ready
- The significant commands are the following:
conda install -c defaults -c conda-forge "pandoctools>=1.4.4" pandoctools-ready
But it's recommended to create a dedicated conda environment for the Pandoctools.
Via pip
- (on Windows) Install Git together with Bash,
- Install Pandoc (maybe pip would also install it but I'm not sure),
- Install latest stable pandoc-crossref (compatible with pandoc version) to the dedicated virtual environment's
.\Scripts
(Windows) or./bin
(Unix) folder. - Install Pandoctools:
pip install pandoctools pandoctools-ready
Useful tips (reload imported modules in Hydrogen, R kernel, LyX)
Alternatives to R Markdown (Markdown-based Literate Programming)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.