mdtocf 1.1.0

Markdown files/directory publishing to Atlassian Confluence

Requirements

This Python package has been tested and designed for:

• Ubuntu 18.04 LTS
• Python 3.7.5 and several python libraries:
  • PickelDB v0.9
  • Mistune v2.0 Markdown Parser
  • Atlassian Python API v1.5

Please see requirements.txt for specific python packages/modules versions required.

Missing Features (Todo)

• Attachments (e.g. images, pdf, etc.)

Install

Download the package and prepare Python environment alternatives:

git clone "https://github.com/olafrv/mdtocf.git"
cd mdtocf
make virtualenv

Install the package for its use:

Note: If you skip virtual environment you should ensure using python >= 3.7

source venv/bin/activate         # Activate virtual environment (optional)
make install                     # Option 1: Use local package in ./mdtocf
make install-pypi                # Option 2: Install package from PyPI
mkdir -p ~/dbs                   # Create temporal database directory
deactivate                       # Deactivate virtual environment (if activated)

See an example code in mdtocf.py and the target test-publish inside Makefile show some parameters examples.

Publish using local script

Note: If you skip virtual environment you should ensure using python >= 3.7

source venv/bin/activate               # Virtual environment (if created)
PYTHON=$(make python-path)             # Used: venv/bin/python or $PATH (python3.7, python3 or python)
${PYTHON} -m mdtocf.mdtocf --help
${PYTHON} -m mdtocf.mdtocf \ 
    --confluenceUsername "[e-mail]" \
    --confluenceApiToken "****************" \
    --confluenceUrl "https://olafrv.atlassian.net" \
    --confluenceSpace "TEST" \
    --confluenceParentPageId "33114" \
    --confluencePageTitlePrefix "[Test] " \
    --markdownDir ./examples \
    --db ~/dbs/examples.db
deactivate                             # Deactivate virtual environment (if activated)

Publish using Docker (Image locally built)

make docker
docker run --rm -it mdtocf --help
docker run --rm -it \
    --mount type=bind,source="$(pwd)"/examples,target=/mdtocf/examples \
    --mount type=bind,source=~/dbs,target=/mdtocf/dbs \
    mdtocf \
    --confluenceUsername "[e-mail]" \
    --confluenceApiToken "****************" \
    --confluenceUrl "https://olafrv.atlassian.net"   \
    --confluenceSpace "TEST" \
    --confluenceParentPageId "33114" \
    --confluencePageTitlePrefix "[Test] " \
    --markdownDir "./examples" \
    --db ~/dbs/examples.db

Publish using Docker (Image downloaded from Github's Packages)

# Check <VERSION> in https://github.com/olafrv/mdtocf/packages 
export IMAGE=docker.pkg.github.com/olafrv/mdtocf/mdtocf:<VERSION> 
docker run --rm -it $IMAGE --help
docker run --rm -it \
    --mount type=bind,source="$(pwd)"/examples,target=/mdtocf/examples \
    --mount type=bind,source=~/dbs,target=/mdtocf/dbs \
    $IMAGE
    --confluenceUsername "[e-mail]" \
    --confluenceApiToken "****************" \
    --confluenceUrl "https://olafrv.atlassian.net"   \
    --confluenceSpace "TEST" \
    --confluenceParentPageId "33114" \
    --confluencePageTitlePrefix "[Test] " \
    --markdownDir ./examples \
    --db ~/dbs/examples.db

Output and Results

Output:

DEL => Id: 3409835, Title: [Test] Folder B
DEL => Id: 3409844, Title: [Test] 1
DEL => Id: 3409824, Title: [Test] Folder A
DEL => Id: 3409853, Title: [Test] Page AA
DEL => Id: att3409862, Title: example.png
DEL Att. => Title: example.png
DEL => Id: 3409867, Title: [Test] Example Page
UPD => Title: [Test] Folder A
Can't find '[Test] Folder A' page on the https://olafrv.atlassian.net/wiki!
IDX => Title: [Test] Folder B
Can't find '[Test] Folder B' page on the https://olafrv.atlassian.net/wiki!
UPD => Title: [Test] 1
Can't find '[Test] 1' page on the https://olafrv.atlassian.net/wiki!
UPD => Title: [Test] Page AA
Can't find '[Test] Page AA' page on the https://olafrv.atlassian.net/wiki!
UPD Att. => Title: example.png
UPD Att. => Title: example.png
UPD => Title: [Test] Example Page
Can't find '[Test] Example Page' page on the https://olafrv.atlassian.net/wiki!

The "Can't find..." means "not found but creating..." (Python Atlassian API).

Results in Confluence

Rendering and publishing ./examples produce the following final result in Confluence:

About Markdown Compatibility

This scripts depends on Mistune v2 Markdown Parser, compatible with CommonMark

The (optional) metadata heading in markdown (.md) files likes the one which follows below used by Hugo, it is not part of CommonMarkdown standard, but just a popular way of specify in YAML markdown metadata usable for external tools.

---
title: My Page Title
date: 2019-03-26T08:47:11+01:00
draft: true
chapter: true
kind: index
---

It is parsed and partially used by this script to organize the content in Attlasian Confluence. A test for this can be run:

make test-re

Uninstall

source venv/bin/activate         # Activate virtual environment (optional)
make uninstall                   # Remove installed package and dependencies
deactivate                       # Deactivate virtual environment (if activated)

Development & Testing

make python-version     # Print detected Python version (also after target "dev")
make python-path        # Print detected Python binary (also after target "dev")
make dev                # Virtualenv and install (./mdtocf)
make test-re            # Test markdown metadata regexp
make test-publish       # Publish ./examples to Atlassian
make test-docker        # Test docker image
make test-gihub-docker  # Test github docker package image
git commit -a           # After increment VERSION file content
#make github-package    # Already done by target "github-release"
#make pypi-live         # Already done by target "github-release"
make github-release     # Will trigger .github/workflows/mdtocf.yml
make clean              # Delete temporal dirs, files and docker images

References

Markdown

• https://spec.commonmark.org
• https://spec.commonmark.org/dingus/

Mistune v2

• https://github.com/lepture/mistune
• https://mistune.readthedocs.io/en/latest/

Python v3

• https://docs.python.org/3/
• https://docs.python.org/3/howto/regex.html
• https://pypi.org/project/atlassian-python-api/
• https://atlassian-python-api.readthedocs.io/confluence.html

Confluence and Storage Format (Cloud API)

• https://developer.atlassian.com/cloud/confluence/rest/
• https://confluence.atlassian.com/doc/confluence-server-documentation-135922.html
• https://confluence.atlassian.com/doc/confluence-storage-format-790796544.html
• https://confluence.atlassian.com/doc/macros-139387.html
• https://confluence.atlassian.com/conf59/code-block-macro-792499083.html
• https://confluence.atlassian.com/confkb/how-to-get-confluence-page-id-648380445.html
• https://confluence.atlassian.com/doc/delete-or-restore-a-page-139429.html