SSB Publish Markdown

Features

• Write SSB articles using Quarto Markdown (qmd) files.
• Create SSB components using Python:
  • Highchart
  • Factbox
• Insert components in article using simple Markdown syntax.

Requirements

• A VSCode or Jupyter service in Dapla Lab.
• An SSB project.

Installation

poetry add ssb-pubmd

Latest development version:

poetry add ssb-pubmd@latest --source testpypi --allow-prereleases

Usage

poetry run ssb-pubmd create "My article title"

Follow further instructions in template file, and see the Reference Guide for details on creating components.

Contributing

Setup

1. Install uv (python and package manager):

   curl -LsSf https://astral.sh/uv/install.sh | sh
   

   (See link for installation on Windows.)

2. Clone this repo.

   git clone https://github.com/statisticsnorway/ssb-pubmd.git
   

Run tests

From the root folder (where pyproject.toml is), run:

uv run pytest

Testing the whole integration

• When you push changes to a branch, the package is published to Test PyPI by the release workflow.
• The test package can then be installed from a Dapla service and tested directly.

Steps

Use the following steps to test the most recently pushed change (to any branch):

1. Open a Jupyter service in Dapla Dev.

2. Open a Jupyter terminal and run the following commands:

   ssb-project create my-project
   cd my-project
   poetry source add testpypi https://test.pypi.org/simple/ -p explicit
   poetry add "pandas<3"
   poetry add ssb-pubmd@latest --source testpypi --allow-prereleases
   poetry run ssb-pubmd create my-article
   

3. You can then continue testing in the same Dapla service. If you push new changes, you need to fetch these changes in the Dapla service with the following command:

   poetry update ssb-pubmd
   

   Note that it takes about a minute (sometimes longer) for the PyPI Test package to be updated after pushing a change.

Architecture

Dependency graph:

graph LR
   subgraph driving[Interface]
      cli[cli]
   end
   subgraph core[Core]
      project[create_project]
      article[render_article]
      docprocessor[Document processor]
      docpublisher[Document publisher]
   end
   subgraph driven[Adapters]
      cmsclient[Cms client]
      storage[Storage]
   end
   cmsservice[Cms service]

   cli --> project
   cli --> article --> docpublisher

   article --> docprocessor

   docpublisher --> cmsclient
   docpublisher --> storage

   cmsclient --> cmsservice

   classDef empty width:0px,height:0px;

Note: Cms Service refers to statisticsnorway/ssbno-app-pubmd.

General document flow:

1. The user asks to render their notebook file through the command-line interface.
2. The core method render_article uses the document processor to parse the file into a general Document object.
3. The Document object is passed into the document publisher, which extracts the content and passes it to the CMS client.
4. The CMS client sends the content to the CMS service and returns the parsed response.
5. The document processor stores the response, so that the id and path can be reused in future requests (to modify existing CMS content instead of creating new content).

Flow of components:

1. Components are defined programmatically by the user in the notebook file.
2. When the notebook is executed by the document processor, the create_component function (from __init__.py) stores the component data in a temporary storage file, which is then added to the Document object. The temporary storage file is then deleted, to ensure sensitive data is not stored on disk.
3. The document publisher first extracts all the components from the Document objects and sends these to the Cms Client. During this process, the Document object is also modified so that components in the document tree are replaced with Cms references.
4. Finally the whole document is extracted (as html) and sent to the Cms client, with correct component references in the html.

Updates to Mimir content types

This package exposes a Python interface to mimir content types; see the source file interface/models.py. When mimir types change, the corresponding Python interface can be updated following these steps.

Adding a new field

There are two simple steps of adding an extra field to an exposed content type:

1. Decide how the extra field should be exposed to the user.

   • The exposed models are defined in interface/models.py.

   • Example of adding a required string field:

     from pydantic import BaseModel
     class MyModel(BaseModel):
          extra_field: str
     

   • If the string field should be optional, a default value is given:

     from pydantic import BaseModel
     class MyModel(BaseModel):
          extra_field: str | None = None
          another_field: str = "default-string"
     

   • See existing models for more examples. Remember to add a description of the field in the documentation string.

2. Add the extra field to the payload (that will be sent to the CMS).

   • The payload is created in the source file adapters/cms_client/mimir.py.

   • If we have added a field to the Highchart model, we would add it to the payload in the method _create_highchart_payload.

   • The payload must match the Mimir content type. For instance, Highchart has an xlabel field which should be given as xAxisLabel in the payload.

     payload["xAxisTitle"] = data.xlabel
     

   • The easiest way to see how the payload should look is to use the Content Viewer in Content Studio, or the generated Enonic types.

License

Distributed under the terms of the MIT license, SSB Publish Markdown is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Credits

This project was generated from Statistics Norway's SSB PyPI Template.