rolumns 0.1.8

Squishes data into rows and columns

Rolumns

Rolumns is a Python package for squishing data into rows and columns.

Full documentation is online at rolumns.dev.

Installation

Rolumns requires Python 3.7 or later.

pip install rolumns

Usage

A simple dataset

Say we have this data that we want to transform into a table with "Name" and "Email" columns:

{
    "name": "Robert Pringles",
    "email": "[e-mail]"
}

We'll start by creating and adding our columns to a rolumns.Columns instance.

The first argument of add is the name and the second is the value key:

import rolumns

data = {
    "name": "Robert Pringles",
    "email": "[e-mail]",
}

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", "email")

To create a list of rows that make up a table, we pass those columns into a rolumns.renderers.RowsRenderer instance then call render:

renderer = rolumns.renderers.RowsRenderer(columns)
rows = renderer.render(data)

This creates an iterable list of lists that can be passed into something like openpyxl to emit an Excel spreadsheet:

[
    ["Name",            "Email"],
    ["Robert Pringles", "[e-mail]"],
]

You can also render to Markdown via the rolumns.renderers.MarkdownRenderer:

renderer = rolumns.renderers.MarkdownRenderer(columns)
markdown = renderer.render(data)

Name	Email
Robert Pringles	[e-mail]

Multiple rows

Since single-row tables are boring, you can pass an iterable list into the renderers to get multiple rows:

data = [
    {
        "name": "Robert Pringles",
        "email": "[e-mail]",
    },
    {
        "name": "Daniel Sausage",
        "email": "[e-mail]",
    },
    {
        "name": "Charlie Marmalade",
        "email": "[e-mail]",
    },
]

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", "email")

renderer = rolumns.renderers.MarkdownRenderer(columns)
rows = renderer.render(data)

Name	Email
Robert Pringles	[e-mail]
Daniel Sausage	[e-mail]
Charlie Marmalade	[e-mail]

Reading child objects

Say our data has been extended with an "address" key. To include a "Planet" column in our output, set its path to address.planet to drill into the address object:

data = [
    {
        "name": "Robert Pringles",
        "email": "[e-mail]",
        "address": {
            "planet": "Earth",
        },
    },
    {
        "name": "Charlie Marmalade",
        "email": "[e-mail]",
        "address": {
            "planet": "Mars",
        },
    },
]

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", "email")
columns.add("Planet", "address.planet")

renderer = rolumns.renderers.MarkdownRenderer(columns)
rows = renderer.render(data)

Name	Email	Planet
Robert Pringles	[e-mail]	Earth
Charlie Marmalade	[e-mail]	Mars

Column sets and repeating data

You can add as many columns to an rolumns.Columns instance as you like, and you can add at most one repeating group too.

For example, let's say our data has been extended to include employment history:

[
    {
        "name": "Robert Pringles",
        "email": "[e-mail]",
        "positions": [
            {
                "year": 2008,
                "title": "Founder",
            },
            {
                "year": 2009,
                "title": "CEO",
            },
        ],
    },
    {
        "name": "Charlie Marmalade",
        "email": "[e-mail]",
        "positions": [
            {
                "year": 2009,
                "title": "Engineer",
            },
            {
                "year": 2010,
                "title": "Senior Engineer",
            },
            {
                "year": 2011,
                "title": "CTO",
            },
        ],
    },
]

If we want to keep "Name" and "Email" in our table, but repeat those rows for each employment position, we start by adding the first two columns exactly as before:

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", "email")

To add the repeating group, we append a new rolumns.Columns instance to our root set by calling add_group. The parameter describes where to find the repeating data, and the columns we add to that new set describe where to find their values within that group:

positions = columns.add_group("positions")
positions.add("Year", "year")
positions.add("Title", "title")

The full code sample looks like this:

data = [
    {
        "name": "Robert Pringles",
        "email": "[e-mail]",
        "positions": [
            {
                "year": 2008,
                "title": "Founder",
            },
            {
                "year": 2009,
                "title": "CEO",
            },
        ],
    },
    {
        "name": "Charlie Marmalade",
        "email": "[e-mail]",
        "positions": [
            {
                "year": 2009,
                "title": "Engineer",
            },
            {
                "year": 2010,
                "title": "Senior Engineer",
            },
            {
                "year": 2011,
                "title": "CTO",
            },
        ],
    },
]

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", "email")

positions = columns.add_group("positions")
positions.add("Year", "year")
positions.add("Title", "title")

renderer = rolumns.renderers.MarkdownRenderer(columns)
rows = renderer.render(data)

Name	Email	Year	Title
Robert Pringles	[e-mail]	2008	Founder
Robert Pringles	[e-mail]	2009	CEO
Charlie Marmalade	[e-mail]	2009	Engineer
Charlie Marmalade	[e-mail]	2010	Senior Engineer
Charlie Marmalade	[e-mail]	2011	CTO

Naturally, groups can also be grouped:

data = [
    {
        "name": "Robert Pringles",
        "email": "[e-mail]",
        "positions": [
            {
                "year": 2008,
                "title": "Founder",
                "awards": [
                    {"award": "Best Coffee"},
                ],
            },
            {
                "year": 2009,
                "title": "CEO",
                "awards": [
                    {"award": "Fastest Doughnut Run"},
                    {"award": "Tie of the Year"},
                ],
            },
        ],
    },
    {
        "name": "Charlie Marmalade",
        "email": "[e-mail]",
        "positions": [
            {
                "year": 2009,
                "title": "Engineer",
                "awards": [
                    {"award": "Engineer of the Decade"},
                    {"award": "Least Security Vulnerabilities"},
                ],
            },
            {
                "year": 2010,
                "title": "Senior Engineer",
                "awards": [
                    {"award": "Best Code Reviews"},
                ],
            },
            {
                "year": 2011,
                "title": "CTO",
                "awards": [
                    {"award": "Cloud Uplift of the Century"},
                ],
            },
        ],
    },
]

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", "email")

positions = columns.add_group("positions")
positions.add("Year", "year")
positions.add("Title", "title")

awards = positions.add_group("awards")
awards.add("Award", "award")

renderer = rolumns.renderers.MarkdownRenderer(columns)
rows = renderer.render(data)

Name	Email	Year	Title	Award
Robert Pringles	[e-mail]	2008	Founder	Best Coffee
Robert Pringles	[e-mail]	2009	CEO	Fastest Doughnut Run
Robert Pringles	[e-mail]	2009	CEO	Tie of the Year
Charlie Marmalade	[e-mail]	2009	Engineer	Engineer of the Decade
Charlie Marmalade	[e-mail]	2009	Engineer	Least Security Vulnerabilities
Charlie Marmalade	[e-mail]	2010	Senior Engineer	Best Code Reviews
Charlie Marmalade	[e-mail]	2011	CTO	Cloud Uplift of the Century

Translating values

Sometimes, you need to translate a value before it's rendered; perhaps to convert a string to a Python datetime instance, or to format a number as currency.

So, for example, let's say we want to censor the email addresses we're writing out to the table.

First, we write a function that takes a rolumns.translators.TranslationState state and returns the translated value:

def censor(state: rolumns.translators.TranslationState) -> str:
    value = str("" if state.value is None else state.value)

    if len(value) < 3:
        return "".join(["*" * len(value)])

    return value[0] + "".join(["*" * (len(value) - 2)]) + value[-1]

Then, when we add the "Email" column to the column set, we pass an explicit rolumns.Source instance rather than just the string path:

# columns.add("Email", "email")
columns.add("Email", Source("email", translator=censor))

Here's the full code sample:

import rolumns
import rolumns.renderers
import rolumns.translators

data = [
    {
        "name": "Robert Pringles",
        "email": "[e-mail]",
    },
    {
        "name": "Daniel Sausage",
        "email": "[e-mail]",
    },
    {
        "name": "Charlie Marmalade",
        "email": "[e-mail]",
    },
]

columns = rolumns.Columns()
columns.add("Name", "name")
columns.add("Email", rolumns.Source("email", translator=censor))

renderer = rolumns.renderers.MarkdownRenderer(columns)
rows = renderer.render(data)

Name	Email
Robert Pringles	b**************p
Daniel Sausage	d****************p
Charlie Marmalade	c******************p

A collection of translators are provided within rolumns.translators.

User-defined fields

User-defined fields allow you to pivot a table.

For example, given this date:

[
    {
        "name": "Robert Pringles",
        "address": "Earth",
        "email": "[e-mail]",
        "title": "CEO"
    },
    {
        "name": "Daniel Sausage",
        "address": "Mars",
        "email": "[e-mail]",
        "title": "Head Chef"
    },
    {
        "name": "Charlie Marmalade",
        "address": "Pluto",
        "email": "[e-mail]",
        "title": "CTO"
    }
]

...we already know how to create table like this:

Name	Address	Email	Title
Robert Pringles	Earth	[e-mail]	CEO
Daniel Sausage	Mars	[e-mail]	Head Chef
Charlie Marmalade	Pluto	[e-mail]	CTO

...but we might prefer to pivot the same data like this:

Name	Property	Value
Robert Pringles	Address	Earth
Robert Pringles	Email	[e-mail]
Robert Pringles	Title	CEO
Daniel Sausage	Address	Mars
Daniel Sausage	Email	[e-mail]
Daniel Sausage	Title	Head Chef
Charlie Marmalade	Address	Pluto
Charlie Marmalade	Email	[e-mail]
Charlie Marmalade	Title	CTO

To achieve this, we start by creating a typical column set and adding the "Name" column:

columns = rolumns.Columns()
columns.add("Name", "name")

Now we'll create a rolumns.groups.ByUserDefinedFields instance and add our user-defined fields. The first argument describes the field name and the second argument describes the path to the value:

group = rolumns.groups.ByUserDefinedFields()
group.append("Address", "address")
group.append("Email", "email")
group.append("Title", "title")

Now we'll pass this grouping to the root column set, where in previous examples we've passed the path to the collection to iterate:

udfs = columns.add_group(group)

To add the field names and values to the column set, we call the same add function as before. The paths are specified by the ByUserDefinedFields.NAME and ByUserDefinedFields.VALUE constants:

udfs.add("Property", rolumns.groups.ByUserDefinedFields.NAME)
udfs.add("Value", rolumns.groups.ByUserDefinedFields.VALUE)

The full code sample looks like this:

data = [
    {
        "name": "Robert Pringles",
        "address": "Earth",
        "email": "[e-mail]",
        "title": "CEO",
    },
    {
        "name": "Daniel Sausage",
        "address": "Mars",
        "email": "[e-mail]",
        "title": "Head Chef",
    },
    {
        "name": "Charlie Marmalade",
        "address": "Pluto",
        "email": "[e-mail]",
        "title": "CTO",
    },
]

columns = rolumns.Columns()
columns.add("Name", "name")

group = rolumns.groups.ByUserDefinedFields()
group.append("Address", "address")
group.append("Email", "email")
group.append("Title", "title")

udfs = columns.add_group(group)
udfs.add("Property", rolumns.groups.ByUserDefinedFields.NAME)
udfs.add("Value", rolumns.groups.ByUserDefinedFields.VALUE)

renderer = rolumns.renderers.MarkdownRenderer(columns)
rows = renderer.render(data)

Name	Property	Value
Robert Pringles	Address	Earth
Robert Pringles	Email	[e-mail]
Robert Pringles	Title	CEO
Daniel Sausage	Address	Mars
Daniel Sausage	Email	[e-mail]
Daniel Sausage	Title	Head Chef
Charlie Marmalade	Address	Pluto
Charlie Marmalade	Email	[e-mail]
Charlie Marmalade	Title	CTO