mell 2.0.2

A Metaprogramming Logic Layer designed to generate anything from template files

Mell

Mell is a Metaprogramming Logic Layer designed to generate anything from template files.

Why do I need this?

There are moments in life that you may find yourself needing to customize an entire project, not only an e-mail or a single webpage in a backend response. These are the moments that you may want to use mell. So far, I have used it in the following situations:

• Generating VHDL code for a static neural network, variating a few parameters. Mell is much more flexible than the generic atributes available in the language.
• Generating model classes for an ORM. I defined the model classes and relations as metadata and asked mell to generate them for me in C#.
• Generating resumes to send to job applications. I use latex to generate my resume. Instead of changing multiple configuration files in my latex project I change only a single metadata file, customizing the letter, company name, color, and so on. Mell generates a latex source that I compile to PDF.

Conceptually, you may use mell in two directions. A metadata used with multiple styles, representing different projections of the same thing, or a single style used by multiple metadata, creating different things with the same look.

As an example, consider the situation that we have metadata describing a mobile app and a style that can render an Android app. We could change the metadata and generate different Android apps, or we could change the style and render the same app on different platforms, like an IOS app.

When should I not use this?

I don't recommend using this if you are not confortable programming in the stack you are generating code to, as the template files may elevate your project complexity. This works better if are at a point where you feel like everything is just the same with a few different parameters. These few parameters will likely become your metadata when you use mell.

You may constantly find that mell may be replaced by reflection or similar concepts on your programming language of choice. This is true and the answer to "which of them is better?" depends on your requirements. Mell may be more efficient as many logic rules are evaluated during rendering time, whereas reflection adds more complexity during the program execution. However, reflection is also simpler and more flexible during runtime. Mell is also more suitable to generate configuration files based on global parameters, like a kubernetes' configuration file or a django's settings.

Concepts

To use this library you must understand a few concepts. These are:

• metadata: These is data describing what we want to generate. We use json format for it.
• style: This is the set of features that are necessary to transform the metadata into something. It is composed by templates, assets, static files, plugins, and logic rules.
• generated folder: This is where the rendered files will be saved and you must never change these files. However, you may want to execute or compile them if you are generating a webserver or a latex template, for instance.

A style is composed of the following concepts:

• template: file snippets with a few missing parts. Mell will fill these parts with metadata when it generates the files and copy them to the generated folder, keeping the original path structure.
• static: files that will not be modified. Mell will copy them directly to the generated folder, keeping the original path structure.
• asset: files used by your style that are not automatically used by mell.
• plugin: Scripts that will be executed by mell. These usually access the files in the asset folder.
• logic: Scripts that will be executed in order by mell. These are used to validate and extend the metadata.

Basic folder structure

The following table describes the folders used by mell.

Folder	Description
<root>	The base folder that contains all folders described here
<root>/style	The base folder for template, static, plugin, and asset
<root>/generate	this will hold the generated data, never edit this folder
<root>/meta	holds all metadata as json files
<root>/style/template	holds the template files that will be automatically rendered and written to the generated folder
<root>/style/static	contains static files that will be copied as they are to the generate folder
<root>/style/plugin	contains scripts that will be executed by mell. Use these to render multiple files from templates in the asset folder
<root>/style/asset	contains template and other files that are not automatically used by mell. They may be used by plugins or other tools
<root>/style/logic	contains scripts used automatically executed by mell to transform the metadata

If you want to create a new project using this structure with the root folder named testing_mell, you can use the following command.

mell --new testing_mell

How to install / uninstall it?

# To install
pip install mell

# To upgrade
pip install --upgrade mell

# To uninstall
pip uninstall mell

After installing the module you should be able to access the command mell via terminal. If it doesn't, you may try the following options: (1) check that your $PATH variable includes the local bin directory that pip uses; (2) install it in a virtual environment, like virtualenv; or (3) try to install it at the system level, running pip as root;

Generating Hello Worlds

This will demonstrate how to use mell in a simple use case, changing the language for a static interface. You may be thinking now, "what a naive example, most web frameworks have much more powerful internationalization tools and I would never use it for that". Indeed, me neither. I used this in my resumes in latex, though. By doing this I had a different metadata for each place I would apply, some in portuguese, some in english, some specific for each position. Another benefit is that I could update the resume by changing only a single file. I could generate old ones again, they had their own metadata, and so on.

Step 1. Create a new mell project

Execute the following command to create the standard structure.

mell --new template_test
cd template_test

Now, we will create a second style, named style2. We will make style render a program that prints a message using python and style2 will render a program that prints a message using cpp.

mell --new-style style2

Step 2. Create the metadata files

This is the content for <root>/meta/en.json.

{
    "message": "Hello World"
}

This is the content for <root>/meta/pt.json.

{
    "message": "Olá Mundo" 
}

Step 3. Create the contents for style and style2

This is the content for <root>/style/template/main.py:

print("|= meta.message =|")

This is the content for <root>/style2/template/main.cpp.

#include <iostream>

int main()
{
    std::cout << "|= meta.message =|" << std::endl;
    return 0;
}

Step 4. Generating the programs

If we only pass the metadata name to mell, it will use the default folders for style and generate folder. The following program will use the metadata file <root>/meta/en.json, the style folder <root>/style, and render everything to <root>/generate.

# This must be executed inside the <root> folder
mell en

The command above will generate the following content in <root>/generate/main.py

print("Hello World")

The following is the program generated if we execute mell pt.

print("Olá Mundo")

To generate the cpp program we need to tell mell to use the style in style2. This can be done with the following command.

# This must be executed inside the <root> folder
mell --style style2 en

The command above will generate the following content in <root>/generate/main.cpp

#include <iostream>

int main()
{
    std::cout << "Hello World" << std::endl;
    return 0;
}

The following is the program generated if we execute mell --style style2 pt.

#include <iostream>

int main()
{
    std::cout << "Olá Mundo" << std::endl;
    return 0;
}

Step 5. Improving the structure

If you pretend to use just one style, the default folder name is fine, but for multiple style it is better to rename and put them into something more representative, like styles/cpp and styles/python. You may also want to save the output to different output folders. You can do this using the parameter --generate. The following is an example of all four combinations using the new style structure.

# Create new folders to keep our styles
mkdir styles

# Rename the style folders
mv style styles/python
mv style2 styles/cpp

# Generate the programs and save them in different folders
mell --style styles/cpp --generate generates/cpp/en en
mell --style styles/cpp --generate generates/cpp/pt pt
mell --style styles/python --generate generates/python/en en
mell --style styles/python --generate generates/python/pt pt

This is how the project looks like in the end.

.
├── generates
│   ├── cpp
│   │   ├── en
│   │   │   └── main.cpp
│   │   └── pt
│   │       └── main.cpp
│   └── python
│       ├── en
│       │   └── main.py
│       └── pt
│           └── main.py
├── meta
│   ├── en.json
│   └── pt.json
└── styles
    ├── cpp
    │   ├── asset
    │   ├── plugin
    │   ├── static
    │   └── template
    │       └── main.cpp
    └── python
        ├── asset
        ├── plugin
        ├── static
        └── template
            └── main.py

The Pipeline

Sometimes it is important to understand the order in which the operations are executed. The list below describe these operations in their standard order.

1. Load metadata
2. Execute logic scripts
3. Actions to generate the output:

4. Clean output folder [clean]
5. Copy static files [static]
6. Render templates and copy them to output [template]
7. Execute plugin scripts [plugin]

Loading the metadata and executing the logic scripts is always executed first and can't be changed. The list of actions to generate the output, though, can be modified or canceled. To modify the list of actions we use the command --do <action_name>. By default, mell will execute all of them but if we use the parameter --do it will execute only the tasks it received. A few examples are listed bellow.

# These two calls are the same
mell --do clean --do static --do template --do plugin data
mell data

# This will only generate output files based on the template folder
mell --do template data

# This will only clean the generated folder
mell --do clean data

# The word 'nothing' is a special keyword. It will not do any action but will still load the metadata and execute logic scripts. This is useful when used with -v (verose mode) or --show-metadata (display the metadata after executing the logic scripts).
mell --do nothing data
mell --do nothing -v data
mell --do nothing --show-metadata data

Tutorials

• Metadata - Explains how the metadata work and how to inherit and extend from existing metadata;
• Template - Explains the template syntax and how to customize it;
• Plugin and Asset - Shows how to use a plugin script to generate multiple output files from a single template;
• Logic - Shows how to extend extend the input metadata, generating more metadata and preventing complex rules in template files.

List of useful command options

These are a few examples of common operations.

# This will create a folder named project_name with the recommended root folder structure
mell --new project_name

# This will create a folder named style2 with the recommended style folder structure (use it inside the root directory to keep things organized)
mell --new-style project_name

# Create a new plugin file as <root>/style/plugin/plugin_name.py
mell --new-plugin plugin_name

# Create a new logic file as <root>/style/logic/<timestamp>.logic_name.py
mell --new-logic logic_name

# Display the version number and exit
mell --version

# Display more info during execution (verbose mode)
mell -v en

# Display less info during execution (quiet mode)
mell -q en

# Specify what we want to generate
mell --do clean --do static --do template --do plugin en

# Only clean the output folder
mell --do clean en

# Only generate files from templates
mell --do template en

# Specify a different style folder. This will make mell use the folders template, asset, plugin, and static that inside it.
mell --style style2 en

# Specify a different generate folder. This is useful if you have multiple styles and want to generate different things on different folders.
mell --generate generate2 en

# An example with custom style names, distinct output folders and two metadata files. We are assuming the style folders are on local directory and named python and cpp.
mell --style styles/python --generate generates/python/en en
mell --style styles/python --generate generates/python/pt pt
mell --style styles/cpp --generate generates/cpp/en en
mell --style styles/cpp --generate generates/cpp/pt pt

Source Code

The source code is available here