Yet another makefile tool. This one is meant to be super fast, super easy and crossplatform while leaving full power to the user at any time.

These details have been verified by PyPI

Project links

GitHub Statistics

Maintainers

mactul

These details have not been verified by PyPI

License
- OSI Approved :: Apache Software License
Operating System
- OS Independent
Programming Language
- Python :: 3

Project description

PowerMake

PowerMake

What is PowerMake?

Powermake is a utility that compiles C/C++/AS/ASM code, just like Make, Ninja, cmake, or xmake.

His goal is to give full power to the user, while being cross-platform, easier to use than Make, and faster than Ninja.

For which project is PowerMake suitable?

PowerMake was specifically designed for complex projects that have very complicated compilation steps, with a lot of pre-built tasks that need to be compiled on multiple operating systems with different options.

Advantages of PowerMake

Extremely fast:
- PowerMake is faster than Ninja/make/xmake when building a project for the first time.
- There are still some improvements to do to detect that there is nothing to do for a huge codebase because PowerMake doesn't store hidden dependencies (header files). But with less than 2000 files, this step is almost instant.
Cross-Platform:
- PowerMake can detect the compiler installed on your machine and give you an abstraction of the compiler syntax.
  - This currently works well with GCC/G++/Clang/Clang++/Clang-CL/MSVC/NASM, but other compilers will be added.
- Because it's written in python it works on almost all machines and you can always write the compilation instructions for your machine and your compiler.
Gives you complete control of what you are doing. Nothing is hidden and any behavior can be overwritten.

Disadvantages of PowerMake

PowerMake is very young so it changes a lot with each version and you may have to write some features by yourself (the whole point of PowerMake is that you can write missing features).
Because PowerMake gives you full control, the tool can't know what you are doing during the compilation step. For example, if we want to import dependencies from another PowerMake, the only thing we can do for you is run the PowerMake where it stands and scan its output directory. This works well but has some limitations...

Philosophy

All other Make-like utilities that I know parse a file to understand directives from it.

PowerMake does the opposite. You write a python script, you do whatever you like in this script and you call PowerMake functions to help you compile your code.
This gives you complete control; you can retrieve files from the web, read and write files, and even train a Neural Network if you want, and at any time you can use Powermake functions to help you in your compilation journey.

Installation

pip install -U powermake

Quick Example

This example compiles all .c and .cpp files that are recursively in the same folder as the python script and generate an executable named program_test

[!WARNING]
PowerMake calculates all paths from its location, not the location where it is run.
For example, python ./folder/makefile.py will do the same as cd ./folder && python ./makefile.py

[!NOTE]
In this documentation, we often assume that your makefile is named makefile.py, it makes things easier to explain. Of course, you can name your makefile the name you like the most.

import powermake


def on_build(config: powermake.Config):

    files = powermake.get_files("**/*.c", "**/*.cpp")

    objects = powermake.compile_files(config, files)

    print(powermake.link_files(config, objects))


powermake.run("program_test", build_callback=on_build)

See more examples

Documentation

[!NOTE]
This documentation is not complete, if you struggle to do something, do not hesitate to ask a question in the discussions section, it may be that the feature you search for is undocumented.

Command line arguments

To benefit from the command line parser, you have to use the powermake.run function.

If no arguments are passed through the command line, the default behavior is to trigger the build callback.
You can also write python makefile.py build, python makefile.py clean, or python makefile.py install [install_location] to trigger one of the three different callbacks.

There is also the python makefile.py config command, which doesn't trigger a callback but enters into an interactive mode for editing a configuration file.

Alternatively, you can also use the option -b or --build, -c or --clean, -i or --install, and -f or --config.
This alternative has a great advantage: you can combine multiple tasks. For example, running python makefile.py -bci will first trigger the clean callback, then the build callback, and finally the install callback.

[!IMPORTANT]
The order will always be config -> clean -> build -> install.

You can also replace the -b argument with -r (using -br does the same as -r) and this will force the makefile to recompile everything, without trying to figure out which file needs to be recompiled.

There are many more options you can add such as -d (--debug), -q (--quiet), -v (--verbose), etc...

All these options can be listed by running python makefile.py -h or if you haven't created a makefile yet, by directly calling the module python -m powermake -h or just powermake -h if the pip installation is in your path.

[!IMPORTANT]
While python makefile.py install and python makefile.py --install takes the install_location as an optional argument, this argument has been disabled with the -i option, because writing -bic would have triggered the install callback with the location c

Toolchain detection

PowerMake infers the various toolchain programs to be used using everything it knows.
Most of the time, just setting up the C compiler configuration (or the C++ compiler or the linker, etc...) will be sufficient for PowerMake to determinate the whole toolchain.

[!NOTE]
Only the unspecified fields are inferred, the field explicitly assigned in the json configuration (see powermake.Config) are left unchanged. The only exception is when CC, CXX or LD env variables are specified (see below)

The environment variables CC, CXX and LD can be used to overwrite the C compiler, C++ compiler and linker tools path.
For example, the command below will compile using the afl-* toolchain. The C++ compiler and the linker are inferred from the C compiler (but only if they are not specified in the json configuration).

CC=afl-gcc python makefile.py -rvd

This is especially useful to quickly compile with a different toolchain. For example if you want to exceptionally compile an executable for Windows using Linux:

CC=x86_64-w64-mingw32-gcc python makefile.py -rvd

PowerMake flags translation

When this is possible, PowerMake tries to translate C/C++/AS/ASM/LD flags.

Most flags are unknown to PowerMake, in this case they are simply transmitted to the compiler and it's your job to ensure the compatibility with the different targets through the use of if/else blocks.

However, the most common flags are automatically translated by PowerMake.
Their is also some flags that PowerMake defines that doesn't exist in any compiler, these are set of useful flags for a situation.

Here is the list of flags translated by PowerMake:

[!NOTE]
Only compiler flags are listed, they are also translated for the linker but most of the time this ends up being just the removal of the flag.

PowerMake Flag	Description
-w	Inhibit all warning messages.
-Werror	Make all warnings into errors.
-Wall	Activate warnings about all constructs that are unlikely to be intended.
-Wextra	Enable flags that are likely to catch bugs even though they may warn on perfectly valid code.
-Wpedantic	Warn when the code isn't ISO (typically when C/C++ extension are used).
-pedantic	Same a -Wpedantic
-Wswitch	Warn when a switch on an enum lacks a case (enabled by -Wall)
-Wswitch-enum	Like -Wswitch but warns even if their is a default case
-fanalyzer	When supported by the compiler, run the code into a static analyzer to detect some bugs.
-Weverything	Enable as most warning as possible, even the noisy and irrelevant ones.
-Wsecurity	Enable all warnings that have a little chance to catch a security issue.
-O0	Disable all optimizations.
-Og	Enable optimizations that don't interfere with the debugger. This is better than -O0 for debugging because some warnings and analysis require some optimization.
-O1	Enable optimization but try to mitigate compile time.
-O	Same as -O1.
-O2	Performs nearly all supported optimizations.
-O3	Optimize aggressively for speed.
-Ofast	Enable all -O3 optimizations + some some optimization that can brake the program.
-Os	Optimize for size.
-Oz	Optimize aggressively for size rather than speed.
-fomit-frame-pointer	Omit the frame pointer in functions that don’t need one.
-m32	If supported, switch to x86 architecture (you should prefer using config.set_target_architecture).
-m64	If supported, switch to x64 architecture (you should prefer using config.set_target_architecture).
-march=native	Generate a program optimized for CPUs that have the same capabilities of the host.
-mtune=native	Optimize a program for the specific CPU of the host, even if this program will run slower or not at all on any other machine.
-mmx	Enable mmx vectorization.
-msse	Enable sse vectorization.
-msse2	Enable sse2 vectorization.
-msse3	Enable sse3 vectorization.
-mavx	Enable avx vectorization.
-mavx2	Enable avx2 vectorization.
-g	Compile with debug symbols.
-fsecurity=1	Enable all flags that can mitigate security issues with negligible impact on performance. Warnings included.
-fsecurity=2	Enable all flags that can mitigate security issues.
-fsecurity	same as -fsecurity=2.
-ffuzzer	Enable the address sanitizer and the fuzzer.

powermake.run

powermake.run(target_name: str, *, build_callback: callable, clean_callback: callable = default_on_clean, install_callback: callable = default_on_install, args_parsed: argparse.Namespace = None)

It's the entry point of most programs.
This function parses the command line and generates a powermake.Config object, containing all the information required for the compilation, from the compiler path to the level of verbosity to use.

Then, depending on the command line arguments, this function will call the clean callback, the build callback, the install callback, or all of them.

The target_name is a string that will be stored in the config and which will be used for auto-naming. You should set this to the name of your executable or the name of your library.

The build_callback and the clean_callback only take 1 argument: The powermake.Config object generated.

Example:

import powermake


def on_build(config: powermake.Config):
    print("The build callback was called !")
    print(f"Compiling the project {config.target_name}...")

def on_clean(config: powermake.Config):
    print("The clean callback was called !")
    print(f"Erasing the project {config.target_name}...")


powermake.run("my_project", build_callback=on_build, clean_callback=on_clean)

The install_callback takes 2 arguments: The powermake.Config object and a string location that can be None if the user hasn't specified anything on the command line.

[!TIP]
It's often a very good idea to use the install_callback as a "pre-install script" and then call powermake.default_on_install.

Example:

import powermake


def on_build(config: powermake.Config):
    print("The build callback was called !")
    print(f"Compiling the lib {config.target_name}...")

def on_install(config: powermake.Config, location: str):
    if location is None:
        # No location is explicitly provided so we change the default for our convenience.
        location = "/usr/local/"
    
    # This ensures that the file "my_lib.h" will be exported into /usr/local/include/my_lib/my_lib.h
    # The .so or .a that corresponds will be copied into /usr/local/lib/my_lib.so
    config.add_exported_headers("my_lib.h", subfolder="my_lib")

    powermake.default_on_install(config, location)


powermake.run("my_lib", build_callback=on_build, clean_callback=on_clean)

The args_parsed argument should be left to None in most cases, to understand his purpose, see powermake.ArgumentsParser.parse_args

powermake.Config

This is the most important object of this library.
It contains everything you need for your compilation journey. For example, it stores the C compiler alongside the path to the build folder.

Most of the time, this object is created by powermake.run and you don't need to worry about the constructor of this object (which is a bit messy...).

But one thing you have to know is that the construction of this object involves 4 steps:

step 1: It checks the value of the CC and CXX environment variables and if they are set, it sets the path of the C compiler and the C++ compiler with these variables. This makes sure that powermake is compatible with clang scan-build utility.
step 2: It loads the local config file, by default stored at ./powermake_config.json just next to the makefile.py (or whatever name your makefile has)
step 3: It completes the local config with the global config, by default, stored in your home, at ~/.powermake/powermake_config.json (If you create an env variable named POWERMAKE_CONFIG, you can override this location.).
step 4: For all fields that are left empty, powermake will try to create a default value from your platform information.

In theory, after the end of these 4 steps, all members of the powermake.Config object should be set.
In rare cases, if powermake was enabled to detect a default compiler, the c_compiler, cpp_compiler, archiver, and linker members can be None.
In this situation, it's your responsibility to give them a value before the call to the powermake.compile_files function.

If you haven't, we recommend you try compiling your code without setting any powermake_config.json. In most cases, the automatic detection of your environment does a good job of finding your compiler/system/etc...

We provide a tool to interactively set your configuration file, you use it either by running python -m powermake or python makefile.py config, but this tool cannot configure everything, so we provide here an example of a powermake_config.json.
Here, everything is set, but you should set the bare minimum, especially, you shouldn't set the "host_architecture", it's way better to let the script find it.
Please note that this example is incoherent, but it shows as many options as possible.

{
    "nb_jobs": 8,
    "compile_commands_dir": ".vscode/",
    "host_operating_system": "Linux",
    "target_operating_system": "Windows",
    "host_architecture": "x64",
    "target_architecture": "x86",

    "c_compiler": {
        "type": "gcc",
        "path": "/usr/bin/gcc"
    },
    "cpp_compiler": {
        "type": "clang++"
    },
    "as_compiler": {
      "type": "gcc",
      "path": "/usr/bin/gcc"
    },
    "asm_compiler": {
      "type": "nasm",
      "path": "/usr/bin/nasm"
    },
    "archiver": {
        "type": "ar",
        "path": "/usr/bin/ar"
    },
    "linker": {
        "type": "gnu",
        "path": "/usr/bin/cc"
    },
    "shared_linker": {
        "type": "g++",
        "path": "/usr/bin/g++"
    },

    "obj_build_directory": "./build/objects/",
    "lib_build_directory": "./build/lib/",
    "exe_build_directory": "./build/bin/",

    "defines": ["WIN32", "DEBUG"],
    "additional_includedirs": ["/usr/local/include", "../my_lib/"],
    "shared_libs": ["mariadb", "ssl", "crypto"],
    "c_flags": ["-fanalyzer", "-O3"],
    "cpp_flags": ["-g", "-O0"],
    "c_cpp_flags": ["-Wswitch"],
    "as_flags": [],
    "asm_flags": ["-s"],
    "c_cpp_as_asm_flags": ["-Wall", "-Wextra"],
    "ar_flags": [],
    "ld_flags": ["-static", "-no-pie"],
    "shared_linker_flags": ["-fPIE"],

    "exported_headers": ["my_lib.h", ["my_lib_linux.h", "my_lib/linux"], ["my_lib/windows.h", "my_lib/windows"]]
}

Members

All fields have the same name in the powermake_config.json and in the powermake.Config object, so we have grouped them below.

nb_jobs

This number determines how many threads the compilation should be parallelized.
If non-set or set to zero, this number is chosen according to the number of CPU cores you have.

Set this to 1 to disable multithreading.

This can be overwritten by the command-line.

compile_commands_dir

If this is set, powermake.compile_files will generate a compile_commands.json in the directory specified by this parameter.

host_operating_system

A string representing the name of your operating system. For the moment it doesn't serve any purpose, but you can access it if needed.

[!TIP]
It's not recommended to set this in the json file, the autodetection should do a better job.

target_operating_system

A string representing the name of the operating system for which the executable is for.
It's used to determine the subfolder of the build folder and for the functions target_is_linux, target_is_windows, etc...

You can write this in the json configuration, but only if you are doing cross-compilation, on the other hand, you should let powermake retrieve this value.
[!WARNING]
Note that if you change this value in the script after the config is loaded, obj_build_directory, lib_build_directory and exe_build_directory will not be updated

host_architecture

A string representing the architecture of your system, which can be "amd64", "x64", "x86", "i386", etc...
If you need an easier string to work with, use config.host_simplified_architecture which can only be "x86", "x64", "arm32" or "arm64".
For the moment it doesn't serve any purpose, but you can access it if needed.

[!TIP]
It's not recommended to set this in the json file, the autodetection should do a better job.

target_architecture

A string representing the architecture of the executable, which can be "amd64", "x64", "x86", "i386", etc...
If you need an easier string to work with, use config.target_simplified_architecture which can only be "x86", "x64", "arm32" or "arm64".
It's used to determine the subfolder of the build folder and to set the compiler architecture. However, for the moment, gcc and clang only switch from 64 bits to 32 bits. If you are on x64 and you set the target_architecture to "arm32", you will in reality compile for x86. You have to give the path of a cross-compiler to achieve what you want.

You can write this in the json configuration, but only if you are doing cross-compilation, on the other hand, you should let powermake retrieve this value.
[!WARNING]
Note that if you change this value in the script after the config is loaded, the environment will not be reloaded and the compiler will keep the previous architecture

c_compiler

This one is different in the json config and the loaded config.

In the json config, it's defined as an object with 2 fields, like this:

"c_compiler": {
    "type": "gcc",
    "path": "/usr/bin/gcc"
},

If the "path" field is omitted, the compiler corresponding to the type is searched in the path. For example if "type" is "msvc", the compiler "cl.exe" is searched in the path.

If the "type" field is omitted, his default value is "gnu".

The "type" field can have the value "gnu", "gcc", "clang", "msvc" or "clang-cl".
It determines the syntax that should be used. For example, if you are using mingw, the syntax of the compiler is the same as the gcc syntax, and the default callback used by powermake.run if the default_callback is unspecified but you can use it whenever you want. our compiler should be set like this:
```
"c_compiler" {
    "type": "gcc",
    "path": "C:\\msys64\\ucrt64\\bin\\gcc.exe"
}
```
[!NOTE]
For mingw on Windows, you should simply set C:\msys64\ucrt64\bin in your PATH and powermake will be able to find it automatically
The "path" field indicates where is the executable of the compiler. Note that PATH searching is always applied, so "gcc" work as well as "/usr/bin/gcc" For mingw on Linux, your compiler can be set like this:
```
"c_compiler" {
    "type": "gcc",
    "path": "x86_64-w64-mingw32-gcc"
}
```

When the powermake.Config object is loaded, the c_compiler member is no longer a dict, it's a virtual class that inherits from powermake.compilers.Compiler and which can generate compile commands. see [documentation is coming]

cpp_compiler

The cpp_compiler behave exactly like the c_compiler but the possible types are:

gnu++
g++
clang++
msvc

You can also use one of the c_compiler types, but in this case you must add a path, or the compilers will not be C++ compilers.

as_compiler

This compiler is used to compile GNU Assembly (.s and .S files)

The as_compiler behave exactly like the c_compiler but the possible types are:

gnu
gcc
clang

You can also use one of the asm_compiler types if you have to compile a .s or .S file with nasm or something like that.

asm_compiler

This compiler is used to compile .asm files (generally Intel asm)

The asm_compiler behave exactly like the c_compiler but the only type currently supported is:

nasm

You can also use one of the as_compiler types if you have to compile a .asm file with a GNU assembler.

archiver

The archiver is the program used to create a static library.

The configuration in the json behave exactly like the c_compiler but the possible types are:

gnu
ar
llvm-ar
msvc

Once loaded, the config.archiver is a virtual class that inherits from powermake.archivers.Archiver.

linker

The configuration in the json behave exactly like the c_compiler but the possible types are:

gnu
gnu++
gcc
g++
clang
clang++
ld
msvc

Once loaded, the config.linker is a virtual class that inherits from powermake.linkers.Linker.

shared_linker

The configuration in the json behaves exactly like config.linker but is used to link shared libraries.

Once loaded, the config.shared_linker is a virtual class that inherits from powermake.shared_linkers.SharedLinker.

obj_build_directory

This is the directory in which all .o (or equivalent) will be stored.

[!TIP]
It's not recommended to set this in the json file, the automatic path generation should do a better job, ensuring that debug/release, windows/Linux, or x86/x64 doesn't have any conflict.

lib_build_directory

This is the directory in which all .a, .so, .lib, .dll, etc... will be stored.

[!TIP]
It's not recommended to set this in the json file, the automatic path generation should do a better job, ensuring that debug/release, windows/Linux, or x86/x64 doesn't have any conflict.

exe_build_directory

This is the directory in which the linked executable will be stored.

[!TIP]
It's not recommended to set this in the json file, the automatic path generation should do a better job, ensuring that debug/release, windows/Linux, or x86/x64 doesn't have any conflict.

defines

This is a list of some defines that will be used during the compilation process.

This list is merged from the local and global config

[!TIP]
It's not recommended to set this in the json file, it makes much more sense to add these defines directly in the makefile with config.add_defines, if needed, in a conditional statement like if config.target_is_windows():

additional_includedirs

This is a list of additional includedirs that will be used during the compilation process.

This list is merged from the local and global config

[!TIP]
It's not recommended to set this in the json file, it makes much more sense to add these includedirs directly in the makefile with config.add_includedirs, if needed, in a conditional statement like if config.target_is_windows():

shared_libs

This is a list of shared libraries that will be used for the link.

This list is merged from the local and global config

[!TIP]
It's not recommended to set this in the json file, it makes much more sense to add these shared libs directly in the makefile with config.add_shared_libs, if needed, in a conditional statement like if config.target_is_windows():