kconfiglib 1.0.6

A flexible Python Kconfig parser

A Python library for doing stuff with Kconfig-based configuration systems. Can extract information, query and set symbol values, and read and write .config files. Highly compatible with the scripts/kconfig/*conf utilities in the kernel, usually invoked via make targets such as menuconfig and defconfig.

Supports both Python 2 and Python 3 without modification, and should also run on non-*nix platforms

Table of contents

• Installation

  • Installation with pip

  • Installation for the Linux kernel

  • Manual installation

• Documentation

• Examples

• Test suite

• Notes

• Thanks

Installation

Installation with pip

Kconfiglib is available on PyPI and can be installed with e.g.

$ pip(3) install kconfiglib --user

All releases have a corresponding tag in the git repository, e.g. v1.0.6. Semantic versioning is used.

Installation for the Linux kernel

After installing with pip(3), apply makefile.patch by running e.g. the following commands in the kernel root:

$ wget https://raw.githubusercontent.com/ulfalizer/Kconfiglib/master/makefile.patch
$ git am makefile.patch

If you do not wish to install anything, the following manual approach will work as well:

$ git clone git://github.com/ulfalizer/Kconfiglib.git
$ git am Kconfiglib/makefile.patch

(Warning: The directory name Kconfiglib/ is significant in this case, because it’s added to PYTHONPATH by the new targets in makefile.patch.)

In addition to creating a handy interface, the make targets created by the patch (scriptconfig and iscriptconfig) are needed to pick up environment variables set in the kernel makefiles and later referenced in the Kconfig files (ARCH, SRCARCH, and KERNELVERSION as of Linux v4.14.0-rc1). The documentation explains how the make targets are used. The compatibility tests in the test suite also needs them.

Please tell me if the patch does not apply. It should be trivial to apply manually, as it’s just a block of text that needs to be inserted near the other *conf: targets.

Manual installation

The entire library is contained in kconfiglib.py. Just drop it somewhere.

Documentation

The (extensive) documentation is generated by running the following command in the same directory as kconfiglib.py:

$ pydoc kconfiglib

For HTML output, use

$ pydoc -w kconfiglib

You could also browse the docstrings directly in kconfiglib.py.

Please tell me if something is unclear to you or can be explained better. The Kconfig language has some dark corners.

Examples

• The examples/ directory contains simple example scripts. See the documentation for how to run them.

• genboardscfg.py from Das U-Boot generates some sort of legacy board database by pulling information from a newly added Kconfig-based configuration system (as far as I understand it :).

• gen-manual-lists.py generated listings for an appendix in the Buildroot manual. (The listing has since been removed.)

• SConf builds an interactive configuration interface (like menuconfig) on top of Kconfiglib, for use e.g. with SCons.

• kconfig-diff.py – a script by dubiousjim that compares kernel configurations.

• Originally, Kconfiglib was used in chapter 4 of my master’s thesis to automatically generate a “minimal” kernel for a given system. Parts of it bother me a bit now, but that’s how it goes with old work.

Test suite

The test suite is run with

$ python(3) Kconfiglib/testsuite.py

It must be run from the top-level kernel directory, and requires that the git repository has been cloned into it and makefile.patch applied.

NOTE: Some tests currently overwrite .config in the kernel root, so make sure to back it up.

The test suite consists of a set of selftests and a set of compatibility tests that compare (character for character) configurations generated by Kconfiglib with configurations generated by scripts/kconfig/conf for a number of cases. You might want to use the “speedy” option; see testsuite.py.

The test suite might fail for a few configurations for kernels older than April 2016, when a fix was added to Kconfig that’s also mirrored in Kconfiglib (see this commit). This is harmless, and only counts as a fail since the test suite compares literal output from the kconfig version that’s bundled with the kernel.

Kconfiglib is much faster than the test suite would indicate. Most of the time is spent waiting around for make or the C utilities. Adding some multiprocessing to the test suite would make sense.

Notes

• Useful information can be extracted from internal data structures. The expression format is pretty simple for example: A && B && (!C || D == 3) is represented as the tuple structure (_AND, A, (_AND, B, (_OR, (_NOT, C), (_EQUAL, D, 3)))); see the Config._parse_expr() docstring.

  It’s hard to come up with good APIs for dealing with expressions given how general they are, so feel free to look at them directly if none of the exposed APIs will suffice (modifying them is dangerous though, because it breaks dependency tracking). Maybe I’ll officially document the expression format and add a bunch of accessors later. The internal format is unlikely to change in either case, and would probably be returned directly.

  If you come up with some good generally-usable APIs involving expressions, please tell me. Make sure they also make sense for expressions involving || (or) and ! (not).

• Kconfiglib works well with PyPy. It gives a nice speedup over CPython when batch processing a large number of configurations (like the test suite does).

• Kconfiglib assumes the modules symbol is MODULES and will warn if option modules is set on some other symbol. Let me know if this is a problem for you. Adding proper option modules support should be pretty easy.

• At least two things make it awkward to replicate a menuconfig-like interface in Kconfiglib at the moment (but see SConf, as mentioned above).

  • There are no good APIs for figuring out what other symbols change in value when the value of some symbol is changed, to allow for “live” updates in the configuration interface. The simplest workaround is to refetch the value of each currently visible symbol every time a symbol value is changed.

  • menuconfig sometimes creates cosmetic menus implicitly by looking at dependencies. For example, a list of symbols where all symbols depend on the first symbol creates a cosmetic menu rooted at the first symbol. Recreating such menus is awkward.

    There is already basic support internally though, because it’s needed to get obscure choice behavior right. See _determine_actual_symbols() and its helper _has_auto_menu_dep_on().

• Using __slots__ on classes would speed things up a bit and save memory. It’d remove some flexibility though.

• fpemud has put together Python bindings to internal functions in the C implementation. This is an alternative to Kconfiglib’s all-Python approach.

• The test suite failures (should be the only ones) for the following Blackfin defconfigs on e.g. Linux 3.7.0-rc8 are due to a bug in the C implementation:

  • arch/blackfin/configs/CM-BF537U_defconfig

  • arch/blackfin/configs/BF548-EZKIT_defconfig

  • arch/blackfin/configs/BF527-EZKIT_defconfig

  • arch/blackfin/configs/BF527-EZKIT-V2_defconfig

  • arch/blackfin/configs/TCM-BF537_defconfig

Thanks

Thanks to Philip Craig for adding support for the allnoconfig_y option and fixing an obscure issue with comments inside choices (that didn’t affect correctness but made outputs differ). allnoconfig_y is used to force certain symbols to y during make allnoconfig to improve coverage.