Skip to main content

A simple library to allow easy package/module level introspection

Project description

Module Discovery Utils


A simple library to make it easier to dynamically find and discover objects from a predefined set of packages.

There is often a pattern used in python where subclasses of an object are defined, and by virtua of being a subclass, those objects are either directly or indirectly added to a registry.

Examples of this are Flask routes, and Sqlalchemy tables.

This is all well and good, but the problem is, you need to import those modules to register them. A common solution is to create a registry file of sorts, sometimes in the file.

Often, you have a significant number of classes, and are frequently adding new ones, or removing old ones. However, maintaining a single file that imports all of them can be cumbersome, and falls afoul of python style rules and formatting tools, as the imports are technically not used in the registry file.

Thus, using python's importlib, pkgutils, and introspection tools, there are a number of ways to dynamically scan a package, load all submodules, and parse out objects of interest.

I have implemented and reimplemented this sort of code many, many times, and I hope I can prevent others from doing the same in the future.


To recursively import all modules in a group of packages:

from module_discovery_utils import load_all_modules_in_packages
from blah import package, package2



from module_discovery_utils import load_all_modules_in_packages
from blah import package, package2

load_all_modules_in_packages([package, package2])

To find all subclasses in a package:

from module_discovery_utils import find_subclasses_in_packages
from blah import package, package2, BaseClass1, BaseClass2

find_subclasses_in_packages([package, package2], [BaseClass1, BaseClass2])

You can also technically do:

from module_discovery_utils import load_all_modules_in_packages
from blah import package, package2, BaseClass

load_all_modules_in_packages([package, package2])
subclasses = BaseClass.__subclasses__()

However, the problem I have with that solution is there are cases (such as testing) where you may not want to retrieve all defined subclasses.

It's pratically impossible to control what gets imported when in a python program, and you don't want to potentially rely on import order, or imports from other parts of the program to determine which subclasses are "visible" at any given time.

To find all instances of a given base class in a given set of packages:

from module_discovery_utils import find_instances_in_packages
from blah import package, package2, BaseClass1, BaseClass2

find_instances_in_packages([package, package2], [BaseClass1, BaseClass2])

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for module-discovery-utils, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size module-discovery-utils-1.0.1.tar.gz (3.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page