pykale 0.2.0

Very cool library with lots of great ideas on moving toward 'green', efficient multimodal machine learning and AI.

Kevin Carlberg, AI Research Science Manager at Facebook Reality Labs (quoted from tweet).

Getting Started | Documentation | Contributing | Discussions | Changelog

PyKale is a library in the PyTorch ecosystem aiming to make machine learning more accessible to interdisciplinary research by bridging gaps between data, software, and end users. Both machine learning experts and end users can do better research with our accessible, scalable, and sustainable design, guided by green machine learning principles. PyKale has a unified pipeline-based API and focuses on multimodal learning and transfer learning for graphs, images, and videos at the moment, with supporting models on deep learning and dimensionality reduction.

PyKale is one of only four libraries officially listed under "Multimodal" in the PyTorch Landscape, alongside MMF from Meta, NeMo from NVIDIA, and USB from Microsoft. More distinctively, PyKale is the only library in this category positioned for training rather than modeling, reflecting our focus on reusable workflows, evaluation, adaptation, and deployable multimodal AI systems.

PyKale enforces standardization and minimalism, via green machine learning concepts of reducing repetitions and redundancy, reusing existing resources, and recycling learning models across areas. PyKale will enable and accelerate interdisciplinary, knowledge-aware machine learning research for graphs, images, and videos in applications including bioinformatics, graph analysis, image/video recognition, and medical imaging, with an overarching theme of leveraging knowledge from multiple sources for accurate and interpretable prediction.

See our arXiv preprint and four short introductory videos on YouTube: Why build PyKale? How was PyKale built? What's in PyKale? and a 5-min summary.

Event using PyKale

• Open Biomedical Multimodal AI Research Workshop @ EMBC-2025

Pipeline-based API

• loaddata loads data from disk or online resources as input
• prepdata preprocesses data to fit machine learning modules below (transforms)
• embed embeds data in a new space to learn a new representation (feature extraction/selection)
• predict predicts a desired output
• evaluate evaluates the performance using some metrics
• interpret interprets the features and outputs via post-prediction analysis mainly via visualization
• pipeline specifies a machine learning workflow by combining several other modules

Example usage

• examples demonstrate real applications on specific datasets with a standardized structure.

How to Use

Step 0: Installation

PyKale supports Python 3.10, 3.11, or 3.12. Before installing pykale, we suggest you to first install PyTorch matching your hardware, and if graphs will be used, install PyTorch Geometric following its official instructions.

Simple installation of pykale from PyPI:

pip install pykale

For more details and other options, please refer to the installation guide.

Step 1: Tutorials and Examples

Start with a brief tutorial walking through API usage in examples or interactive Jupyter notebook tutorials, e.g. or for a basic digit classification problem.

For additional learning materials, please refer to our tutorial repository, which supports our multimodal AI workshops.

Browse through the examples to see the usage of PyKale in performing various prediction tasks in a wide range of applications, using a variety of settings, e.g. with or without PyTorch Lightning.

Ask questions on PyKale's GitHub Discussions tab if you need help or create an issue if you find something wrong.

Step 2: Building and Contributing

Build new modules and/or projects with PyKale referring to the tutorial, e.g., on how to modify an existing pipeline or build a new one.

This is an open-source project welcoming your contributions. You can contribute in three ways:

• Star and fork PyKale to follow its latest developments, share it with your networks, and ask questions about it.
• Use PyKale in your project and let us know any bugs (& fixes) and feature requests/suggestions via creating an issue.
• Contribute via branch, fork, and pull for minor fixes and new features, functions, or examples to become one of the contributors.

See contributing guidelines for more details. You can also reach us via email if needed. The participation in this open source project is subject to Code of Conduct.

Who We Are

The Team

PyKale is maintained by Haiping Lu, Shuo Zhou, Xianyuan Liu, and Peizhen Bai, with contributions from many other contributors.

Citation

    @inproceedings{pykale-cikm2022,
      title     = {{PyKale}: Knowledge-Aware Machine Learning from Multiple Sources in {Python}},
      author    = {Haiping Lu and Xianyuan Liu and Shuo Zhou and Robert Turner and Peizhen Bai and Raivo Koot and Mustafa Chasmai and Lawrence Schobs and Hao Xu},
      booktitle = {Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM)},
      doi       = {10.1145/3511808.3557676},
      year      = {2022}
    }

Please consider citing our CIKM2022 paper above if you find PyKale useful to your research.

Acknowledgements

The development of PyKale is partially supported by the following grants:

• EPSRC Network Plus: Tomorrow's Engineering Research Challenges (grant no. UKRI396) - "UK Open Multimodal AI Network (UKOMAIN)"
• University of Sheffield: Centre for Machine Intelligence (CMI) - AI Research Engineering (AIRE).
• Wellcome Trust Innovator Awards: Digital Technologies (grant no. 215799/Z/19/Z) - "Developing a Machine Learning Tool to Improve Prognostic and Treatment Response Assessment on Cardiac MRI Data".