Skip to main content

FLightcase toolbox for Federated Learning

Project description

FLightcase :airplane::briefcase:

A federated Learning toolbox for neuro-image research, based on secure copy protocol (SCP) via secure shell (SSH).
It was first introduced in a preprint in medrXiv[1], and now contains a Command-Line Interface (CLI): FLightcase

PyPI


Requirements

  • Unix-based operating system for each node in the network
  • All computers in the same network (e.g. connected via VPN), identifiable via an IP Address
  • All datasets in the Brain Imaging Data Structure (BIDS)([2])

In brief

The FLightcase toolbox works by sending files via SCP between computers. To ensure full transmission of a file, a .txt file is sent to mark transmission completion. Each node (server and clients) prepares a "workspace", which is a local directory that collect files. Files are shared between computers by knowing each other's workspace location. This, and other information, is available in a JSON metadata file included in each workspace directory. The server additionally defines the Federated Learning plan in a JSON file, containing the parameters for the FL process.


How to get started

Install FLightcase

Install FLightcase in a virtual environment or conda environment on each node. Example for virtual environment:

  1. python3 -m venv .venv (creates virtual environment called ".venv" in current workspace)
  2. source .venv/bin/activate (activate virtual environment)
  3. pip3 install FLightcase (optional: define version number, e.g. pip3 install FLightcase==0.1.0)

Preparing the workspaces

The FLightcase prepare-workspace command is used for preparing the local workspace. Define two flags:

  • --who: client or server
  • --workspace_path: path to workspace directory

Running it will first check whether the workspace path exists, and create it if this is not the case.
Then, an instruction menu is to be completed in your terminal:

  • --who client: Preparation of the client_node_settings.json file
  • --who server: Preparation of the server_node_settings.json, the FL_plan.json file and the architecture.py file

When preparing each JSON file, there are two options:

  1. Copying the template file to the workspace. The template can then be filled in manually with any preferred text editor.
  2. Filling the template file in the terminal. The instruction menu will, step by step, complete the template with you. It will then be saved to the workspace. For the server, the architecture.py will always be copied to the workspace without being completed in the terminal. Please update this file to your preferred network architecture.

Running FLightcase

Make sure your terminal is in the virtual environment that contains FLightcase on each node. Then:

  1. On client nodes, run: FLightcase run-client --settings_path /path/to/client_node_settings.json
  2. On the server node, run FLightcase run-server --settings_path /path/to/server_node_settings.json

Enjoy the show! :woman_dancing::man_dancing:

Note:

A file clean-up will be performed in the FL workspace after federated learning has completed. All files except the node settings, FL plan and architecture will be moved to a subdirectory marked by date and time stamp. The FL_plan and architecture will be copied to this subdirectory to keep a log of the exact experiment that was performed.


Need help or discovered a problem?

If you experience problems with this GitHub repository, please do not hesitate to create an issue, or send a mail to stijn.denissen@vub.be


References

[1] Denissen, S., Grothe, M., Vaneckova, M., Uher, T., Laton, J., Kudrna, M., ... & Nagels, G. (2023). Transfer learning on structural brain age models to decode cognition in MS: a federated learning approach. medRxiv, 2023-04.

[2] Gorgolewski, K. J., Auer, T., Calhoun, V. D., Craddock, R. C., Das, S., Duff, E. P., ... & Poldrack, R. A. (2016). The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. Scientific data, 3(1), 1-9.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flightcase-0.1.2.tar.gz (11.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

FLightcase-0.1.2-py3-none-any.whl (9.6 kB view details)

Uploaded Python 3

File details

Details for the file flightcase-0.1.2.tar.gz.

File metadata

  • Download URL: flightcase-0.1.2.tar.gz
  • Upload date:
  • Size: 11.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.12.4

File hashes

Hashes for flightcase-0.1.2.tar.gz
Algorithm Hash digest
SHA256 fac19b0783e6e7528f713e087942edc1d67457f7fcb6e3b8ab40f41f0245dd98
MD5 91a67ad2b5232b4268ad5ae6f4ce5b39
BLAKE2b-256 0a71e874f3d53c7a65c40320ad29ebc904d04131258de7013a9ab0dbca949b0f

See more details on using hashes here.

File details

Details for the file FLightcase-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: FLightcase-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 9.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.0.1 CPython/3.12.4

File hashes

Hashes for FLightcase-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 ab55788d1361dfca948d1854d980f967a01956cec760d989f14e30c068e1af0c
MD5 503c3c2a8d2d4549882318e460193e0c
BLAKE2b-256 6ee62304bcc14ead20169b01b433241c3f7caad13bb6ba99fd2d101731b7ec9b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page