Skip to main content

Implementation of NeuroMechFly v2, framework for simulating embodied sensorimotor control in adult Drosophila

Project description

Simulating embodied sensorimotor control with NeuroMechFly v2

Documentation | Preprint | Discussion Board

Python: 3.9–3.12 License: MPL 2.0 PyPI version Repo Size

Docker Docker image size

overview_video

This repository contains the source code for FlyGym, the Python library for NeuroMechFly v2, a digital twin of the adult fruit fly Drosophila melanogaster that can see, smell, walk over challenging terrain, and interact with the environment (see our NeuroMechFly v2 paper).

NeuroMechFly consists of the following components:

  • Biomechanical model: The biomechanical model is based on a micro-CT scan of a real adult female fly (see our original NeuroMechFly publication). We have adjusted several body segments (in particular in the antennae) to better reflect the biological reality.
  • Vision: The fly has compound eyes consisting of individual units called ommatidia arranged on a hexagonal lattice. We have simulated the visual inputs on the fly’s retinas.
  • Olfaction: The fly has odor receptors in the antennae and the maxillary palps. We have simulated the odor inputs experienced by the fly by computing the odor/chemical intensity at these locations.
  • Hierarchical control: The fly’s Central Nervous System consists of the brain and the Ventral Nerve Cord (VNC), a hierarchy analogous to our brain-spinal cord organization. The user can build a two-part model — one handling brain-level sensory integration and decision making and one handling VNC-level motor control — with an interface between the two consisting of descending (brain-to-VNC) and ascending (VNC-to-brain) representations.
  • Leg adhesion: Insects have evolved specialized adhesive structures at the tips of the legs that enable locomotion vertical walls and overhanging ceilings. We have simulated these structures in our model. The mechanism by which the fly lifts the legs during locomotion despite adhesive forces is not well understood; to abstract this, adhesion can be turned on/off during leg stance/swing.
  • Mechanosensory feedback: The user has access to the joint angles, forces, and contact forces experienced by the fly.

NeuroMechFly formulates the control of the simulated fly as a partially observable Markov Decision Process (MDP) and implements the Gym interface. This allows the user to use a wide range of reinforcement learning algorithms to train the fly to perform tasks. The standardized interface also allows the user to easily implement their own premotor computation and/or sensory preprocessing processes.

This package is developed at the Neuroengineering Laboratory, EPFL.

Installation and dependencies

In brief:

pip install "flygym"
# or pip install "flygym[examples]" to install additional dependencies needed for examples

Alternatively, we provide a Docker image. See our website for details, especially if you plan to install FlyGym in the developer mode (i.e. if you plan to make changes to the code). Dependencies are specified in setup.py and will be installed automatically upon installation using pip. Installation should take no more than a few minutes. The PyPI version of the current release of FlyGym is indicated on the shield at the top of this page. No special, paid software is required to use FlyGym.

Demos

See our website for tutorials, including expected outputs. For code blocks that take more than a few seconds to run, the running time (on a 2020 MacBook Pro with M1 processor running macOS 13.5.2) is indicated, typically in the form of a progress bar.

Reproducing results in the paper

We are constantly working on expanding the package and improving its usability; therefore the package is subject to change. To reproduce the exact results demonstrated in our preprint, use FlyGym 0.1.0 and analysis code here.

Citation

If you use NeuroMechFly in your work, please cite the following papers:

@article{WangChen2023,
  author = {Sibo Wang-Chen and Victor Alfred Stimpfling and Pembe Gizem \"{O}zdil and Louise Genoud and Femke Hurtak and Pavan Ramdya},
  title = {NeuroMechFly 2.0, a framework for simulating embodied sensorimotor control in adult Drosophila},
  year = {2023},
  doi = {10.1101/2023.09.18.556649},
  URL = {https://www.biorxiv.org/content/early/2023/09/18/2023.09.18.556649},
  journal = {bioRxiv}
}

@article{LobatoRios2022,
  doi = {10.1038/s41592-022-01466-7},
  url = {https://doi.org/10.1038/s41592-022-01466-7},
  year = {2022},
  month = may,
  volume = {19},
  number = {5},
  pages = {620--627},
  author = {Victor Lobato-Rios and Shravan Tata Ramalingasetty and Pembe Gizem \"{O}zdil and Jonathan Arreguit and Auke Jan Ijspeert and Pavan Ramdya},
  title = {{NeuroMechFly}, a neuromechanical model of adult {Drosophila} melanogaster},
  journal = {Nature Methods}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flygym-1.1.0.tar.gz (23.3 MB view details)

Uploaded Source

Built Distribution

flygym-1.1.0-py3-none-any.whl (23.3 MB view details)

Uploaded Python 3

File details

Details for the file flygym-1.1.0.tar.gz.

File metadata

  • Download URL: flygym-1.1.0.tar.gz
  • Upload date:
  • Size: 23.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.0

File hashes

Hashes for flygym-1.1.0.tar.gz
Algorithm Hash digest
SHA256 c840cd24f0c2c8c3e3afc4b8268bfda3303655384ff6b3862647173e320990c8
MD5 d5cc91b9a53ad59ee9edbceb8bc5abee
BLAKE2b-256 1c33aa08c63bb062d0d972959008ac157694b9d47866305c3f24c739dbba977e

See more details on using hashes here.

File details

Details for the file flygym-1.1.0-py3-none-any.whl.

File metadata

  • Download URL: flygym-1.1.0-py3-none-any.whl
  • Upload date:
  • Size: 23.3 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.11.0

File hashes

Hashes for flygym-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ca37e104dc0c6bd1a0f3504070b73b01c1e080837456ca98a2a4777a5ae947a4
MD5 aa2ea9ddf259ed7d6e01046e23ef5f88
BLAKE2b-256 605b6069ca27a019cb73cbdf849fdb065ca0317c1c8ba28aa879f28921ead95e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page