VPRTempo 0.2.0

VPRTempo: A Fast Temporally Encoded Spiking Neural Network for Visual Place Recognition

VPRTempo - Temporally encoded spiking neural network for visual place recognition

This repository contains code for VPRTempo, a spiking neural network that uses temporally encoding to perform visual place recognition tasks. The network is based off of BLiTNet and adapted to the VPRSNN framework.

VPRTempo is built on a torch.nn network that employs custom learning rules based on the temporal codes of spikes in order to train layer weights.

In this repository, we provide two networks:

• VPRTempo: Our base network architecture to perform visual place recognition (fp32)
• VPRTempoQuant: A modified base network with Quantization Aware Training (QAT) enabled (int8)

To use VPRTempo, please follow the instructions below for installation and usage.

License & Citation

This repository is licensed under the MIT License

If you use our code, please cite the following paper:

@misc{hines2023vprtempo,
      title={VPRTempo: A Fast Temporally Encoded Spiking Neural Network for Visual Place Recognition}, 
      author={Adam D. Hines and Peter G. Stratton and Michael Milford and Tobias Fischer},
      year={2023},
      eprint={2309.10225},
      archivePrefix={arXiv},
      primaryClass={cs.RO}
}

Installation and setup

VPRTempo uses PyTorch with CUDA GPU acceleration. Follow the installation instructions based on your operating system and hardware specifications. MacOS has no compatibly with CUDA.

Get the repository

Download the Github repository.

git clone https://github.com/QVPR/VPRTempo.git
cd ~/VPRTempo

Once downloaded, please install the required dependencies to run the network through one of the following options:

Option 1: Pip install

Dependencies for VPRTempo can downloaded from our PyPi package.

# For Windows/Linux systems
!pip install vprtempo

# For MacOS
!pip install vprtempomacos

Option 2: Local requirements install

Dependencies can be installed either through our provided requirements.txt files.

# For Windows/Linux
!pip install -r requirements.txt

# For MacOS
!pip instal -r requirements_macos.txt

Option 3: Conda install

:heavy_exclamation_mark: Recommended: Use Mambaforge instead of conda.

# Windows/Linux - CUDA enabled
conda create -n vprtempo -c pytorch -c nvidia python torchvision torchaudio pytorch-cuda=11.7 cudatoolkit prettytable tqdm numpy pandas scikit-learn

# Windows/Linux - CPU only
conda create -n vprtempo python pytorch torchvision torchaudio cpuonly prettytable tqdm numpy pandas scikit-learn -c pytorch

# MacOS
conda create -n vprtempo -c conda-forge python prettytable tqdm numpy pandas scikit-learn -c pytorch pytorch::pytorch torchvision torchaudio

Datasets

VPRTempo was developed to be simple to train and test a variety of datasets. Please see the information below about running a test with the Nordland traversal dataset and how to organize custom datasets.

Nordland

VPRTempo was developed and tested using the Nordland traversal dataset. This software will work for either the full-resolution or down-sampled datasets, however our paper details the full-resolution datasets.

To simplify first usage, we have set the defaults in VPRTempo.py to train and test on a small subset of Nordland data. We recommend downloading Nordland and using the ./src/nordland.py script to unzip and organize the images into the correct file and naming structure.

Custom datasets

In general, data should be organised in the ./dataset folder in the following way in order to train the network on multiple traversals of the same location.

--dataset
  |--training
  |  |--traversal_1
  |  |--traversal_2
  |
  |--testing
  |  |--test_traversal

If you wish to specify a different directory where data is stored, modify the --data_dir default argument in main.py. Similarly, if you wish to train/query different traversals modify --database_dirs and --query_dir in main.py accordingly.

Usage

Both the training and testing is handled by the VPRTempo.py script. Initial installs do not contain any pre-defined networks and will need to be trained prior to use.

Pre-requisites

• Training and testing data is organized as above (see Datasets on how to set up the Nordland or custom datasets)
• The VPRTempo conda environment has been activated

Once these two things have been setup, run VPRTempo.py to train and test your first network with the default settings.

Issues, bugs, and feature requests

If you encounter problems whilst running the code or if you have a suggestion for a feature or improvement, please report it as an issue.