Skip to main content

A priority sampling tool.

Project description

Priority Memory

Fast priority experience replay buffer.

The implementation is based on sum tree, or segmentation tree.

  • Set the priority of each sample at anytime.
  • When you do not know the priority of the sample, you can append them to the buffer, and they will show up in the next sampling batch.
  • When the buffer is full, drop the samples with lowest priority.

The time complexity for sampling a batch with batch size m from a dataset with n samples is O(mlogn), for setting priority for the batch is O(mlogn).

Usage

pip install priority_memory

from priority_memory import FastPriorReplayBuffer

buffer = FastPriorReplayBuffer(8000)
buffer.append(features=[0.1, 0.1, 0.1], prior=1)
buffer.append(features=[0.2, 0.2, 0.2], prior=2)
buffer.append(features=[0.3, 0.3, 0.3], prior=3)
buffer.append(features=[0.4, 0.4, 0.4], prior=4)
indexes, data, weights = buffer.sample_with_weights(batch_size=2)

mae = [10, 20]
buffer.set_weights(indexes, mae)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

priority_memory-0.0.1.tar.gz (7.4 kB view hashes)

Uploaded source

Built Distribution

priority_memory-0.0.1-py3-none-any.whl (14.3 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page