ExhaustiveWeightedRandomSampler is an advanced version of WeightedRandomSampler
Project description
ExhaustiveWeightedRandomSampler
ExhaustiveWeightedRandomSampler can exhaustively sample the indices with a specific weight over epochs.
Installation
pip install exhaustive-weighted-random-sampler
Usage & Comparasion
import torch
from torch.utils.data import WeightedRandomSampler
from exhaustive_weighted_random_sampler import ExhaustiveWeightedRandomSampler
sampler = WeightedRandomSampler([1, 1, 1, 1, 1, 1, 1, 1, 1, 10], num_samples=5)
for i in range(5):
print(list(sampler))
"""
output:
[4, 3, 9, 3, 4]
[0, 5, 0, 9, 8]
[9, 9, 0, 9, 2]
[9, 9, 7, 9, 9]
[9, 9, 9, 9, 9]
explain: there are no 1 and 6, but 0 appears three times
"""
sampler = ExhaustiveWeightedRandomSampler([1, 1, 1, 1, 1, 1, 1, 1, 1, 10], num_samples=5)
for i in range(5):
print(list(sampler))
"""
output:
[4, 6, 9, 9, 9]
[1, 0, 9, 9, 5]
[9, 7, 3, 8, 9]
[9, 2, 1, 9, 9]
[8, 9, 7, 3, 2]
explain: all the 0 to 8 appears in the yield results.
"""
Use in DDP
It can be used in DDP
if pytorch-ignite has been installed.
from ignite.distributed import DistributedProxySampler
from torch.utils.data import DataLoader
dataset = ...
sampler = DistributedProxySampler(
ExhaustiveWeightedRandomSampler(weights, num_samples=10000)
)
loader = DataLoader(dataset, sampler=sampler, ...)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for exhaustive-weighted-random-sampler-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6aec0368a6cb3e73380b487b78812a38f0399c64272336e7993592bf76e428f6 |
|
MD5 | b6f07ad6c3e6a3620ebc80e17d6fe33a |
|
BLAKE2b-256 | c7f26752de715714a48ef8f4491d5ded9838cb0bd5107688ed8ce3d6334a88d9 |
Close
Hashes for exhaustive_weighted_random_sampler-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0592f9ee40b8bb8299f339bb54816da5ccead2ca04164de0d27e33ccaeab1894 |
|
MD5 | 2b098dfc847d7a3842e5f199dee9bf9a |
|
BLAKE2b-256 | 918c1b72273e8b92f614e06efcba32990dcaac64486071771271f1747c88e0df |