Aftab Header


🇪🇸🇲🇽🇨🇺 Español | 🇮🇷🇦🇫🇹🇯 فارسی | 🇮🇹🇨🇭 Italiano | 🇫🇷🇧🇪🇨🇭 Français | 🇩🇪🇦🇹🇨🇭 Deutsch | 🇳🇱🇧🇪🇸🇷 Nederlands | 🇵🇹🇧🇷🇦🇴 Português | 🇸🇦🇱🇧🇮🇶 العربية | 🇷🇺🇧🇾🇰🇿 Русский | 🇨🇳🇸🇬🇹🇼 中文 | 🇯🇵 日本語 | 🇰🇷 한국어 | 🇮🇳 हिन्दी | 🇮🇩 Bahasa Indonesia | 🇧🇩🇮🇳 বাংলা | 🇻🇳 Tiếng Việt | 🇹🇷 Türkçe

Overview

Aftab (Persian: آفتاب, meaning "sun" or "sun rays") is a benchmarking framework for evaluating CNN-based encoders in PQN across Atari games. It provides standardized training, evaluation, and reproducibility tools for deep reinforcement learning research.

We have compiled a few videos comparing PQN and Aftab agents. Watch them here.

Encoder Experiments

IQM HNS
IQM HNS (Last 50M Frames)

Hadamax Experiments

IQM HNS
IQM HNS (Last 50M Frames)

References:

Q-Values Experiments

IQM HNS
IQM HNS (Last 50M Frames)

References:

Procgen (Overfitting Prevention) Experiments

IQM PNS
IQM PNS (Last 50M Frames)

Installation

Install via pip:

pip install aftab

Alternatively, you can clone the repository and install in editable mode.

git clone https://github.com/tahashieenavaz/aftab.git aftab_source
pip install -e aftab_source

We highly recommend using Micromamba for creating virtual environments with instructions detailed here.

Training Agents

Currently JAX API is under development and is planned to be finished by the end of 2026. Contributions are highly encouraged.

from aftab import Aftab
from aftab import aftab_environments

seeds = [1, 2, 3, 4]

for environment in aftab_environments:
    agent = Aftab(encoder="gamma", frames="pilot")
    for seed in seeds:
        agent.train(environment=environment, seed=seed)
        agent.log()

Custom Encoder Injection

You can define your own encoder as a PyTorch module and pass it to the agent:

import torch
from aftab import Aftab

class CustomImageEncoder(torch.nn.Module):
    pass

agent = Aftab(encoder=CustomImageEncoder)

Results

All experimental results are organized by experiment category. Each section contains:

  • Tables: numerical results (HNS/PHS and raw scores)
  • Charts: IQM normalized scores and training curves

Encoder Experiments

Tables

Charts

Hadamax Experiments

Tables

Charts

Q-Value Experiments

Tables

Charts

Procgen Experiments

Tables

Model Complexity

Base Variants

Variant Encoder Parameters Regression Head Parameters Total Parameters Encoder FLOPs Regression Head FLOPs Total FLOPs
PQN 78,304 1,686,500 1,764,804 7.734 1.610 9.347
Alpha 174,752 1,782,948 1,957,700 27.541 1.610 29.151
Beta 89,008 1,782,948 1,871,956 61.515 1.610 63.126
Gamma 117,168 1,725,364 1,842,532 22.901 1.610 24.512
Delta 78,552 1,850,588 1,929,140 6.143 1.774 7.917
Epsilon 80,112 2,179,828 2,259,940 13.252 2.101 15.354
Zeta 77,232 2,537,396 2,614,628 25.362 2.462 27.824
Eta 78,400 23,739,460 23,817,860 28.422 23.663 52.085
Theta 76,288 1,127,428 1,203,716 9.065 1.053 10.118

Note: The Eta variant has significantly more parameters than other variants, primarily due to the encoder producing a large number of features.

Hadamax Variants

Variant Encoder Parameters Regression Head Parameters Total Parameters Encoder FLOPs Regression Head FLOPs Total FLOPs
Hadamax 156,608 3,968,516 4,125,124 159.014 3.969 162.984
Gamma-Hadamax-Valid 234,336 1,609,220 1,843,556 122.001 1.610 123.611
Gamma-Hadamax-Same 234,336 3,280,388 3,514,724 129.300 3.281 132.581

Hyperparameters

Hyperparameter Value
Learning rate $2.5 \times 10^{-4}$
Training environments 128
Test environments 8
Optimizer Rectified Adam
Weight decay 0
$\epsilon$ $1 \times 10^{-5}$
$\beta_{1}$ 0.9
$\beta_{2}$ 0.999
Total Frames 200,000,000
Loss function Mean Squared Error
Scheduler Linear Annealing
$\epsilon$-greedy exploration 10% of total frames
Discount factor ($\gamma$) 0.99
GAE ($\lambda$) 0.65
Epochs 2
Batch size 4096

Used in encoder and Hadamax experiments.

Statistical Significance

Encoder Experiments

Wilcoxon Signed Rank Test Wilcoxon Signed Rank Test (Corrected)
Probability of Improvement

Hadamax Experiments

Wilcoxon Signed Rank Test Wilcoxon Signed Rank Test (Corrected)
Probability of Improvement

Q-Value Experiments

Wilcoxon Signed Rank Test Wilcoxon Signed Rank Test (Corrected)
Probability of Improvement

Reproducibility

Due to the stochastic nature of deep reinforcement learning, exact reproducibility via fixed datasets is not feasible.
Instead, we provide a set of random seeds used in our experiments.

from aftab import aftab_seeds

print(aftab_seeds)

Full experiment replication:

from aftab import Aftab
from aftab import aftab_environments
from aftab import aftab_seeds

for environment in aftab_environments:
    agent = Aftab()
    for seed in aftab_seeds:
        agent.train(environment=environment, seed=seed)
        agent.log()

A comprehensive set of Atari environments is available via EnvPool:
https://envpool.readthedocs.io/en/latest/env/atari.html#available-tasks

Procgen environments use their native RGB observations with shape (3, 64, 64). Aftab reads each task's EnvPool configuration and only applies supported options. Atari-only options such as noop, frame_skip, frame_stack, train_episodic_life, and EnvPool reward clipping are therefore not passed to Procgen.

A comprehensive set of Procgen environments is available via EnvPool:

https://envpool.readthedocs.io/en/latest/env/procgen.html#available-tasks

Hardware

Nvidia A40 GPUs were used to run all the experiments in this experiment.

Specification Details
GPU Memory 48 GB GDDR6 with error-correcting code (ECC)
GPU Memory Bandwidth 696 GB/s
Interconnect NVIDIA NVLink 112.5 GB/s (bidirectional); PCIe Gen4: 64 GB/s
NVLink 2-way low profile (2-slot)
Display Ports 3x DisplayPort 1.4*
Max Power Consumption 300 W
Form Factor 4.4" (H) x 10.5" (L), Dual Slot
Thermal Passive
vGPU Software Support NVIDIA Virtual PC, NVIDIA Virtual Applications, NVIDIA RTX Virtual Workstation, NVIDIA Virtual Compute Server, NVIDIA AI Enterprise
vGPU Profiles Supported See the Virtual GPU Licensing Guide
NVENC / NVDEC 1x / 2x (includes AV1 decode)
Secure Boot Secure and Measured Boot with Hardware Root of Trust (optional)
NEBS Ready Level 3
Power Connector 8-pin CPU

Citation

@article{aftab2026drl,
  title={Aftab: A Comprehensive Benchmark of CNN Encoders and Advanced Value Functions in Parallelized Q-Networks},
  author={Shieenavaz, Taha and Zareshahraki, Shabnam and Nanni, Loris},
  journal={arXiv preprint arXiv:YYMM.NNNNN},
  year={2026}
}

Related Works

@misc{2407.04811,
  Title = {Simplifying Deep Temporal Difference Learning},
  Author = {Matteo Gallici and Mattie Fellows and Benjamin Ellis and Bartomeu Pou and Ivan Masmitja and Jakob Nicolaus Foerster and Mario Martin},
  Year = {2024},
  Eprint = {arXiv:2407.04811},
}

@misc{2403.03950,
  Title = {Stop Regressing: Training Value Functions via Classification for Scalable Deep RL},
  Author = {Jesse Farebrother and Jordi Orbay and Quan Vuong and Adrien Ali Taïga and Yevgen Chebotar and Ted Xiao and Alex Irpan and Sergey Levine and Pablo Samuel Castro and Aleksandra Faust and Aviral Kumar and Rishabh Agarwal},
  Year = {2024},
  Eprint = {arXiv:2403.03950},
}

@misc{1511.06581,
  Title = {Dueling Network Architectures for Deep Reinforcement Learning},
  Author = {Ziyu Wang and Tom Schaul and Matteo Hessel and Hado van Hasselt and Marc Lanctot and Nando de Freitas},
  Year = {2015},
  Eprint = {arXiv:1511.06581},
}

@misc{1806.04613,
  Title = {Improving Regression Performance with Distributional Losses},
  Author = {Ehsan Imani and Martha White},
  Year = {2018},
  Eprint = {arXiv:1806.04613},
}

@misc{1602.04621,
  Title = {Deep Exploration via Bootstrapped DQN},
  Author = {Ian Osband and Charles Blundell and Alexander Pritzel and Benjamin Van Roy},
  Year = {2016},
  Eprint = {arXiv:1602.04621},
}

Useful Links

License

© 2025 Taha Shieenavaz.
Licensed under CC BY-NC 4.0: https://creativecommons.org/licenses/by-nc/4.0/

Project links

Data verified by PyPI on Aug 10, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (1.0.5)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for underdash from gravatar.com underdash

Credits

Author: Taha Shieenavaz

GitHub Statistics

Data verified by PyPI on Aug 10, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT License (Creative Commons Attribution-NonCommercial 4.0 International)

Requires

Python >=3.9

Tags

reinforcement learning dqn aftab drl pqn deep reinforcement learning cnn

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

aftab-1.0.2.tar.gz (59.5 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

aftab-1.0.2-py3-none-any.whl (67.3 kB view details)

Uploaded Python 3

File details

Details for the file aftab-1.0.2.tar.gz.

File metadata

  • Download URL: aftab-1.0.2.tar.gz
  • Upload date:
  • Size: 59.5 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for aftab-1.0.2.tar.gz
Algorithm Hash digest
SHA256 1f61d76cd06479c53b8fb6efdb19e5d5c9f855caf4773ec5a824b937fd5d78d6
MD5 510b7cf69b7b8e1cb51fc042de9eb841
BLAKE2b-256 a98ea034194817c587ab3f6bc7deca94b446f9788251a14825a8e9e415101a61

See more details on using hashes here.

Provenance

The following attestation bundles were made for aftab-1.0.2.tar.gz:

Publisher: publish.yaml on tahashieenavaz/aftab

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file aftab-1.0.2-py3-none-any.whl.

File metadata

  • Download URL: aftab-1.0.2-py3-none-any.whl
  • Upload date:
  • Size: 67.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for aftab-1.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 d835ff23f6d47d95e1dd434ca9a23c5bbf332721bc9e06fe075721cb6e2e9d29
MD5 185ea092107c86ca9e164240291f2b74
BLAKE2b-256 9bb7e107b68e0ceb1284207882a0471bef518605d9b939ec38e126fa899e0ea3

See more details on using hashes here.

Provenance

The following attestation bundles were made for aftab-1.0.2-py3-none-any.whl:

Publisher: publish.yaml on tahashieenavaz/aftab

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

1.0.5

2 files

1.0.4

2 files

1.0.3

2 files

This release

1.0.2 This release

2 files

1.0.1

2 files

1.0.0

2 files

0.1.61

2 files

0.1.60

2 files

0.1.59

2 files

0.1.58

2 files

0.1.57

2 files

0.1.56

2 files

0.1.55

2 files

0.1.54

2 files

0.1.53

2 files

0.1.52

2 files

0.1.51

2 files

0.1.50

2 files

0.1.49

2 files

0.1.48

2 files

0.1.47

2 files

0.1.46

2 files

0.1.45

2 files

0.1.44

2 files

0.1.43

2 files

0.1.42

2 files

0.1.41

2 files

0.1.40

2 files

0.1.39

2 files

0.1.38

2 files

0.1.37

2 files

0.1.36

2 files

0.1.35

2 files

0.1.34

2 files

0.1.33

2 files

0.1.31

2 files

0.1.30

2 files

0.1.29

2 files

0.1.28

2 files

0.1.27

2 files

0.1.26

2 files

0.1.25

2 files

0.1.24

2 files

0.1.23

2 files

0.1.22

2 files

0.1.21

2 files

0.1.20

2 files

0.1.19

2 files

0.1.18

2 files

0.1.16

2 files

0.1.15

2 files

0.1.14

2 files

0.1.13

2 files

0.1.12

2 files

0.1.11

2 files

0.1.1

2 files

0.1.0

2 files

0.0.99

2 files

0.0.97

2 files

0.0.96

2 files

0.0.95

2 files

0.0.94

2 files

0.0.93

2 files

0.0.92

2 files

0.0.91

2 files

0.0.90

2 files

0.0.89

2 files

0.0.88

2 files

0.0.87

2 files

0.0.86

2 files

0.0.85

2 files

0.0.84

2 files

0.0.83

2 files

0.0.82

2 files

0.0.81

2 files

0.0.80

2 files

0.0.79

2 files

0.0.78

2 files

0.0.77

2 files

0.0.76

2 files

0.0.75

2 files

0.0.74

2 files

0.0.73

2 files

0.0.72

2 files

0.0.71

2 files

0.0.70

2 files

0.0.69

2 files

0.0.68

2 files

0.0.67

2 files

0.0.66

2 files

0.0.65

2 files

0.0.64

2 files

0.0.63

2 files

0.0.62

2 files

0.0.61

2 files

0.0.60

2 files

0.0.59

2 files

0.0.58

2 files

0.0.57

2 files

0.0.56

2 files

0.0.55

2 files

0.0.54

2 files

0.0.53

2 files

0.0.52

2 files

0.0.51

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page