A reinforcement learning environment for the Multi Port Stowage Planning problem
Project description
This package implements the mathematical model of the "Multi Port Stowage Planning" (MPSP) problem, introduced in Avriel et al. 1998, as a reinforcement learning environment using the Gymnasium API.
The Env class has the following input parameters:
- R (int): number of rows
- C (int): number of columns
- N (int): number of ports
The env observations consist of a dictionary with the one hot encoded bay, and the upper triangular part of the transportation matrix as a flattened array.
Screenshot from the environment. The left matrix is the bay, and the right matrix is the transportation matrix. "x" denotes blocking containers. The reward decreases by one for each reshuffle necessitated by the agents stowage plan.
Note that since the mathematical model disregards any stability constraints, the env automatically swaps the column order in a lexocographic manor, based on the containers in the columns. This reduces the state space, by a factor of almost $C!$. Furthmore, the transportation matrix, is reshifted up and to the left, for each port, to reduce the state space even further. This means that from the point of view of the agent, it never leaves the first port.
When using the environment, you should call env.close()
before exiting your program, to avoid memory leaks.
Minimal usage with Stable Baselines:
from MPSPEnv import Env
from sb3_contrib.ppo_mask import MaskablePPO
train_env = Env(R=8, C=4, N=5)
model = MaskablePPO(
"MultiInputPolicy",
train_env
)
model.learn(
total_timesteps=1e6,
progress_bar=True
)
train_env.close()
If you wish to try the env yourself, placing containers using keys 0-3 and removing containers using keys 4-7, you can use the following script:
from MPSPEnv import Env
import pygame, numpy as np
def get_action_from_key(key):
return key - pygame.K_0 if pygame.K_0 <= key <= pygame.K_7 else None
if __name__ == "__main__":
env = Env(3, 4, 6)
env.reset()
while True:
env.render()
for event in pygame.event.get():
if event.type == pygame.QUIT:
env.close()
exit()
elif (
event.type == pygame.KEYDOWN
and (action := get_action_from_key(event.key)) is not None
):
_, reward, done, _, _ = env.step(action)
if done:
env.render()
pygame.time.wait(1000)
env.close()
exit()
pygame.time.wait(10)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for MPSPEnv-0.1.5-cp311-cp311-macosx_14_0_arm64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ecb42d24382140ce09adc4dfb8158798de941988d1b2667d70920dfcca1a75a1 |
|
MD5 | 6f380b4ed6ab341e3de7a1d3475d1746 |
|
BLAKE2b-256 | ba12488a5604d9395dda21701f019d1e0d97e1b440b4d753def60883968a4db5 |