ethical-smart-grid 2.0.0

A RL environment for learning ethically-aligned behaviours in a Smart Grid simulator.

Ethical Smart Grid Simulator

Authors: Clément Scheirlinck, Rémy Chaput

Description

This is a third-party Gym environment, focusing on learning ethically-aligned behaviours in a Smart Grid use-case.

A Smart Grid contains several prosumer (prosumer-consumer) agents that interact in a shared environment by consuming and exchanging energy. These agents have an energy need, at each time step, that they must satisfy by consuming energy. However, they should respect a set of moral values as they do so, i.e., exhibiting an ethically-aligned behaviour.

Moral values are encoded in the reward functions, which determine the "correctness" of an agent's action, with respect to these moral values. Agents receive rewards as feedback that guide them towards a better behaviour.

Installation

You may install Ethical Smart Grid through:

• PyPi, using pip install ethical-smart-grid (latest stable version);
• pip and GitHub, using pip install git+https://github.com/ethicsai/ethical-smart-grid.git (you may specify the version at the end of the URL);
• GitHub, using git clone https://github.com/ethicsai/ethical-smart-grid (development version, not stable).

If you also wish to use argumentation-based reward functions, please install AJAR through pip install git+https://github.com/ethicsai/ajar.git@v1.0.0, or pip install -r requirements.txt if you cloned this repository.

Quick usage

After installing, open a Python shell (3.7+), and execute the following instructions:

from smartgrid import make_basic_smartgrid
from algorithms.qsom import QSOM

env = make_basic_smartgrid(max_step=10)
model = QSOM(env)

done = False
obs = env.reset()
while not done:
    actions = model.forward(obs)
    obs, rewards, terminated, truncated, _ = env.step(actions)
    print(rewards)
    model.backward(obs, rewards)
    done = all(terminated) or all(truncated)

env.close()

This will initialize a SmartGrid environment, learning agents that use the QSOM algorithm, and run the simulation for 10 steps (configurable through the max_step=10 argument).

To go further, please refer to the documentation; the Custom scenario and Adding a new model pages can be particularly interesting to learn, respectively, how to configure the environment, and how to implement a new learning algorithm. Finally, extending the environment allows creating new components (agents' profiles, reward functions, ...) to further customize the environment.

Versioning

This project follows the Semver (Semantic Versioning): all versions respect the <major>.<minor>.<patch> format. The patch number is increased when a bugfix is released. The minor number is increased when new features are added that do not break the code public API, i.e., it is compatible with the previous minor version. Finally, the major number is increased when a breaking change is introduced; an important distinction is that such a change may not be "important" in terms of lines of code, or number of features modified. Simply changing a function's return type can be considered a breaking change in the public API, and thus worthy of a "major" update.

Building and testing locally

This GitHub repository includes actions that automatically test the package and build the documentation on each commit, and publish the package to PyPi on each release.

Instructions to perform these steps locally are given here, for potential new contributors or forks:

• Running the tests

Tests are defined using unittest and run through pytest; please install it first: pip install pytest. We must add the current folder to the PYTHONPATH environment variable to let pytest import the smartgrid module when executing the tests: export PYTHONPATH=$PWD (from the root of this repository). Then, launch all tests with pytest tests.

• Building the documentation

The documentation is built with Sphinx and requires additional requirements; to install them, use pip install -r docs/requirements.txt. Then, to build the documentation, use cd docs && make html. The built documentation will be in the docs/build/html folder. It can be cleaned using make clean while in the docs folder. Additionally, the source/modules folder is automatically generated from the Python docstrings in the source code; it can be safely deleted (e.g., with rm -r source/modules) to force re-building all documentation files.

• Building and publishing releases

This project uses hatch to manage the building and publishing process; please install it with pip install hatch first.

To build the package, use hatch build at the root of this repository. This will create the source distribution (sdist) at dist/ethica_smart_grid_simulator-<version>.tar.gz, and the built distribution (wheel) at dist/ethical_smart_grid_simulator-<version>-py3-none-any.whl.

To publish these files to PyPi, use hatch publish.

Community

The community guidelines are available in the CONTRIBUTING.md file; you can find a (short) summary below.

Getting support

If you have a question (something that is not clear, how to get a specific result, ...), do not hesitate to create a new Discussion under the Q&A category.

Please do not use the issue tracker for support, to avoid cluttering it.

Report a bug

If you found a bug (an error raised, or something not working as expected), you can report it on the Issue Tracker.

Please try to be as precise as possible.

Contributing

We very much welcome and appreciate contributions!

For fixing bugs, or improving the documentation, you can create a Pull Request.

New features are also welcome, but larger features should be discussed first in a new Discussion under the Ideas category.

All ideas, suggestions, and requests are also welcome for discussion.

License

The source code is licensed under the MIT License. Some included data may be protected by other licenses, please refer to the LICENSE.md file for details.

Citation

If you use this package in your research, please cite the corresponding paper:

Scheirlinck, C., Chaput, R., & Hassas, S. (2023). Ethical Smart Grid: a Gym environment for learning ethical behaviours. Journal of Open Source Software, 8(88), 5410. https://doi.org/10.21105/joss.05410

@article{Scheirlinck_Ethical_Smart_Grid_2023,
  author = {Scheirlinck, Clément and Chaput, Rémy and Hassas, Salima},
  doi = {10.21105/joss.05410},
  journal = {Journal of Open Source Software},
  month = aug,
  number = {88},
  pages = {5410},
  title = {{Ethical Smart Grid: a Gym environment for learning ethical behaviours}},
  url = {https://joss.theoj.org/papers/10.21105/joss.05410},
  volume = {8},
  year = {2023}
}