Neuralpy — teach a neural network by letting it try

neuralpy is a small Python package for training a real, fully connected neural network from scratch with reinforcement learning. The learner is a Deep Q-Network (DQN) with experience replay and a target network. It uses NumPy for matrix arithmetic; no pretrained model, API key, or internet connection is required to train.

This is version 0.1.0, designed to be understandable and hackable.

Install

With Python 3.10 or newer, open a terminal in this folder:

python -m pip install -e .

Run the included working game:

python examples/treasure_game.py

This trains from scratch, reports progress, tests 100 new games, and saves treasure_brain.npz.

First steps

from neuralpy import Brain

# Actions that YOUR game recognizes; no open-source AI model involved.
brain = Brain(actions=["left", "right"], goal=my_reward_function)

# game.reset() starts a level and returns a numeric list, e.g. [0.25, 0.80]
# game.step(action) updates the game and returns (new_list, finished, info_dict)
history = brain.learn(game, episodes=500, max_steps=100)

brain.save("my_brain")
result = brain.play(game)
print(result)

The goal is a function returning a reward after each action, for example:

def my_reward_function(game):
    if game.won:
        return 100
    if game.dead:
        return -100
    return -0.01  # Small penalty encourages finishing quickly

It does not understand a plain-English goal automatically. The game/developer must make success measurable.

Build your own game

Every game needs these two methods:

class MyGame:
    def reset(self):
        # Reset a new level. Return a 1D list of input numbers.
        self.player_x = 0
        self.finished = False
        return [self.player_x / 10]

    def step(self, action):
        # Called once for each action the brain chooses.
        if action == "right":
            self.player_x += 1
        elif action == "left":
            self.player_x -= 1
        self.player_x = max(0, min(10, self.player_x))
        self.finished = self.player_x == 10
        return [self.player_x / 10], self.finished, {"won": self.finished}


def goal(game):
    return 10 if game.finished else -0.01


game = MyGame()
brain = Brain(actions=["left", "right"], goal=goal)
brain.learn(game, episodes=300, max_steps=25)
print(brain.play(game, max_steps=25))

Built-in ways to define a goal

Instead of writing a reward function, choose one:

# Give the agent the CHANGE in score at each step:
Brain(actions=["left", "right"], goal="maximize", target="score")

# Reward every step the game remains active:
Brain(actions=["dodge", "wait"], goal="survive")

# Reward reaching a game field called finish:
Brain(actions=["left", "right"], goal="reach", target="finish")

# Reward events listed in step info (e.g. {"events": ["coin"]}):
Brain(actions=["left", "right"], goals={"coin": 5, "win": 100, "die": -50})

For maximize and reach, target must be a game attribute or a key in the info dictionary returned by step(). maximize requires a numeric starting target available after reset(). With the events option, emit events on the step when they occur; info={"events": ["coin"]} or info={"coin": True} works.

If your environment already returns its own reward, leave out goal:

Brain(actions=[0, 1])  # The environment returns (obs, reward, done, info)

It also accepts Gymnasium-style reset() -> (observation, info) and step() -> (observation, reward, terminated, truncated, info); you still supply an appropriate action list.

Commands

If your reward is a custom Python function, pass it again when resuming training because Python functions are intentionally not serialized:

brain = Brain.load("my_brain", goal=my_reward_function)
brain.learn(game, episodes=100)

Loading without the callback is fine for calling brain.think(...) or brain.play(...).

How it learns

1. The game provides its current numeric state (e.g. player position and treasure position).
2. At the beginning, the brain often selects a random action.
3. The game responds with a new observation and any progress/failure signals.
4. The goal function computes a numeric reward.
5. The brain stores (state, action, reward, next_state, finished) in replay memory.
6. A feedforward neural network learns to estimate the expected future reward of each action (Q-values), using gradient descent, replay batches, and a target network.
7. Over episodes, the brain explores less and increasingly chooses actions it expects to work.

The network is built and trained using NumPy, without any teacher language model.

Limits of this first version

• Discrete actions only, like "jump", "left", or 0 and 1.
• Fixed-length, numeric, 1D observations only. A screenshot, sound clip, or raw text requires a separate encoder.
• Uses a straightforward DQN, which is not guaranteed to learn every game. Complex tasks need thoughtful rewards, more compute, and perhaps more advanced algorithms.
• The game needs a reset() method and a step(action) method. A free-form arbitrary application is not automatically controllable.
• Saving persists weights and settings, not the replay buffer; training can resume with fresh experience.
• Results vary by reward design, random exploration, game difficulty, and training length. Even a trained agent can make mistakes.

Development / tests

python -m pip install -e '.[dev]'
python -m pytest -q

The project uses the MIT license.