Skip to main content

Accuracy-based Learning Classifier Systems with Rule Combining

Project description

XCS-RC

Accuracy-based Learning Classifier Systems with Rule Combining mechanism, shortly XCS-RC for Python3, loosely based on Martin Butz's XCS Java code (2001). Read my PhD thesis here for the complete algorithmic description.

Rule Combining is novel function that employs inductive reasoning, replacing all Darwinian genetic operation like mutation and crossover. It can handle binaries and real, reaching better correctness rate and population size quicker than several XCS instances. My earlier papers comparing them can be obtained at here and here.


Relevant links


Installation

pip install xcs-rc

Initialization

import xcs_rc
agent = xcs_rc.Agent()

Classic Reinforcement Learning cycle

# input: binary string, e.g., "100110" or decimal array
state = str(randint(0, 1))

# pick methods: 0 = explore, 1 = exploit, 2 = explore_it
action = agent.next_action(state, pick_method=1)

# determine reward and apply it, e.g.,
reward = agent.maxreward if action == int(state[0]) else 0.0
agent.apply_reward(reward)

Partially Observable Markov Decision Process (POMDP) environment

# create env and agent
env = xcs_rc.MarkovEnv('maze4')  # maze4 is built-in
env.add_agents(num=1, tcomb=100, xmax=50)
agent = env.agents[0]

for episode in range(8000):
    steps = env.one_episode(pick_method=2)  # returns the number of taken steps

Data classification

agent.train(X_train, y_train)
cm = agent.test(X_test, y_test)  # returns the confusion matrix
preds, probs = agent.predict(X)  # returns lists of predictions and probabilities

Print population, save it to CSV file, or use append mode

agent.pop.print(title="Population")
agent.save('xcs_population.csv', title="Final XCS Population")
agent.save('xcs_pop_every_100_cycles.csv', title="Cycle: ###", save_mode='a')

Finally, inserting rules to population

# automatically load the last set (important for append mode)
agent.load("xcs_population.csv", empty_first=True)
agent.pop.add(my_list_of_rules)  # from a list of classifiers

Main Parameters

XCS-RC Parameters

  • tcomb: combining period, number of learning cycles before the next rule combining
  • predtol: prediction tolerance, maximum difference between two classifiers to be combined
  • prederrtol: prediction error tolerance, threshold for deletion of inappropriately combined rules

How to Set

agent.tcomb = 50 # perform rule combining every 50 cycles
agent.predtol = 20.0 # combines rules whose prediction value differences <= 20.0
agent.prederrtol = 10.0 # remove if error > 10.0, after previously below it

Parameters from original XCS

  • all related to mutation and crossover is removed
  • the others are kept and accessible (e.g., agent.alpha = 0.15)

Results

Classical Problems: multiplexer and Markov environment:

Binary MP11-HIGH

Real MP6-HIGH

Markov Maze4

Flappy Bird from PyGame Learning Environment:

Flappy Bird XCS-RC plot

Flappy Bird XCS-RC youtube

Youtube: CartPole-v0 Benchmark from OpenAI Gym:

CartPole XCS-RC

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xcs-rc-1.0.0.tar.gz (14.4 kB view hashes)

Uploaded source

Built Distribution

xcs_rc-1.0.0-py3-none-any.whl (12.3 kB view hashes)

Uploaded py3

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page