catsim 0.20.0

Computerized Adaptive Testing Simulator

A Python package for computerized adaptive testing (CAT) simulations. It provides multiple methods for:

• test initialization (selecting the initial ability of the examinees)
• item selection
• ability estimation
• test stopping

These methods can either be used in a standalone fashion to power other software or be used with catsim to simulate the application of computerized adaptive tests, given a sample of examinees, represented by their ability levels, and an item bank, represented by their parameters according to some logistic Item Response Theory model.

Installation

Install it using pip install catsim.

Basic Usage

There is a Colab Notebook teaching the basics of catsim!

1. Have an item matrix;
2. Have a sample of examinee proficiencies, or a number of examinees to be generated;
3. Create an initializer, an item selector, a ability estimator and a stopping criterion;
4. Pass them to a simulator and start the simulation.
5. Access the simulator's properties to get specifics of the results;
6. Plot your results.

from catsim.initialization import RandomInitializer
from catsim.selection import MaxInfoSelector
from catsim.estimation import NumericalSearchEstimator
from catsim.stopping import MinErrorStopper
from catsim.simulation import Simulator
from catsim import ItemBank
initializer = RandomInitializer()
selector = MaxInfoSelector()
estimator = NumericalSearchEstimator()
stopper = MinErrorStopper(0.4, max_items=20)
Simulator(ItemBank.generate_item_bank(100), 10).simulate(initializer, selector, estimator, stopper)

Dependencies

All dependencies are listed on setup.py and should be installed automatically.

To run the tests, you will need to install the testing requirements with pip install catsim[testing].

To generate the documentation, install the necessary dependencies with pip install catsim[docs].

To ensure code is valid and formatted before submission, install the necessary development dependencies with pip install catsim[dev].

Compatibility

catsim is compatible and tested against Python 3.7, 3.8, 3.9, 3.10, 3.11, 3.12.

Important links

• Official source code repo: https://github.com/douglasrizzo/catsim
• HTML documentation (stable release): https://douglasrizzo.com.br/catsim
• Issue tracker: https://github.com/douglasrizzo/catsim/issues

Citing catsim

You can cite the package using the following bibtex entry:

@article{catsim,
  author = {Meneghetti, Douglas De Rizzo and Aquino Junior, Plinio Thomaz},
  title = {Application and simulation of computerized adaptive tests through the package catsim},
  year = 2018,
  month = jul,
  archiveprefix = {arXiv},
  eprint = {1707.03012},
  eprinttype = {arxiv},
  journal = {arXiv:1707.03012 [stat]},
  primaryclass = {stat}
}

What's a CAT

Computerized adaptive tests are educational evaluations, usually taken by examinees in a computer or some other digital means, in which the examinee's ability is evaluated after the response of each item. The new ability is then used to select a new item, closer to the examinee's real ability. This method of test application has several advantages compared to the traditional paper-and-pencil method or even linear tests applied electronically, since high-ability examinees are not required to answer all the easy items in a test, answering only the items that actually give some information regarding his or hers true knowledge of the subject at matter. A similar, but inverse effect happens for those examinees of low ability level.

More information is available in the docs and over at Wikipedia.

If you are looking for IRT item parameter estimation

catsim does not implement item parameter estimation. I have had great joy outsourcing that functionality to the mirt R package along the years. However, since many users request packages with item parameter estimation capabilities in the Python ecosystem, here are a few links. While I have not used them personally, specialized packages like these are hard to come by, so I hope these are helpful.

• eribean/girth
• eribean/girth_mcmc
• nd-ball/py-irt