A web traffic generator based on SURGE, Statistics and Locust

# WTG-SEAL

A Web Traffic Generator based on SURGE, Statistics and Locust

## Overview

WTG-SEAL is a web traffic generator based on a statistical approach inspired by SURGE [1] and implemented using Locust.

In fact, it is expected that the input files for WTG-SEAL are the output files from SURGE, although one should be able to create their own files.

## Getting Started

### Installing with pip

To install the latest release of WTG-SEAL, just run

pip install wtg-seal


### Requirements

In order to generate your locustfile.py using WTG-SEAL, you will need of two text files formatted as follow:

#### Documents Definitions

A text file where each line describes a web document, which is defined as a collection of one or more objects. For instance, a docdef.txt with the following content:

1
5 8


maps to a document represented by a single file 1.txt and another document represented by the files 5.txt and 8.txt. Each document is identified by a sequential number (the order it appears on the file, starting from 0).

Ideally, this is the objout.txt file generated by SURGE.

### Documents Requests Sequence

A text file where each line contains a document identifier (i.e an integer) in the order they were to be requested to a web server. For instance, a docseq.txt with the following content:

0
0
1
0


represents requests to documents 0, 0, 1, 0. This is used by WTG-SEAL to calculate the weight of each locust task.

Ideally, this is the name.txt file generated by SURGE.

### Server Side

Considering the previous example, it would be necessary that files named 1.txt, 5.txt and 8.txt exist on a web-server to where the requests will be made.

### Generating web traffic

Once you have the appropriate files, just run

wtgseal docdef.txt docseq.txt


This will create a file in the current directory named locustfile.py.

From here, you will rely on Locust to generate the web traffic towards a web server of your choice. Just run

locust


## Contributing

For contributing or making your own modifications to the source code, run:

git clone https://github.com/mchoji/wtg-seal.git
cd wtg-seal
pipenv install -e .
pipenv install --dev


### Pre-commit hooks

In order to make sure the hooks will run, please don't forget to install the pre-commit package:

cd wtg-seal
pipenv install --dev
pipenv run pre-commit install
pipenv run pre-commit autoupdate


## Versioning

Versions defined according to SemVer. For the versions available, see the tags on this repository.

## References

[1] Barford, P., & Crovella, M. (1998, June). Generating representative web workloads for network and server performance evaluation. In Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems (pp. 151-160).

## Project details

Uploaded py2 py3