csle-collector
This library contains scripts and programs for collecting data from the emulation.
Re-generate gRPC files
To re-generate the gRPC files, run:
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/client_manager/. ./protos/client_manager.proto --mypy_out=./src/csle_collector/client_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/kafka_manager/. ./protos/kafka_manager.proto --mypy_out=./src/csle_collector/kafka_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/elk_manager/. ./protos/elk_manager.proto --mypy_out=./src/csle_collector/elk_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/docker_stats_manager/. ./protos/docker_stats_manager.proto --mypy_out=./src/csle_collector/docker_stats_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/snort_ids_manager/. ./protos/snort_ids_manager.proto --mypy_out=./src/csle_collector/snort_ids_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/host_manager/. ./protos/host_manager.proto --mypy_out=./src/csle_collector/host_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/ossec_ids_manager/. ./protos/ossec_ids_manager.proto --mypy_out=./src/csle_collector/ossec_ids_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/traffic_manager/. ./protos/traffic_manager.proto --mypy_out=./src/csle_collector/traffic_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/ryu_manager/. ./protos/ryu_manager.proto --mypy_out=./src/csle_collector/ryu_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/five_g_core_manager/. ./protos/five_g_core_manager.proto --mypy_out=./src/csle_collector/five_g_core_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/five_g_cu_manager/. ./protos/five_g_cu_manager.proto --mypy_out=./src/csle_collector/five_g_cu_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/five_g_du_manager/. ./protos/five_g_du_manager.proto --mypy_out=./src/csle_collector/five_g_du_manager/.
Requirements
- Python 3.8+
csle-basegrpcio(for the collector API)grpcio-tools(for the collector API)scipy(for statistical models of client processes)confluent-kafka(for interacting with Kafka)docker(for interacting with Docker)requests(for interacting with the Ryu Controller's web interface)netifaces(networking utility)PyYaml(for parsing yaml files)pycryptodome(for cryptographic functions)websockets(for monitoring 5G networks)websocket-client(for monitoring 5G networks)
Development Requirements
- Python 3.8+
flake8(for linting)flake8-rst-docstrings(for linting docstrings)tox(for automated testing)pytest(for unit tests)pytest-cov(for unit test coverage)mypy(for static typing)mypy-extensions(for static typing)mypy-protobuf(for static typing)types-PyYaml(for static typing)types-paramiko(for static typing)types-protobuf(for static typing)types-requests(for static typing)types-urllib3(for static typing)sphinx(for API documentation)sphinxcontrib-napoleon(for API documentation)sphinx-rtd-theme(for API documentation)
Installation
# install from pip
pip install csle-collector==<version>
# local install from source
$ pip install -e csle-collector
# or (equivalently):
make install
# force upgrade deps
$ pip install -e csle-collector --upgrade
# git clone and install from source
git clone https://github.com/Kim-Hammar/csle
cd csle/simulation-system/libs/csle-collector
pip3 install -e .
# Install development dependencies
$ pip install -r requirements_dev.txt
Development tools
Install all development tools at once:
make install_dev
or
pip install -r requirements_dev.txt
API documentation
This section contains instructions for generating API documentation using sphinx.
Latest Documentation
The latest documentation is available at https://kim-hammar.github.io/csle//docs/csle-collector
Generate API Documentation
First make sure that the CSLE_HOME environment variable is set:
echo $CSLE_HOME
Then generate the documentation with the commands:
cd docs
sphinx-apidoc -f -o source/ ../src/csle_collector/
make html
To update the official documentation at https://kim-hammar.github.io/csle/, copy the generated HTML files to the documentation folder:
cp -r build/html ../../../../docs/_docs/csle-collector
To run all documentation commands at once, use the command:
make docs
Static code analysis
To run the Python linter, execute the following command:
flake8 .
# or (equivalently):
make lint
To run the mypy type checker, execute the following command:
mypy .
# or (equivalently):
make types
Unit tests
To run the unit tests, execute the following command:
pytest
# or (equivalently):
make unit_tests
To run tests of a specific test suite, execute the following command:
pytest -k "ClassName"
To generate a coverage report, execute the following command:
pytest --cov=csle_collector
Run tests and code analysis in different python environments
To run tests and code analysis in different python environemnts, execute the following command:
tox
# or (equivalently):
make tests
Create a new release and publish to PyPi
First build the package by executing:
python3 -m build
# or (equivalently)
make build
After running the command above, the built package is available at ./dist.
Push the built package to PyPi by running:
python3 -m twine upload dist/*
# or (equivalently)
make push
To run all commands for the release at once, execute:
make release
Author & Maintainer
Kim Hammar kimham@kth.se
Copyright and license
Creative Commons
(C) 2020-2026, Kim Hammar
1 maintainer
Metadata
1 maintainer
Release files for csle-collector 0.10.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| csle_collector-0.10.1.tar.gz | 203.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| csle_collector-0.10.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 448.2 kB
Release files / csle_collector-0.10.1.tar.gz
| Download URL | csle_collector-0.10.1.tar.gz |
|---|---|
| Size | 203.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bfef2c1ab3f5904c90662261e0fd5fce54e17836086cba676c3a663cec684ab5
|
|
BLAKE2b-256 checksum How to use checksums |
56c0ac92df4b20d31450d0c79a6748cc7b41fdd2ff8c3a66b0d58aee934d5217
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.11
|
Release files / csle_collector-0.10.1-py3-none-any.whl
| Download URL | csle_collector-0.10.1-py3-none-any.whl |
|---|---|
| Size | 245.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
09d2bd6ea53bac340ca798d78d583285d80404de5ec9f44b87259861b56e2c7e
|
|
BLAKE2b-256 checksum How to use checksums |
888d047b39adec159b568ac173d5e5f222e47ab619d8d84ca4340e750ed2ca4f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.11
|
