csle-collector
This library contains scripts and programs for collecting data from the emulation.
Re-generate gRPC files
To re-generate the gRPC files, run:
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/client_manager/. ./protos/client_manager.proto --mypy_out=./src/csle_collector/client_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/kafka_manager/. ./protos/kafka_manager.proto --mypy_out=./src/csle_collector/kafka_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/elk_manager/. ./protos/elk_manager.proto --mypy_out=./src/csle_collector/elk_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/docker_stats_manager/. ./protos/docker_stats_manager.proto --mypy_out=./src/csle_collector/docker_stats_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/snort_ids_manager/. ./protos/snort_ids_manager.proto --mypy_out=./src/csle_collector/snort_ids_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/host_manager/. ./protos/host_manager.proto --mypy_out=./src/csle_collector/host_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/ossec_ids_manager/. ./protos/ossec_ids_manager.proto --mypy_out=./src/csle_collector/ossec_ids_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/traffic_manager/. ./protos/traffic_manager.proto --mypy_out=./src/csle_collector/traffic_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/ryu_manager/. ./protos/ryu_manager.proto --mypy_out=./src/csle_collector/ryu_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/five_g_core_manager/. ./protos/five_g_core_manager.proto --mypy_out=./src/csle_collector/five_g_core_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/five_g_cu_manager/. ./protos/five_g_cu_manager.proto --mypy_out=./src/csle_collector/five_g_cu_manager/.
python -m grpc_tools.protoc -I./protos/ --python_out=./src/csle_collector/. --grpc_python_out=./src/csle_collector/five_g_du_manager/. ./protos/five_g_du_manager.proto --mypy_out=./src/csle_collector/five_g_du_manager/.
Requirements
- Python 3.8+
csle-basegrpcio(for the collector API)grpcio-tools(for the collector API)scipy(for statistical models of client processes)confluent-kafka(for interacting with Kafka)docker(for interacting with Docker)requests(for interacting with the Ryu Controller's web interface)netifaces(networking utility)PyYaml(for parsing yaml files)pycryptodome(for cryptographic functions)websockets(for monitoring 5G networks)websocket-client(for monitoring 5G networks)
Development Requirements
- Python 3.8+
flake8(for linting)flake8-rst-docstrings(for linting docstrings)tox(for automated testing)pytest(for unit tests)pytest-cov(for unit test coverage)mypy(for static typing)mypy-extensions(for static typing)mypy-protobuf(for static typing)types-PyYaml(for static typing)types-paramiko(for static typing)types-protobuf(for static typing)types-requests(for static typing)types-urllib3(for static typing)sphinx(for API documentation)sphinxcontrib-napoleon(for API documentation)sphinx-rtd-theme(for API documentation)
Installation
# install from pip
pip install csle-collector==<version>
# local install from source
$ pip install -e csle-collector
# or (equivalently):
make install
# force upgrade deps
$ pip install -e csle-collector --upgrade
# git clone and install from source
git clone https://github.com/Kim-Hammar/csle
cd csle/simulation-system/libs/csle-collector
pip3 install -e .
# Install development dependencies
$ pip install -r requirements_dev.txt
Development tools
Install all development tools at once:
make install_dev
or
pip install -r requirements_dev.txt
API documentation
This section contains instructions for generating API documentation using sphinx.
Latest Documentation
The latest documentation is available at https://kim-hammar.github.io/csle//docs/csle-collector
Generate API Documentation
First make sure that the CSLE_HOME environment variable is set:
echo $CSLE_HOME
Then generate the documentation with the commands:
cd docs
sphinx-apidoc -f -o source/ ../src/csle_collector/
make html
To update the official documentation at https://kim-hammar.github.io/csle/, copy the generated HTML files to the documentation folder:
cp -r build/html ../../../../docs/_docs/csle-collector
To run all documentation commands at once, use the command:
make docs
Static code analysis
To run the Python linter, execute the following command:
flake8 .
# or (equivalently):
make lint
To run the mypy type checker, execute the following command:
mypy .
# or (equivalently):
make types
Unit tests
To run the unit tests, execute the following command:
pytest
# or (equivalently):
make unit_tests
To run tests of a specific test suite, execute the following command:
pytest -k "ClassName"
To generate a coverage report, execute the following command:
pytest --cov=csle_collector
Run tests and code analysis in different python environments
To run tests and code analysis in different python environemnts, execute the following command:
tox
# or (equivalently):
make tests
Create a new release and publish to PyPi
First build the package by executing:
python3 -m build
# or (equivalently)
make build
After running the command above, the built package is available at ./dist.
Push the built package to PyPi by running:
python3 -m twine upload dist/*
# or (equivalently)
make push
To run all commands for the release at once, execute:
make release
Author & Maintainer
Kim Hammar kimham@kth.se
Copyright and license
Creative Commons
(C) 2020-2026, Kim Hammar
Metadata
Release files for csle-collector 0.10.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| csle_collector-0.10.1.tar.gz | 203.1 kB | Without provenance | Details for csle_collector-0.10.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| csle_collector-0.10.1-py3-none-any.whl | Python 3 | none | any | Details for csle_collector-0.10.1-py3-none-any.whl |
Total release size: 448.2 kB
Release files / csle_collector-0.10.1.tar.gz
| Download URL | csle_collector-0.10.1.tar.gz |
|---|---|
| Size | 203.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bfef2c1ab3f5904c90662261e0fd5fce54e17836086cba676c3a663cec684ab5
|
|
BLAKE2b-256 checksum How to use checksums |
56c0ac92df4b20d31450d0c79a6748cc7b41fdd2ff8c3a66b0d58aee934d5217
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.11
|
Release files / csle_collector-0.10.1-py3-none-any.whl
| Download URL | csle_collector-0.10.1-py3-none-any.whl |
|---|---|
| Size | 245.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
09d2bd6ea53bac340ca798d78d583285d80404de5ec9f44b87259861b56e2c7e
|
|
BLAKE2b-256 checksum How to use checksums |
888d047b39adec159b568ac173d5e5f222e47ab619d8d84ca4340e750ed2ca4f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.11
|
Security for csle-collector 0.10.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.