A file system monitor that can publish file change events to MQTT broker.
Project description
fs-monitor-mqtt
Overview
The monitor script listens to filesystem events such as created
, modified
, deleted
, moved
, and publishes them to a MQTT broker.
The topic of the published message is the absolute path of the changed file that is being monitored. An example payload is a JSON object as follows:
{
"timestamp": "2023-10-03T15:42:20.373182",
"event_type": "modified",
"is_directory": false,
"src_path": "/tmp/1"
}
Dependencies
- watchdog: Cross-platform file system events monitoring.
- paho-mqtt: MQTT client library.
- typer: CLI application framework based on Python type hints.
More dependencies can be found in pyproject.toml
.
PyPI Package
The package is available on PyPI.
Installation and Usage
pip install fs-monitor-mqtt
fs-monitor-mqtt --path /tmp --address localhost --port 1883
Log level
The default log level is INFO
. To change the log level, set the environment variable LOG_LEVEL
to one of the following values: DEBUG
, INFO
, WARNING
, ERROR
, CRITICAL
.
For example,
LOG_LEVEL=DEBUG fs-monitor-mqtt --path /tmp --address localhost --port 1883
Run with Docker
Prerequisites
- Docker
Instructions
-
Build the Docker image including the application and its dependencies.
docker build -t fs-monitor-mqtt .
-
Start a MQTT broker.
docker run -d -p 1883:1883 --name mosquitto eclipse-mosquitto mosquitto -c /mosquitto-no-auth.conf
-
Start a MQTT client that subscribes to all MQTT topics (so that messages can be received and verified easily).
docker run --network=host eclipse-mosquitto mosquitto_sub -t '#' -h 'localhost' -p 1883
-
In a separate terminal, start
fs-monitor-mqtt
docker container in the background. The default configuration is to monitor the/tmp
directory, publish to the MQTT broker running on the host machine with the default port of1883
.docker run -d --network=host --name fs-monitor-mqtt fs-monitor-mqtt --path /tmp --address localhost --port 1883
-
(optional) Follow the output of the
fs-monitor-mqtt
container.docker logs fs-monitor-mqtt -f
-
Make a change to the file system that is being monitored. For example, create a new file.
docker exec fs-monitor-mqtt touch /tmp/1
-
Verify that the MQTT client receives the message. For example,
{"timestamp": "2023-10-03T20:03:55.026184", "event_type": "created", "is_directory": false, "src_path": "/tmp/1"} {"timestamp": "2023-10-03T20:03:55.026759", "event_type": "modified", "is_directory": true, "src_path": "/tmp"} {"timestamp": "2023-10-03T20:03:55.026860", "event_type": "modified", "is_directory": false, "src_path": "/tmp/1"} {"timestamp": "2023-10-03T20:03:55.026920", "event_type": "modified", "is_directory": true, "src_path": "/tmp"}
-
Stop the
fs-monitor-mqtt
container.docker rm -f fs-monitor-mqtt
-
Stop the MQTT broker.
docker rm -f mosquitto
Development
Prerequisites
- Python > 3.9
Instructions
Run default make
command which does the following:
- install poetry
- install dependencies
- install pre-commit hooks
- run unit tests (Pytest)
- start the MQTT broker
- run integration tests (Robot)
- finally stopping the MQTT broker.
Check more commands in the Makefile
.
Unit Tests
Run unit tests with make test
. The unit tests are written using Pytest.
Integration Tests
Run integration tests with make integration_test
. The integration tests are written using Robot Framework.
Future Work
- Health check mechanism.
- Performance tests.
- GitHub Actions for CI/CD.
- Add more unit tests.
- Add more integration tests, for example: MQTT connection failure.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for fs_monitor_mqtt-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f071066980ceb8ad4ebf2fb2ba52c29bd06950fbdacfad8f8d8a4aa139d7f492 |
|
MD5 | 5de493a639c08108e2ea341f3345c11c |
|
BLAKE2b-256 | 9f6bd4fe95e9e39875fa92bc074c801b63debc9bfffdccf67d546dcb9290b3fa |