Host stats
Project description
hoststats
hoststats
captures resource usage (CPU, memory, network, disk) on a set of
remote hosts over a period of time.
Collection can be started and stopped from a client host via HTTP or the included Python API. Results are written to a CSV file on the client machine.
The hoststats
server
The hoststats
server must run on each host from which you wish to collect
metrics, and port 5000
must be accessible to the client.
Using Docker
# Run container in background
docker run -d --rm -p 5000:5000 shillaker/hoststats:0.0.5
# Check
curl http://localhost:5000/ping
Using Pip
# Install
pip3 install hoststats
# Start the server in the background, e.g.
nohup hoststats start > /var/log/hoststats.log 2>&1 &
# Check
curl http://localhost:5000/ping
The hoststats
client
The hoststats
client host can access hoststats
servers in two ways.
Python API
from hostats.client import HostStats
# Create list of IPs/ hostnames for target hosts
ip_list = ["1.2.3.4", "5.6.7.8"]
# Set up the client
hs = HostStats(ip_list)
# Start collection
hs.start_collection()
# Wait some time
# Write stats to CSV
hs.stop_and_write_to_csv("hoststats.csv")
HTTP API
Note that although the HTTP API works, the data that comes out requires more processing.
# Check a given host is running the server and accessible
curl http://<target_host>:5000/ping
# Start the recording
curl http://<target_host>:5000/start
# Wait some time
# Get stats as JSON
curl http://<target_host>:5000/stop > /tmp/hoststats.json
Proxies
If your client host can't directly access the target hosts, you can specify a
proxy server, which must also have the hoststats
server running. This proxy
can also be included in the list of target hosts.
This can be useful in environments like Kubernetes deployments, where you'd get
a list of the internal IPs/ hostnames, then use a single externally accessible
endpoint or stand-alone hoststats
container to access those internal hosts.
To use a proxy, you just need to provide an extra argument to the HostStats
constructor:
from hostats.client import HostStats
# List of IPs/ hostnames accessible from the proxy
ip_list = ["1.2.3.4", "5.6.7.8"]
# Proxy IP/ hostname accessible from the client
proxy_ip = "9.8.7.6"
# Set up the client
hs = HostStats(ip_list, proxy=proxy_ip)
Handling results
If the data has been written to CSV via the Python API, you can access the data
with the HostStatsResults
class:
from hoststats.results import HostStatsResults
csv_file = "hoststats.csv"
s = HostStatsResults(csv_file)
# Get list of hosts in results
s.get_hosts()
# Get list of available stats
s.get_stats()
# Get timeseries of given stat per host
s.get_stat_per_host("CPU_PCT")
# Get average stat across hosts
s.get_avg_stat("MEMORY_USED")
Development
Ensure pip
and setuptools
are up to date and install requirements.
To develop:
pip3 install -e .
Run tests:
./bin/tests.sh
Developing on a local cluster
If you want to run distributed tests against your local modifications, you can run the following:
# Start up some hoststats containers and enter the client container
./bin/dev.sh
From within this container, run tests:
# Non-distributed tests
nosetests hoststats.tests --nocapture
# Distributed tests
nosetests hoststats.disttest --nocapture
You can then edit files and restart the target containers with:
./bin/dev_restart.sh
Once restarted, you can rerun the tests against servers with your changes.
See the scripts mentioned above and
docker-compose-dev.yml
for more info.
Releasing
To push to PyPI, make sure you have set up Twine keyring
support, or a
pypirc
.
Then increment the version in VERSION
.
Then:
# Tag the code
./bin/tag.sh
# Build the Docker image
./bin/build.sh
# Check the distributed tests passs
./bin/dist_test.sh
# Push the package
./bin/release.sh
Once everything looks good, create a release manually on Github.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for hoststats-0.0.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3cd306b525207afaf8f0992f04b412628ba880a1132861c2e03d315f00a29ef8 |
|
MD5 | b4d3a5769512911c7febda38425eeaea |
|
BLAKE2b-256 | 4bc6be6227b6ba0429a444b50de0229911f977a692134d110b29d6bb7de4153e |