This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Metric collection agent for Numenta Rogue showcase application

Project Description
Numenta Rogue
=============

Metrics collection agent for the Numenta Rogue showcase application and
consists of two primary components: A long-running metric collection agent,
which periodically polls various metrics and records the results to a local
database, and a separate process for forwarding metrics to a Grok server for
analysis.

Each metric collection agent is executed in a continuously-running gevent
Greenlet, with a pre-determined wait interval between samples. Each sample
is cached locally in a dedicated RRDTool database. Individual metrics are
implemented as subclasses of `avogadro.agent.AvogadroAgent()`, and must
implement a `collect()` method, which returns the metric value.

The `rogue-forward` process fetches metric data from the individual RRDTool
databases using the `rrdtool fetch` command, and sends the metric data using
the Grok Custom Metric API. Each time the `rogue-forward` process is
executed, the most recent timestamp is cached locally, and is used as a
starting point for subsequent requests.

Usage
-----

Install RRDTool::

brew install rrdtool

Install in development mode::

python setup.py develop --install-dir=... --script-dir=...

Start key-counter with nohup::

nohup rogue-keycounter > rogue-keys.stdout 2> rogue-keys.stderr < /dev/null &

Make sure that iTerm/Terminal is allowed in System Prefrences > Security & Privacy > Privacy > Accessibility!

Start metric collection agent with nohup::

nohup rogue-agent --prefix=var/db/ --interval=300 --hearbeat=600 > rogue-agent.stdout 2> rogue-agent.stderr < /dev/null &

Forward pending metric data to Grok once::

rogue-forward --server=https://localhost --prefix=var/db

Sample crontab entry::

* * * * * PATH=$PATH:/usr/local/bin PYTHONPATH=... .../rogue-forward --server=... --prefix=... > .../rogue-forward.stdout 2> .../rogue-forward.stderr < /dev/null

Export to local CSV
~~~~~~~~~~~~~~~~~~~

All metric data is written to a local round-robin database, which only retains
the two most recent weeks of data at any given moment in time. Should you want
the data exported to CSV, use the `rogue-export` utility::

rogue-export --prefix=var/db

The exporter keeps track of the position of a given metric, so you can run
`rogue-export` as frequently as you like, and the `.csv` files in `var/db/`,
will be updated accordingly. i.e. you can periodically sync the round-robin
database to a csv file in `var/db/`


Metrics
-------

CPUPercent
~~~~~~~~~~

Total CPU utilization as a percentage, as reported by ``psutil.cpu_percent()``.

MemoryPercent
~~~~~~~~~~~~~

The percentage memory usage calculated as `(total - available) / total * 100`,
as reported by ``psutil.virtual_memory().percent``

DiskReadBytes
~~~~~~~~~~~~~

Number of bytes read (total), as reported by
``psutil.disk_io_counters.read_bytes``

DiskWriteBytes
~~~~~~~~~~~~~

Number of bytes written (total), as reported by
``psutil.disk_io_counters.write_bytes``

DiskReadTime
~~~~~~~~~~~~

Time spent reading from disk (in milliseconds), as reported by
``psutil.disk_io_counters.read_time``

DiskWriteTime
~~~~~~~~~~~~~

Time spent writing to disk (in milliseconds), as reported by
``psutil.disk_io_counters.write_time``

NetworkBytesSent
~~~~~~~~~~~~~~~~

Number of bytes sent, as reported by ``psutil.net_io_counters.bytes_sent()``

NetworkBytesReceived
~~~~~~~~~~~~~~~~~~~~

Number of bytes received, as reported by
``psutil.net_io_counters.bytes_recv()``
Release History

Release History

This version
History Node

1.0.1

History Node

1.0

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
numenta_rogue-1.0.1-py2.7.egg (24.6 kB) Copy SHA256 Checksum SHA256 2.7 Egg Aug 5, 2015
numenta_rogue-1.0.1-py2-none-any.whl (26.6 kB) Copy SHA256 Checksum SHA256 2.7 Wheel Aug 5, 2015
numenta-rogue-1.0.1.tar.gz (25.2 kB) Copy SHA256 Checksum SHA256 Source Aug 5, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting