Tusk Clickhouse Metric Writer
contek-tusk is a metric writer built upon Clickhouse.
1. Connection Initialization
import contek_tusk as tusk
tusk.init(host='localhost', user='tester', password='P@assw0rd')
2. Generic Metric
from contek_tusk import metric
cpu_usage = metric(table='system.cpu')
cpu_usage.write({
'region': 'Tokyo',
'instance_id': 'ABC',
'usage': 0.43
})
cpu_usage.write({
'region': 'Seoul',
'instance_id': 'XYZ',
'usage': 0.22
})
SELECT * FROM system.cpu
| region | instance_id | datetime | usage |
|---|---|---|---|
| Tokyo | ABC | 2022-03-10 06:42:42.746 | 0.43 |
| Seoul | XYZ | 2022-03-10 06:42:42.746 | 0.22 |
Note: the client will automatically use the first DateTime column to record current timestamp. Alternatively, it
can be explicitly specified:
from contek_tusk import metric
from contek_tusk.table import Table
table = Table(database='system', table_name='cpu', time_column='datetime')
cpu_usage = metric(table)
3. Summing Counter
from contek_tusk.counter import counter
counter = counter(table='download.quota_usage')
counter.count({
'resource_id': 'ABC111',
'requester_ip': '1.2.3.4'
}) # default count is 1
counter.count({
'resource_id': 'ABC122',
'requester_ip': '2.2.2.4'
}, 3)
counter.count({
'resource_id': 'ABC111',
'requester_ip': '1.2.3.4'
}, 4) # the new count is 1 + 4 = 5
counter.count({
'resource_id': 'ABC122',
'requester_ip': '1.2.3.4'
}, 2)
SELECT * FROM download.quota_usage
| resource_id | requester_ip | datetime | count |
|---|---|---|---|
| ABC111 | 1.2.3.4 | 2022-03-10 06:42:48.411 | 5 |
| ABC122 | 1.2.3.4 | 2022-03-10 06:42:48.411 | 2 |
| ABC122 | 2.2.2.4 | 2022-03-10 06:42:48.411 | 3 |
Note: the client will automatically use the first Int or UInt column to record the count. Alternatively, it
can be explicitly specified:
from contek_tusk.counter import counter
counter = counter(table='download.quota_usage', count_column='count')
4. Heartbeat Recorder
from contek_tusk.heartbeat import init
init(table='maintenance.heartbeats', app_name='my_app')
After initialization, a recurring task starts in the background and sends a heartbeat to the database every 30 seconds. The heartbeat cycle can be explicitly specified:
from contek_tusk.heartbeat import init
init(table='maintenance.heartbeats', app_name='my_app', heartbeat_period=15)
If the application is running multiple recurring tasks. Their heartbeats can also be registered:
from contek_tusk.heartbeat import beat
beat(task='scan_disk', heartbeat_period=60)
SELECT * FROM maintenance.heartbeats
| app | task | datetime | sequence |
|---|---|---|---|
| my_app | main | 2022-03-10 06:51:01.178 | 0 |
| my_app | main | 2022-03-10 06:51:31.178 | 1 |
| my_app | main | 2022-03-10 06:52:01.178 | 2 |
| my_app | scan_disk | 2022-03-10 06:52:04.215 | 0 |
| my_app | main | 2022-03-10 06:52:31.178 | 3 |
| my_app | main | 2022-03-10 06:53:01.178 | 4 |
| my_app | scan_disk | 2022-03-10 06:53:04.215 | 1 |
5.Logging
import logging
from contek_tusk.logging import init
logger = logging.getLogger(__name__)
init(table='maintenance.logs', app_name='my_app')
logger.info('Hello world!')
logger.warning('Fire drill test.')
logger.error('This is real fire.')
try:
raise ValueError('Guess what?')
except ValueError:
logger.exception('Oh no!')
logger.critical('I give up.')
SELECT * FROM maintenance.logs
| app | datetime | level | logger | line | error | message | stacktrace |
|---|---|---|---|---|---|---|---|
| my_app | 2022-03-10 08:53:03.558 | 4 | example.py | 7 | Hello world! | ||
| my_app | 2022-03-10 08:53:03.558 | 3 | example.py | 8 | Fire drill test. | ||
| my_app | 2022-03-10 08:53:03.558 | 2 | example.py | 9 | This is real fire. | ||
| my_app | 2022-03-10 08:53:03.558 | 2 | example.py | 13 | ValueError | Oh no! | (traceback) |
| my_app | 2022-03-10 08:53:03.558 | 1 | example.py | 14 | I give up. |
6. Full Initialization
Standard
import contek_tusk as tusk
import contek_tusk.heartbeat as heartbeat
import contek_tusk.logging as logging
tusk.init(host='localhost', user='tester', password='P@assw0rd')
tusk.set_app_name('my_app')
heartbeat.init(table='maintenance.heartbeats')
logging.init(table='maintenance.logs')
YAML
app: 'my_app'
host: 'localhost'
user: 'tester'
password: 'P@assw0rd'
heartbeat:
database: 'maintenance'
table: 'heartbeats'
logging:
database: 'maintenance'
table: 'logs'
import contek_tusk.yaml as tusk_yaml
tusk_yaml.for_yaml_file('example.yml')
7. Example Table Schema
maintenance.heartbeats
CREATE TABLE IF NOT EXISTS maintenance.heartbeats
(
`app` String,
`datetime` DateTime64(3),
`task` String,
`sequence` UInt32,
`expiry` DateTime64
)
ENGINE = MergeTree()
ORDER BY
(
`app`,
`datetime`,
`task`,
`sequence`
)
TTL toDateTime(datetime) + INTERVAL 28 DAY DELETE;
maintenance.logs
CREATE TABLE IF NOT EXISTS maintenance.logs
(
`app` String,
`datetime` DateTime64(3),
`level` Int8,
`logger` String,
`line` UInt32,
`error` String,
`message` String,
`stacktrace` String
)
ENGINE = MergeTree()
ORDER BY
(
`category`,
`host`,
`app`,
`datetime`,
`level`,
`logger`,
`line`,
`error`
)
TTL toDateTime(datetime) + INTERVAL 7 DAY DELETE;
8. Customization
To write to tables with different schemas. Specify the column names in TuskHeartbeatConfig and
TuskLoggingConfig.
Metadata
Release files for contek-tusk 2.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| contek-tusk-2.2.tar.gz | 15.5 kB | Without provenance | Details for contek-tusk-2.2.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| contek_tusk-2.2-py3-none-any.whl | Python 3 | none | any | Details for contek_tusk-2.2-py3-none-any.whl |
Total release size: 38.5 kB
Release files / contek-tusk-2.2.tar.gz
| Download URL | contek-tusk-2.2.tar.gz |
|---|---|
| Size | 15.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
5e1ecc113c1e6458c14e52a7498ec9e1ca843b0f1b86dbcff8a82568b3be28e8
|
|
BLAKE2b-256 checksum How to use checksums |
3b109e9fe4db90b82e9e7f92ce43bed5a631607089f98417e2b7470a99b9482a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.10.0
|
Release files / contek_tusk-2.2-py3-none-any.whl
| Download URL | contek_tusk-2.2-py3-none-any.whl |
|---|---|
| Size | 23.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
46939d1b469893670965966aaecbd1165745d972c2ca724877b0dac2a78842ae
|
|
BLAKE2b-256 checksum How to use checksums |
56c08734530d1ab0eef5cbd25fe9964da9f75a52892f4ef8834d87d00a850172
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.8.0 pkginfo/1.8.2 readme-renderer/34.0 requests/2.27.1 requests-toolbelt/0.9.1 urllib3/1.26.9 tqdm/4.63.1 importlib-metadata/4.11.3 keyring/23.5.0 rfc3986/2.0.0 colorama/0.4.4 CPython/3.10.0
|
Security for contek-tusk 2.2
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.