Command line interface for interacting with the Humio API using the humioapi library
Project description
Do things with the Humio API from the command line
This project requires
Python>=3.6.1
This is a companion CLI to the unofficial humioapi library. It lets you use most of its features easily from the command line. If you're looking for the official lib+cli it can be found here: humiolib.
Installation
pip install humiocli
Main features
- Streaming searches with several output formats
- Subsearches (pipe output from one search into a new search)
- Defaults configured through ENV variables
- Splunk-like chainable relative time modifiers
- Switch easily from browser to CLI by passing the search URL to urlsearch
- Ingest data to Humio (but you should use Filebeat for serious things)
- List repositories
First time setup
Start the guided setup wizard to configure your environment
hc wizard
This will help you create an environment file with a default Humio URL and token, so you don't have to explicitly provide them as options later.
All options may be provided by environment variables on the format
HUMIO_<OPTION>=<VALUE>
. If a .env file exists in ~/.config/humio/.env
it
will be automatically sourced on execution without overwriting the
existing environment.
Examples
Execute a search in all repos starting with reponame
and output @rawstring
s
hc search --repo 'reponame*' '#type=accesslog statuscode>=400'
Execute a search using results with fields from another search ("subsearch")
Step 1: Set the output format to or-fields
hc search --repo=auth 'username | select([session_id, app_name])' --outformat=or-fields | jq '.'
This gives a JSON-structure with prepared search strings from all field-value combinations. The special field SUBSEARCH
combines all search strings for all fields.
Example output:
{
"session_id": "\"session_id\"=\"5CF4A111\" or \"session_id\"=\"14C8BCEA\"",
"app_name": "\"app_name\"=\"frontstop\"",
"SUBSEARCH": "(\"session_id\"=\"5CF4A111\" or \"session_id\"=\"14C8BCEA\") and (\"app_name\"=\"frontstop\")"
}
Step 2: Pipe this result to a new search and reference the desired fields:
hc search --repo=auth 'username | select([session_id, app_name])' --outformat=or-fields | hc --repo=frontstop '#type=accesslog {{session_id}}'
Output aggregated results as ND-JSON events
Simple example:
Humios bucketing currently creates partial buckets in both stops depstoping on search period. You may want to provide a rounded start and stop to ensure we only get whole buckets.
hc search --repo 'sandbox*' --start=-60m@m --stop=@m "#type=accesslog | timechart(span=1m, series=statuscode)"
Or with a longer multiline search
hc search --repo 'sandbox*' --start -60m@m --stop=@m "$(cat << EOF
#type=accesslog
| case {
statuscode<=400 | status_ok := 1 ;
statuscode=4* | status_client_error := "client_error" ;
statuscode=5* | status_server_error := "server_error" ;
* | status_broken := 1
}
| bucket(limit=50, function=[count(as="count"), count(field=status_ok, as="ok"), count(field=status_client_error, as="client_error"), count(field=status_server_error, as="server_error")])
| error_percentage := (((client_error + server_error) / count) * 100)
EOF
)"
Upload a parser file to the destination repository, overwriting any existing parser
hc makeparser --repo='sandbox*' customjson
Ingest a single-line log file with an ingest-token associated with a parser
hc ingest customjson
Ingest a multi-line file with a user provided record separator (markdown headers) and parser
hc ingest README.md --separator '^#' --fields '{"#repo":"sandbox", "#type":"markdown", "@host":"localhost"}'
Development
To install the cli and api packages in editable mode:
git clone https://github.com/gwtwod/humiocli.git
poetry install
Create self-contained executables for easy distribution
With Shiv:
git clone https://github.com/gwtwod/humiocli.git
shiv -c hc -o hc humiocli/ -p "/usr/bin/env python3"
With Pex:
git clone https://github.com/gwtwod/humiocli.git
git clone https://github.com/gwtwod/humioapi.git
pex --disable-cache -c hc -o hc humiocli humioapi --python-shebang="/usr/bin/env python3"
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.