Skip to main content

A python client library for Fluentd

Project description


pyfluent is a python client library for Fluentd. It is simple, fast and reliable.

The primary purpose is to provide pythonic way to transmit JSON message to fluentd.

For better performance, pyfluent connects to fluentd’s in_forward plugin and transmit messages that are serialized by MessagePack. While a connection is failed, messages are automatically queued for latter retransmit. When a connection is re-established, all queued messages are retransmitted automatically. So, you don’t worry about losing messages.

pyfluent is available under Apache License, Version 2.0.

Note: due to long time inactivity, I decided to reset the develop branch to 0.2.0 release. All commits in the old develop branch now exist in the server branch.


pyfluent requires python 2.5 or above (include python 3).

You can install using pip:

$ pip install pyfluent


Sending message is super simple. Just like this:

from pyfluent.client import FluentSender
fluent = FluentSender()
fluent.send('Hello pyfluent!')

By default, FluentSender connects localhost:24224 and use empty string for tag. You can change this by passing arguments.

fluent = FluentSender('', 10000, 'pyfluent')
fluent.send('Hello pyfluent!')

Above examples, we passed string as argument of FluentSender.send. For convenience, FluentSender.send make dict automatically before sending.

On the other hand, you can pass dict as argument.

fluent.send({'hello': 'fluent'})

Additionally, you can change tag and timestamp of each message if you want.

import time
fluent.send({'hello': 'fluent'}, '', time.time() - 60)


Since pyfluent logging library implemented like python standard logging library, it is very easy to introduce pyfluent into existing programs.

Here is a basic usage:

import logging
from pyfluent.logging import SafeFluentHandler
handler = SafeFluentHandler('localhost', 24224, 'pyfluent')
logger = logging.getLogger()
logger.addHandler(handler)'hello pyfluent!')

You can obtain a json message like below through fluentd.

2012-03-02 11:50:46 +0900 {"message":"hello pyfluent!"}

As you can see, FluentHandler automatically append log level to the tag.

For convenience, pyfluent provide FluentFormatter to emit more information.

from pyfluent.logging import FluentFormatter
formatter = FluentFormatter('%(asctime)s %(levelname)s %(message)s')
handler.setFormatter(formatter)'get more information')

You can obtain a json message like below.

2012-03-02 11:52:18 +0900 {"threadName":"MainThread","name":"r
t":null,"message":"2012-03-02 12:35:18,321 INFO get more information","funcNa

The default behavior is emit all items in __dict__ of LogRecord except args, asctime, created, exc_info, levelno, msecs, msg, relativeCreated, thread and message. It is customizable if you want, for example, except threadName, module, filename, processName, pathname, lineno and funcName in addition to the default.

formatter.exclude += ['threadName', 'module', 'filename', 'process',
'processName', 'pathname', 'lineno', 'funcName']'suppress unnecessary information')

As a result, you can obtain like below.

2012-03-02 11:54:21 +0900 {"exc_text":null,"message":"2012-03-
02 14:23:21,504 INFO suppress unnecessary information","hostname":"host.examp","name":"root","levelname":"info"}

Please note that:

  • hostname is added automatically by FluentFormatter, so you cannot remove hostname from output information.
  • created is converted to the fluentd’s time.


0.2.1 (2019-01-10)
  • Improve reliability
0.2.0 (2012-04-21)
  • Add FluentSender.
  • Rewrite SafeFluentHandler to use FluentSender internally.
  • Change license to Apache License, Version 2.0 from MIT License.
0.1.2 (2012-03-02)
  • Support python 2.5, 2.6, 3.0, 3.1, 3.2.
0.1.1 (2012-02-26)
  • Fix issue on install from PyPI.
0.1.0 (2012-02-26)
  • First release.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyfluent-0.2.1.tar.gz (5.4 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page