Skip to main content

Simple server to gather data from fluentbit agent (forward protocol)

Project description

Fluentbit-server-py

Fluentd and Fluentbit are two great tools, like swiss knife. You can gather data, work on them and ship them to another FluentX instance. But what happen when you want to ship them to a Python program ? You need to do a workaround based on files, or stuff like that.

Well, that why it exists : if you need to gather data from fluentbit and process them in a python program (AI, exotic database, ...), you need that lib.

It implements a partial version for the forward protocol (https://github.com/fluent/fluentd/wiki/Forward-Protocol-Specification-v1):

  • SSL support
  • authentication based on shared_key
  • transport forward mode only

This library is designed as server for Fluentbit and was not tested on Fluentd (maybe it works ?).

Example

import sys
import logging

from fluentbit_server.fluentbit_authentication import FluentbitAuthentication
from fluentbit_server.fluentbit_transport import FluentbitTransport
from fluentbit_server.fluentbit_request_handler import FluentbitRequestHandler
from fluentbit_server.fluentbit_server import FluentbitServer
from fluentbit_server.fluentbit_ssl import FluentbitSSL
from functools import partial

authentication_factory = partial(FluentbitAuthentication, shared_key="my_shared_key", server_hostname="server.com")
transport_factory = partial(FluentbitTransport, callback=print)
ssl = FluentbitSSL(key_file="etc/toto.com.key", crt_file="etc/toto.com.cert")


def main():
    logging.basicConfig(level=logging.DEBUG)

    server = FluentbitServer(("localhost", 24000), FluentbitRequestHandler, transport_factory, authentication_factory, ssl)

    try:
        server.serve_forever()
    except KeyboardInterrupt:
        pass
    except OSError:
        print("Socket in use")
    finally:
        server.server_close()
        print("closed")
        sys.exit(0)

if __name__ == "__main__":
    main()

Authentication and ssl are optionals so such line will product a basic server (no authentication, no ciphering):

server = FluentbitServer(("localhost", 24000), FluentbitRequestHandler, transport_factory)

Licence

This software is under MIT.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fluentbit-server-py-1.0.7.tar.gz (6.2 kB view details)

Uploaded Source

File details

Details for the file fluentbit-server-py-1.0.7.tar.gz.

File metadata

  • Download URL: fluentbit-server-py-1.0.7.tar.gz
  • Upload date:
  • Size: 6.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.8.5

File hashes

Hashes for fluentbit-server-py-1.0.7.tar.gz
Algorithm Hash digest
SHA256 a9923023bc859dcbf018743f4a2a3d939155697b316c44c2dacf80d38ad2d2c1
MD5 4f9ffba8f6c39ee8f1e531dd1b83cff6
BLAKE2b-256 c141a1e1f12d8948cdc452bd21a7ebcc213a2abbc2bdeca152984320fa86a34e

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page