Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Logging handlers for publishing the logs to Cloud Pub/Sub

Project Description
Logging handlers for publishing the logs to Cloud Pub/Sub.

You can use the pubsub_logging.PubsubHandler or pubsub_logging.AsyncPubsubHandler to publish the logs to Cloud Pub/Sub. You can use this module with the standard Python logging module. It’s recommended that you use AsyncPubsubHandler. PubsubHandler exists only for backward compatibility.

Supported version

Python 2.7 and Python 3.4 are supported.

Installation

$ pip install pubsub-logging

How to use

Here is an example configuration file.

[loggers]
keys=root

[handlers]
keys=asyncPubsubHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=NOTSET
handlers=asyncPubsubHandler

[handler_asyncPubsubHandler]
class=pubsub_logging.AsyncPubsubHandler
level=DEBUG
formatter=simpleFormatter
# Replace {project-name} and {topic-name} with actual ones.
# The second argument indicates number of workers.
args=('projects/{project-name}/topics/{topic-name}', 10)

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s

How to use this config file.

logging.config.fileConfig(os.path.join('examples', 'logging.conf'))
logger = logging.getLogger('root')
logger.info('My first message.')

Here is a dynamic usage example.

pubsub_handler = AsyncPubsubHandler(topic=topic)
pubsub_handler.setFormatter(
    logging.Formatter(
        '%(asctime)s - %(name)s - %(levelname)s - %(message)s'))

logger = logging.getLogger('root')
logger.setLevel(logging.DEBUG)
logger.addHandler(pubsub_handler)
logger.info('My first message.')

The logs are kept in a buffer first, then moved to the process safe queue, and then the background child processes automatically pick up and send them to Cloud Pub/Sub. The flush call blocks until all of the logs are sent to Cloud Pub/Sub.

Authentication

The module uses the Application Default Credentials. You can configure the authentication as follows.

Authentication on App Engine

It should work out of the box. If you’re getting an authorization error, please make sure that your App Engine service account has an Editor or greater permission on your Cloud project.

Authentication on Google Compute Engine

When creating a new instance, please add the Cloud Pub/Sub scope https://www.googleapis.com/auth/pubsub to the service account of the instance.

Authentication anywhere else

As the documentation suggests, create a new service account and download its JSON key file, then set the environment variable GOOGLE_APPLICATION_CREDENTIALS pointing to the JSON key file. Please note that this service account must have Editor or greater permissions on your Cloud project.

Release History

Release History

This version
History Node

0.2.1

History Node

0.2.0

History Node

0.1.6

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pubsub-logging-0.2.1.tar.gz (7.0 kB) Copy SHA256 Checksum SHA256 Source Apr 23, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting