The gnarliest gear in the world 🤙
Project description
Gnar Gear: Gnarly Python Apps
Part of Project Gnar: base • gear • piste • off-piste • edge • powder • genesis • patrol
Get started with Project Gnar on
Gnar Gear sets up a powerful Flask-based Python service with two lines of code:
from gnar_gear import GnarApp
...
GnarApp('my_gnarly_app', production=True, port=80).run()
Installation
pip3 install gnar-gear
Feature List
- Flask app with auto blueprint registration
- Flask WSGI Development Server
- Bjoern WSGI Production Server
- Why Bjoern? Check out these benchmarks!
- And also these benchmarks
- Postgres database connection via Postgres.py
- SES client connection via Boto 3
- JWT configuration via Flask-JWT-Extended
- Peer requests - HTTP requests to other microservices in the app
- External requests - Convenience wrapper around requests
- SQS Message Polling and Sending
- argon2_cffi.PasswordHasher instance
Argon2
was the winner of the 2015 Password Hashing Competition
- Logger configuration
- Error handler with traceback
- Overridable and extendable class-based design
Development Mode
- Flask WSGI server in fault-tolerant debug (watch & reload) mode
- The module loader logs the stack trace of any modules that fail to load
- If a module fails to load, the app continues to watch for file changes
- CORS "disabled" (
Access-Control
response headers set), so you don't need to use other means to circumvent CORS
Production Mode
- Bjoern WSGI server
- CORS "enabled" (
Access-Control
response headers are not set)
Requirements
Bjoern
Bjoern
requires libev
(high performance event loop)
- Install
libev
withbrew install libev
on Mac, or find your platform-specific installation command here
Application Structure
GnarApp
expects to be instantiated in main.py
at <top-level-module>/app
, i.e. the minimum app folder structure is
+ <top-level-module>
+ app
main.py
__init__.py
It is recommended (not required) to place your apis in segregated folders under app
and the tests in a test
folder under the <top-level-module>
, e.g.
+ <top-level-module>
+ app
+ admin
apis.py
constants.py
services.py
+ user
apis.py
constants.py
services.py
__init__.py
main.py
+ test
constants.py
test_app.py
__init__.py
Blueprints
Each Flask Blueprint
must be assigned to a global-level blueprint
variable in its module, e.g.
from flask import Blueprint, jsonify
api_name = 'user'
url_prefix = '/{}'.format(api_name)
blueprint = Blueprint(api_name, __name__, url_prefix=url_prefix)
^^^^^^^^^
@blueprint.route('/get', methods=['GET'])
def user_get():
return jsonify({'status': 'ok'})
By default, the GnarApp
picks up every blueprint
in an auto-scan of the application code.
Overview
The GnarApp
class provides a highly configurable, feature-rich, production-ready Flask-based app.
Parameters
Args (required)
- name: The name of the application's top-level module
- production: Boolean flag indicating whether or not the build is in production mode
- port: The port to bind to the WSGI server
Kwargs (optional)
- env_prefix: Environment variable prefix (defaults to
GNAR
) - log_level: Log level override
- see configure_logger for log level overview
- blueprint_modules: List of modules to find Flask blueprints (default is auto-scan)
- see configure_blueprints for blueprints overview
- sqs: List (or single dict) of SQS queues to poll
- see configure_sqs_polls for SQS polling overview
- no_db: Boolean flag - specify
True
if the app does not need a Postgres connection - no_jwt: Boolean flag - specify
True
if the app does not use JWT headers (i.e. non-api services)
Overridable Behavior
GnarApp.run
simply calls a set of steps in the class. Here is an example of how to override any of the steps:
def postconfig():
log.info('My Postconfig Step!')
ga = GnarApp('my_gnarly_app', production=True, port=80)
ga.postconfig = postconfig
ga.run()
Run Steps
The run steps rely on a set of environment variables which use a configurable prefix
(i.e. the env_prefix
parameter). The default env_prefix
is GNAR
. An example of a Gnar environment variable
using a custom prefix is GnarApp( ..., env_prefix='MY_APP')
and then instead of reading GNAR_LOG_LEVEL
, the
configure_logger
step will read MY_APP_LOG_LEVEL
.
preconfig
- No default behavior - provided as an optional initial step in the app configuration.
configure_flask
- Attaches a
Flask
instance to the Gnar app.
configure_logger
- Attaches the root logger to
sys.stdout
. - Sets the logging level to the first defined:
log_level
parameterGNAR_<app name>_LOG_LEVEL
environment variable, e.g.GNAR_MY_GNARLY_APP_LOG_LEVEL
GNAR_LOG_LEVEL
environment variableINFO
- Reminder: Valid settings (in increasing order of severity) are
DEBUG
,INFO
,WARNING
,ERROR
,CRITICAL
- Sets the log format to the first defined:
GNAR_LOG_FORMAT
'%(asctime)s %(levelname)-8s %(name)s:%(lineno)d %(message)s'
, e.g.:
2018-07-09 15:41:46.420 INFO gear.gnar_app:75 Logging at INFO
- Sets the log format
default_msec_format
to the first defined:GNAR_LOG_FORMAT_MSEC
'%s.%03d'
(e.g..001
)
configure_argon2
-
Attaches an
argon2_cffi.PasswordHasher
instance to the Gnar app. -
Reads the following environment variables
[TYPE: DEFAULT]
to pass into the Argon2 instance:GNAR_ARGON2_TIME_COST
:[INT: 2]
Number of iterations to performGNAR_ARGON2_MEMORY_COST
:[INT: 512]
Amount of memory (in KB) to useGNAR_ARGON2_PARALLELISM
:[INT: 2]
Number of parallel threads (changes the resulting hash value)GNAR_ARGON2_HASH_LEN
:[INT: 16]
Length of the hash in bytesGNAR_ARGON2_SALT_LEN
:[INT: 16]
Length of random salt to be generated for each password in bytesGNAR_ARGON2_ENCODING
:[STR: 'utf-8']
Encoding to use when a string is passed intohash
orverify
-
Note from the docs:
Only tweak these if you’ve determined using CLI that these defaults are too slow or too fast for your use case.
-
To hash a password using
Argon2
:from <top-level-module>.main import app hash = app.generate_password_hash(<plain text password>) # OR hash = app.argon2.hash(<plain text password>)
Note that this creates a randomly salted, memory-hard hash using the Argon2i algorithm.
-
To validate a password with
Argon2
:from <top-level-module>.main import app is_valid = app.check_password_hash(<password hash from database>, <plain text password>) # OR is_valid = app.argon2.verify(<password hash from database>, <plain text password>)
Note that
app.argon2.verify
raises an exception if the password is invalid whereasapp.check_password_hash
does not.
configure_database
- Creates a Postgres database connection and attaches it to the Gnar app
- Reads the following environment variables to set the
host
,dbname
,user
,password
connection string parameters, respectively:GNAR_PG_ENDPOINT
GNAR_PG_DATABASE
GNAR_PG_USERNAME
GNAR_PG_PASSWORD
- Note: The Postgres API primarily consists of
run
,one
, andall
attach_instance
-
Attaches the
GnarApp
instance to theapp.main
module. This enables easy access to the Gnar app from anywhere in the application usingfrom <top-level-module>.main import app
-
The
GnarApp
's runtime assets aredb
,argon2
,flask
,check_password_hash
,generate_password_hash
, andget_ses_client
-
For example, to fetch one result (or
None
) from the database:app.db.one("SELECT * FROM foo WHERE bar='buz'")
configure_blueprints
- By default,
GnarApp
auto-scans every Python module under theapp
folder for blueprints. - Each Flask
Blueprint
must be assigned to a global-levelblueprint
variable in its module. - If you prefer to skip the auto-scan, you can provide a list (or single string) of blueprint modules.
- Each item in the list of module names may use one of two formats:
- Without a
.
in the module name:GnarApp
will look for the module in<top-level-module>.app.<module name>.apis
- With a
.
in the module:GnarApp
will look for the module in<top-level-module>.app.<module name>
- Without a
- Each item in the list of module names may use one of two formats:
configure_errorhandler
- Defines a generic (Exception-level) Flask error handler which:
- Logs the error message and its
traceback
(format_exec) - Returns a 200-level json response containing
{"error": <error message>, "traceback": <traceback>}
- Logs the error message and its
configure_jwt
- Sets the Flask
JWT_SECRET_KEY
variable to the value of theGNAR_JWT_SECRET_KEY
environment variable. - Sets the Flask
JWT_ACCESS_TOKEN_EXPIRES
variable to the value of theGNAR_JWT_ACCESS_TOKEN_EXPIRES_MINUTES
environment variable (default 15 mins). - Attaches a JWTManager instance to the
GnarApp
. - Defines functions for
expired_token_loader
,invalid_token_loader
, andunauthorized_loader
which return meaningful error messages as 200-level json responses containing{"error": <error message>}
.
configure_sqs_polls
-
Configures a set of polls to run at specified intervals to receive messages from SQS queues.
-
Specified via the
sqs
kwarg as a list of dicts (or a single dict) with a set of required (*) and optional properties. See the AWS ReceiveMessage docs for more details on the optional properties:- * queue_name: The name of the SQS queue - the queue URL is retrieved from AWS with this name
- * callback: The function to call with each received message. The message will be deleted from the queue unless the callback function returns
False
. - interval_seconds: The interval (in seconds) between receive message requests - default 60
- attribute_names: A list of attributes to return with each message
- max_number_of_messages: The maximum number of messages to return with each request - 1 to 10, default 1
- message_attribute_names: Metadata to include with the message - more details
- receive_request_attempt_id: The token used for deduplication of ReceiveMessage calls (FIFO queues only)
- visibility_timeout: The duration (in seconds) that the received messages are hidden from subsequent retrieve requests
- wait_time_seconds: The duration (in seconds) for which the call waits for a message to arrive in the queue before returning
-
Each message received by the callback includes the following properties. See the AWS Message docs for more details on these properties:
- Body: Message contents
- Attributes: Map of requested attributes
- MD5OfBody: MD5 Digest of the message body
- MD5OfMessageAttributes: MD5 digest of the message attribute string
- MessageAttributes: Map of custom message metadata
-
Notes:
- Messages are processed sequentially in the order they are received
- The
callback
blocks message processing until it's complete - After all received messages are processed, the queue is immediately checked for new messages
- Once the
receive_message
call returns no messages, aTimer
with the specifiedinterval_seconds
starts the next loop
-
Reads the following environment variables to set the
region_name
,aws_access_key_id
, andaws_secret_access_key
parameters of theboto3.client
call, respectively:GNAR_SQS_REGION_NAME
GNAR_SQS_ACCESS_KEY_ID
GNAR_SQS_SECRET_ACCESS_KEY
-
Example:
def receive_sqs_message(message): redis_connection.set(message['MessageId'], message['Body'], 3600) sqs = {'queue_name': 'gnar-queue', 'callback': receive_sqs_message} GnarApp('piste', production, port, sqs=sqs).run()
configure_after_request
- Adds a JWT Authorization header (Bearer token) to responses which received a valid JWT token in the request.
- In development mode, adds CORS headers to the response (so that you don't need to bother circumventing CORS).
postconfig
- No default behavior - provided as an optional initial step in the app configuration.
Runtime Functionality
peer
-
Handles requests to other microservices in a Kubernetes app.
-
Usage:
from <top-level-module>.main import app peer = app.peer(<< service name >>)
-
Requires environment variables in the form:
- production:
<< SERVICE >>_SERVICE_PORT
, e.g.PISTE_SERVICE_PORT
- development:
GNAR_<< SERVICE >>_SERVICE_PORT
, e.g.GNAR_PISTE_SERVICE_PORT
- production:
-
In production, the environment variable is automatically set by Kubernetes
-
In development, the environment variable should point to another Gnar Gear app on localhost on a separate port
-
The environment variable value must be in the form
<< ip address >>:<< port >>
, e.g.export GNAR_PISTE_SERVICE_PORT='127.0.0.1:9401'
peer Methods
-
Each peer method accepts all the
kwargs
of the underlyingrequests
method in addition to anauto_auth
property.- If
auto_auth
isTrue
and the current route is protected, an Authorization header is added to the request. - Both microservices must use the same
GNAR_JWT_SECRET_KEY
for the authorization to work.
- If
-
Uses requests under the hood:
- request(method, path, **kwargs): Constructs and sends a request
- head(path, **kwargs): Sends a HEAD request
- get(path, params=None, **kwargs): Sends a GET request
- post(path, data=None, json=None, **kwargs): Sends a POST request
- put(path, data=None, **kwargs): Sends a PUT request
- patch(path, data=None, **kwargs): Sends a PATCH request
- delete(path, **kwargs): Sends a DELETE request
-
Example:
from <top-level-module>.main import app response = app.peer('piste').post('login', {'email': 'ic3b3rg@gmail.com'})
external
-
Convenience wrapper around requests
-
Example:
from <top-level-module>.main import app response = app.external.get('https://api.pwnedpasswords.com/range/5ce7d')
generate_password_hash / check_password_hash
-
Convenience wrappers for
app.argon2.hash
andapp.argon2.verify
-
Usage:
from <top-level-module>.main import app hash = app.generate_password_hash(<plain text password>) is_valid = app.check_password_hash(<password hash from database>, <plain text password>)
get_ses_client
-
Exposed as runtime functionality (as opposed to creating the client at initialization) because AWS will close the client after a short period of time
-
Returns an SES connection (using boto3)
-
Reads the following environment variables to set the
region_name
,aws_access_key_id
, andaws_secret_access_key
parameters of theboto3.client
call, respectively:GNAR_SES_REGION_NAME
GNAR_SES_ACCESS_KEY_ID
GNAR_SES_SECRET_ACCESS_KEY
-
Usage:
from <top-level-module>.main import app app.get_ses_client().send_email( ... )
-
See the
Boto 3 Docs
for thesend_email
request syntax.
send_sqs_message
-
Sends a message to an SQS queue.
-
Args (required):
- queue_name: The name of the SQS queue - the queue URL is retrieved from AWS with this name
- message_body: Message contents
-
Kwargs (optional) - See the AWS SendMessage docs for more details on these properties:
- delay_seconds: The length of time, in seconds, for which to delay a specific message
- message_attributes: Metadata to include with the message
- message_deduplication_id: The token used for deduplication of sent messages (FIFO queues only)
- message_group_id: The tag that specifies that a message belongs to a specific message group (FIFO queues only)
-
Returns a dict with the following attributes:
- MessageId: Unique message identifier
- MD5OfMessageAttributes: MD5 digest of the message attribute string
- MD5OfMessageBody: MD5 digest of the message body
- SequenceNumber: Unique, non-consecutive, 128 bit string (FIFO queues only)
-
Reads the following environment variables to set the
region_name
,aws_access_key_id
, andaws_secret_access_key
parameters of theboto3.client
call, respectively:GNAR_SQS_REGION_NAME
GNAR_SQS_ACCESS_KEY_ID
GNAR_SQS_SECRET_ACCESS_KEY
-
Example:
from <top-level-module>.main import app message = app.send_sqs_message('gnar-queue', 'The gnarliest gear in the world 🤙')
Environment Variables
- The environment variables (with configurable prefix) used by
GnarApp
are:GNAR_ARGON2_ENCODING
GNAR_ARGON2_HASH_LEN
GNAR_ARGON2_MEMORY_COST
GNAR_ARGON2_PARALLELISM
GNAR_ARGON2_SALT_LEN
GNAR_ARGON2_TIME_COST
GNAR_JWT_SECRET_KEY
GNAR_LOG_LEVEL
GNAR_PG_DATABASE
GNAR_PG_ENDPOINT
GNAR_PG_PASSWORD
GNAR_PG_USERNAME
GNAR_SES_ACCESS_KEY_ID
GNAR_SES_REGION_NAME
GNAR_SES_SECRET_ACCESS_KEY
GNAR_SQS_REGION_NAME
GNAR_SQS_ACCESS_KEY_ID
GNAR_SQS_SECRET_ACCESS_KEY
- See the relevant sections above for details
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for gnar_gear-1.1.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19ba3ffd753c84372f8aad18428c49dba0c6507a86783af1b2978ea42565c2a6 |
|
MD5 | 206d918fa31fc10872901d2d6aa28142 |
|
BLAKE2b-256 | 946016f54a1a7d314e05f9fdbce7426622a8601c6b67a5e8a34027ff09dcea64 |