micropython-ccrighton-phew 0.0.5

A small webserver and templating library specifically designed for MicroPython on the Pico W.

phew! the Pico (or Python) HTTP Endpoint Wrangler

A small webserver and templating library specifically designed for MicroPython on the Pico W. It aims to provide a complete toolkit for easily creating high quality web based interfaces for your projects.

phew! is ideal for creating web based provisioning interfaces for connected projects using the Raspberry Pi Pico W.

• phew! the Pico (or Python) HTTP Endpoint Wrangler
  • What phew! does:
  • How to use
  • Basic example
  • Running multiple web applications
  • Transport Layer Security (TLS)
    • Generating a Key and Certificate
  • Sessions
    • Session Authentication
  • Function reference
    • server module
      • add_route
      • set_catchall
      • run
    • Session interface
      • create_session
      • remove_session
      • login_required
      • login_catchall
    • Types
      • Request
      • Response
        • Shorthand
      • Session
    • Templates
      • render_template
      • Template expressions
        • Variables
        • Conditional display
        • Includes
    • logging module
      • log(level, text)
      • debug(*items)
      • info(*items)
      • warn(*items)
      • error(*items)
      • set_truncate_thresholds(truncate_at, truncate_to)
    • dns module
      • run_catchall
    • Helper functions
      • connect_to_wifi
      • access_point
      • is_connected_to_wifi
      • get_ip_address
  • Other Resources

What phew! does:

• a basic web server
• optimised for speed (at import and during execution)
• minimal use of memory
• parameterised routing rules /greet/<name>
• templating engine that allows inline python expressions {{name.lower()}}
• GET, POST request methods
• query string decoding and parsing
• catchall handler for unrouted requests
• multipart/form-data, x-www-form-urlencoded, and JSON POST bodies
• string, byte, or generator based responses
• connect_to_wifi and access_point convenience methods

Where possible phew! tries to minimise the amount of code and setup that you, the developer, has to do in favour of picking sane defaults and hiding away bits of minutiae that rarely needs to be tweaked.

How to use

phew! can be installed using pip from the command line or from your favourite IDE. In Thonny this can be achieved by clicking Tools -> Manage packages and searching for micropython-phew.

Basic example

An example web server that returns a random number between 1 and 100 (or optionally the range specified by the callee) when requested:

from phew import server, connect_to_wifi

connect_to_wifi("<ssid>", "<password>")

phew_app = server.Phew()

@phew_app.route("/random", methods=["GET"])
def random_number(request):
  import random
  min = int(request.query.get("min", 0))
  max = int(request.query.get("max", 100))
  return str(random.randint(min, max))

@phew_app.catchall()
def catchall(request):
  return "Not found", 404

phew_app.run()

phew is designed specifically with performance and minimal resource use in mind. Generally this means it will prioritise doing as little work as possible including assuming the correctness of incoming requests.

---

Running multiple web applications

A device may require multiple web apps. For instance, a setup web app for the access point and a configuration web app for normal operation. Phew supports the creation of many apps with registration of routes per app. To create a new app, just create another server.Phew instance.

For concurrent execute of apps, each must be configured to connect to a different port and be run in the same uasyncio loop as tasks.

import uasyncio
from phew import server

phew_app1 = server.Phew()
phew_app2 = server.Phew()

# route methods declared here for both apps

loop = uasyncio.get_event_loop()
phew_app1.run_as_task(loop, host="0.0.0.0", port=80)
phew_app2.run_as_task(loop, host="0.0.0.0", port=8080)
loop.run_forever()

Transport Layer Security (TLS)

Phew supports Transport Layer Security (TLS) by exposing the capabilities of asynicio.

TLS is enabled by providing an ssl.SSLContext to either Phew.run or Phew.run_as_task.

TLS setup introduces about a 5 second delay on a request. This time depends on the key algorithm used.

import ssl

# set up TLS
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.load_cert_chain("cert.der", "key.der")

# start the webserver
phew_app.run(host='0.0.0.0', port=443, ssl=ctx)

See auth example app for a working implementation.

Configuration options for the ssl.SSLContext are documented at Micropython SSL/TLS module.

Generating a key and cert

Generate the key.

openssl ecparam -name prime256v1 -genkey -noout -out key.der -outform DER

Generate the self-signed certificate.

openssl req -new -x509 -key key.der -out cert.der -outform DER -days 365 -node

List the certificate details:

openssl x509 -in cert.der  -text

Copy the key and cert files to the device.

mpremote cp cert.der key.der :

Sessions

Phew provides a simple session api to support authenticated session establishment. Phew.create_session() is called to set up a new session. The session returned contains the session id and max age values to be used in the cookie exchange.

The following code creates the response to set the session cookie. This needs to be run only if the client has provided valid credentials. For instance the client may do a POST request of a username and password as form data.

session = phew_app.create_session()
return server.Response("OK", status=302,
                    headers={"Content-Type": "text/html",
                             "Set-Cookie": f"sessionid={session.session_id}; Max-Age={session.max_age}; Secure; HttpOnly",
                             "Location": "/"})

To ensure that the session id is sent only on TLS sessions, please ensure that the Secure parameter is set in the Set-Cookie header.

Once the session is established, all route handlers that have the login_required() annotation will check that the request contains a cookie with the valid session id set.

@phew_app.route("/", methods=["GET"])
@phew_app.login_required()
def index(request):
    return render_template("index.html", text="Hello World")

Add the login_required() annotation to all route handlers that need authentication. However, do not add it to the login route handler as this will prevent the establishment of a session.

Phew.remove_session(request) is called to end the session. For example, a logout route handler will call it to log the session out.

@phew_app.route("/logout", methods=["GET"])
def logout(request):
    phew_app.remove_session(request)
    return render_template("logout.html")

Session Authentication

The method used for session authentication is within the control of the application using the Phew library.

In the example provided a login form is used that provides username and password in form data.

---

Function reference

server module

The server module provides all functionality for running a web server with route handlers.

add_route

server.add_route(path, handler, methods=["GET"])

Adds a new route into the routing table. When an incoming request is received the server checks each route to find the most specific one that matches the request based on the path and method. If a route is found then the handler function is called with a request parameter that contains details about the request.

def my_handler(request):
  return "I got it!", 200

phew_app.add_route("/testpath", my_handler, methods=["GET"])

Or, alternatively, using a decorator:

@phew_app.route("/testpath", methods=["GET"])
def my_handler(request):
  return "I got it!", 200

set_catchall

phew_app.set_catchall(handler)

Provide a catchall method for requests that didn't match a route.

def my_catchall(request):
  return "No matching route", 404

phew_app.set_catchall(my_catchall)

Or, alternatively, using a decorator:

@phew_app.catchall()
def my_catchall(request):
  return "No matching route", 404

run

phew_app.run(host="0.0.0.0", port=80)

Starts up the web server and begins handling incoming requests.

phew_app.run()

Session interface

create_session

Create a new session that provides the parameters needed for cookie based session establishment.

session = phew_app.create_session()

remove_session

Remove an existing session, resulting in that session no longer being active so the user is logged out.

phew_app.remove_session(request)

login_required

A decorator for a route handler that redirects and requests to decorated routes to the login handler.

@phew_app.route("/hello", methods=["GET"])
@phew_app.login_required()
def hello(request, name):
  return render_template("index.html")

login_catchall

Decorator that sets the handler for the login page. DO NOT decorate with login_required.

@phew_app.login_catchall()
def redirect_to_login(request):
    return server.redirect("/login", status=302)

Types

Request

The Request object contains all of the information that was parsed out of the incoming request including form data, query string parameters, HTTP method, path, and more.

Handler functions provided to add_route and set_catchall will recieve a Request object as their first parameter.

member	example	type	description
protocol	"HTTP/1.1"	string	protocol version
method	"GET" or "POST"	string	HTTP method used for this request
uri	"/path/to/page?parameter=foo"	string	full URI of the request
path	"/path/to/page"	string	just the path part of the URI
query_string	"parameter=foo"	string	just the query string part of the URI
form	{"foo": "bar", "name": "geoff"}	dict	POST body parsed as multipart/form-data
data	[{"name": "jenny"}, {"name": "geoff"}]	any	POST body parsed as JSON
query	{"parameter": "foo"}	dict	result of parsing the query string

At the time your route handler is being called the request has been fully parsed and you can access any properties that are relevant to the request (e.g. the form dictionary for a multipart/form-data request) any irrelevant properties will be set to None.

@phew_app.route("/login", ["POST"])
def login_form(request):
  username = request.form.get("username", None)
  password = request.form.get("password", None)

  # check the user credentials with your own code
  # for example:
  #
  # logged_in = authenticate_user(username, password)

  if not logged_in:
    return "Username or password not recognised", 401

  return "Logged in!", 200

Response

The Response object encapsulates all of the attributes of your programs response to an incoming request. This include the status code of the result (e.g. 200 OK!) , the data to return, and any associated headers.

Handler functions can create and return a Response object explicitly or use a couple of shorthand forms to avoid writing the boilerplate needed.

member	example	type	description
status	200	int	HTTP status code
headers	{"Content-Type": "text/html"}	dict	dictionary of headers to return
body	"this is the response body"	string or generator	the content to be returned

@phew_app.route("/greeting/<name>", ["GET"])
def user_details(request):
  return Response(f"Hello, {name}", status=200, {"Content-Type": "text/html"})

Shorthand

As shorthand instead of returning a Response object the handle may also return a tuple with between one and three values:

• body - either a string or generator method
• status code - defaults to 200 if not provided
• headers - defaults to {"Content-Type": "text/html"} if not provided

For example:

@phew_app.route("/greeting/<name>", ["GET"])
def user_details(request, name):
  return f"Hello, {name}", 200

Session

The Session object contains the attributes of a session. It is returned by the create_session function.

member	example	type	description
session_id	5146c4a8b8a3c83e54b5c06ce009988c	str	128 bit hex encoded session identifier
max_age	86400	int	seconds from session creation to expiry
expires	1609563847	int	seconds from epoch to session expiry

The Session object provides a convenience function for checking expiry:

Session.expired()

Return boolean.

Templates

A web server isn't much use without something to serve. While it's straightforward to serve the contents of a file or some generated JSON things get more complicated when we want to present a dynamically generated web page to the user.

phew! provides a templating engine which allows you to write normal HTML with fragments of Python code embedded to output variable values, parse input, or dynamically load assets.

render_template

render_template(template, param1="foo", param2="bar", ...):

The render_template method takes a path to a template file on the filesystem and a list of named paramaters which will be passed into the template when parsing.

The method is a generator which yields the parsing result in chunks, minimising the amount of memory used to hold the results as they can be streamed directly out rather than having to build the entire result as a string first.

Generally you will call render_template to create the body of a Response in one of your handler methods.

Template expressions

Templates are not much use if you can't inject dynamic data into them. With phew! you can embed Python expressions with {{<expression here>}} which will be evaluated during parsing.

Variables

In the simplest form you can embed a simple value by just enclosing it in double curly braces. It's also possible to perform more complicated transformations using any built in Python method.

  <div id="name">{{name}}</div>

  <div id="name">{{name.upper()}}</div>

  <div id="name">{{"/".join(name.split(" "))}}</div>

Lists

It is possible to perform operations on lists using str.join and list comprehension as in the following table example.

<table>
{{"".join([f"<tr><td>item</td></tr>\r\n" for item in mylist])}}
</table>

Conditional display

If you want to show a value only if some other condition is met then you can use the (slightly clunky) Python tenary operator.

<div>
  You won
  {{"1st" if prize == 1 else ""}}
  {{"2nd" if prize == 2 else ""}}
  {{"3rd" if prize == 3 else ""}}
  prize!
</div>

or

<div>
  You won
  {{["1st", "2nd", "3rd"][prize]}}
  prize!
</div>

While a bit unwieldy this methods works. An alternative would be to select the appropriate value in your handler and simply pass it into the template as a parameter however that would mean having some of your copy embedded into your Python code rather than all of it in one place in the template file.

Includes

You can include another template by calling render_template() again within your outer template.

include.html

Hello there {{name}}!

main.html

<!DOCTYPE html>
<body>
  {{render_template("include.html", name=name)}}
</body>

:warning: Note: you need to explicitly pass through template parameters into the included template - they are not available by default.

logging module

log(level, text)

Add a new entry into the log file.

log("info", "> i'd like to take a minute, just sit right there")
log("error", "> the license plate said 'Fresh' and it had dice in the mirror")

The entry will automatically have the current date and time, the level value, and the amount of free memory in kB prepended:

2022-09-04 15:29:20 [debug    / 110kB] > performing startup
2022-09-04 15:30:42 [info     / 113kB]   - wake reason: rtc_alarm
2022-09-04 15:30:42 [debug    / 112kB]   - turn on activity led
2022-09-04 15:30:43 [info     / 102kB] > running pump 1 for 0.4 second
2022-09-04 15:30:46 [info     / 110kB] > 5 cache files need uploading
2022-09-04 15:30:46 [info     / 107kB] > connecting to wifi network 'yourssid'
2022-09-04 15:30:48 [debug    / 100kB]   - connecting
2022-09-04 15:30:51 [info     /  87kB]   - ip address:  192.168.x.x
2022-09-04 15:30:57 [info     /  79kB]   - uploaded 2022-09-04T15:19:03Z.json 2022-09-04 15:31:01 [info     /  82kB]   - uploaded 2022-09-04T15:28:17Z.json 2022-09-04 15:31:06 [info     /  88kB]   - uploaded 2022-09-04T15:30:43Z.json 2022-09-04 15:31:11 [info     /  95kB]   - uploaded 2022-09-04T15:29:00Z.json 2022-09-04 15:31:16 [info     / 100kB]   - uploaded 2022-09-04T15:29:21Z.json 2022-09-04 15:31:16 [info     /  98kB] > going to sleep

debug(*items)

Shorthand method for writing debug messages to the log.

warn("> this is a story")

info(*items)

Shorthand method for writing information to the log.

num = 123
info("> all about how", num, time.time())

warn(*items)

Shorthand method for writing warnings to the log.

warn("> my life got flipped")

error(*items)

Shorthand method for writing errors to the log.

warn("> turned upside down")

set_truncate_thresholds(truncate_at, truncate_to)

Will automatically truncate the log file to truncate_to bytes long when it reaches truncate_at bytes in length.

# automatically truncate when we're closed to the
# filesystem block size to keep to a single block
set_truncate_thresholds(3.5 * 1024, 2 * 1.024)

Truncation always happens on the nearest line ending boundary so the truncated file may not exactly match the size specified.

dns module

To make implementing device provisioning interfaces (via captive portal) simple phew! provides a catchall DNS server.

If you put the Pico W into access point mode and then run the catchall DNS server it will route all DNS requests back to the local device so that they can be handled.

run_catchall

dns.run_catchall(ip_address)

Pass in the IP address of your device once in access point mode.

Helper functions

connect_to_wifi

connect_to_wifi(ssid, password, timeout=30)

Connects to the network specified by ssid with the provided password.

Returns the device IP address on success or None on failure.

access_point

access_point(ssid, password=None)

Create an access point with the specified SSID. Optionally password protected if provided.

is_connected_to_wifi

result = is_connected_to_wifi()

Returns True if there is an active WiFi connection.

get_ip_address

get_ip_address()

Returns the current IP address if connected to a network or acting as an access point or None otherwise.

Other Resources

Here are some Phew! community projects and guides that you might find useful. Note that code at the links below has not been tested by us and we're not able to offer support with it.

• :link: Hacking Big Mouth Billy Bass
• :link: How to set up a Phew! Access Point
• :link: Wireless Networking Setup Example for Raspberry Pi Pico W