burl 2.0.1

a url shortening django app

burl (brief url) is a URL shortener written in python with the django framework.

As of version 2, this application and repo is for the standalone burl service, providing a docker-packaged reference implementation of django-burl. If you’re looking for a URL-shortener to include in your own django project, see django-burl.

Features include:

• data models and REST API from django-burl

• JWT authentication

• CORS management via django-cors-headers

• swagger-ui

• user model with UUIDField for its primary key

• account management pages/templates

• static assets served via whitenoise

• gunicorn WSGI server

• easy configuration with cfitall

Quick Start

First, configure a postgres user and database to host burl’s data, then create a file /etc/burl/env specifing the environment variables for configuring burl (see below).

Run the latest image from docker hub (remember to change 10.0.0.10 to the ip of the postgres server you configured above):

docker pull wryfi/burl:latest
docker run -dit --name=myburl -p 8000:8000 --env-file /etc/burl/env \
    --add-host=dbhost:10.0.0.10 \
    --restart unless-stopped wryfi/burl:latest run
docker exec -it myburl burl-manager createsuperuser
docker exec -it myburl burl-manager set_default_site --name localhost --domain localhost

Point your browser to http://localhost:8000/admin and create some BURLs!

Or go old school:

curl \
  -X POST -H "Content-Type: application/json" \
  -d '{"username": "dooper", "password": "sooperuser"}' \
  http://localhost:8000/api/v2/token/auth
...
{
  "access":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX3BrIjoxLCJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiY29sZF9zdHVmZiI6IuKYgyIsImV4cCI6MTIzNDU2LCJqdGkiOiJmZDJmOWQ1ZTFhN2M0MmU4OTQ5MzVlMzYyYmNhOGJjYSJ9.NHlztMGER7UADHZJlxNG0WSi22a2KaYSfd1S-AuT7lU",
  "refresh":"eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX3BrIjoxLCJ0b2tlbl90eXBlIjoicmVmcmVzaCIsImNvbGRfc3R1ZmYiOiLimIMiLCJleHAiOjIzNDU2NywianRpIjoiZGUxMmY0ZTY3MDY4NDI3ODg5ZjE1YWMyNzcwZGEwNTEifQ.aEoAYkSJjoWH1boshQAaTkf8G3yn0kapko6HFRt7Rh4"
}

curl \
  -X POST -H "Content-Type: application/json" \
  -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX3BrIjoxLCJ0b2tlbl90eXBlIjoiYWNjZXNzIiwiY29sZF9zdHVmZiI6IuKYgyIsImV4cCI6MTIzNDU2LCJqdGkiOiJmZDJmOWQ1ZTFhN2M0MmU4OTQ5MzVlMzYyYmNhOGJjYSJ9.NHlztMGER7UADHZJlxNG0WSi22a2KaYSfd1S-AuT7lU" \
  -d '{"url": "https://archive.org", "burl": "arc"}' \
  http://localhost:8000/api/v2/burls/
...
{
  "burl": "arc",
  "created": "2022-03-14T16:16:09.353538-05:00",
  "description": "",
  "enabled": true,
  "updated": "2022-03-14T16:16:09.353543-05:00",
  "url": "https://archive.org",
  "user": "aec88b92-267f-430e-b4e2-0c63f4fc411a"
}

curl -IL "http://localhost:8000/arc/"
...
HTTP/1.0 302 Found
Content-Type: text/html; charset=utf-8
Location: https://archive.org
X-Frame-Options: DENY
Content-Length: 0
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
Vary: Origin
Server: Werkzeug/2.0.3 Python/3.10.2
Date: Mon, 14 Mar 2022 21:21:03 GMT

HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Mon, 14 Mar 2022 21:21:04 GMT
Content-Type: text/html; charset=utf-8
Connection: close
vary: Accept-Encoding
Strict-Transport-Security: max-age=15724800
Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0

Configuration

burl uses cfitall for managing its most commonly configured settings. It will search /etc/burl and then ~/.local/etc/burl for a burl.yml or burl.json settings file, and/or read its configuration from a series of environment variables.

Example yaml file:

admin:
  rough_count_min: 1000
api:
  page_size: 25
app:
  burl_blacklist:
  - admin
  - api
  - static
  - media
  debug: false
  default_redirect_url: https://www.wikipedia.org/
  hashid_alphabet: abcdefghjkmnpqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ0123456789
  media_root: /Users/burl/.local/var/burl/media
  static_root: /Users/burl/.local/share/burl/static
  time_zone: America/Los_Angeles
db:
  default:
    engine: django.db.backends.postgresql_psycopg2
    host: 127.0.0.1
    name: burl
    password: burl
    port: 5432
    user: burl
http:
  secure_proxy_ssl_header_name: HTTP_X_FORWARDED_PROTO
  secure_proxy_ssl_header_value: http
  use_x_forwarded_hosts: true
logging:
  app:
    level: warn
  burl:
    level: info
  log_dir: /Users/burl/.local/var/log/burl
mail:
  default_from_email: [e-mail]
  sendgrid_api_key: ''
security:
  allowed_hosts:
  - localhost
  - 127.0.0.1
  cors:
    allow_all_origins: false
    allowed_origin_regexes: []
    allowed_origins: []
  jwt:
    access_lifetime: 600
    refresh_lifetime: 86400
  secret_key: jeirainooyieShaequeeng8av9gah6geiv1ooTh6quoo9meireeRayoo6un7xah
  sendgrid_api_key: ''

Corresponding environment variables:

BURL__ADMIN__ROUGH_COUNT_MIN
BURL__API__PAGE_SIZE
BURL__APP__BURL_BLACKLIST
BURL__APP__DEBUG
BURL__APP__DEFAULT_REDIRECT_URL
BURL__APP__HASHID_ALPHABET
BURL__APP__MEDIA_ROOT
BURL__APP__STATIC_ROOT
BURL__APP__TIME_ZONE
BURL__DB__DEFAULT__ENGINE
BURL__DB__DEFAULT__HOST
BURL__DB__DEFAULT__NAME
BURL__DB__DEFAULT__PASSWORD
BURL__DB__DEFAULT__PORT
BURL__DB__DEFAULT__USER
BURL__HTTP__SECURE_PROXY_SSL_HEADER_NAME
BURL__HTTP__SECURE_PROXY_SSL_HEADER_VALUE
BURL__HTTP__USE_X_FORWARDED_HOST
BURL__LOGGING__APP__LEVEL
BURL__LOGGING__BURL__LEVEL
BURL__LOGGING__LOG_DIR
BURL__MAIL__DEFAULT_FROM_EMAIL
BURL__MAIL__SENDGRID_API_KEY
BURL__SECURITY__ALLOWED_HOSTS
BURL__SECURITY__CORS__ALLOWED_ORIGINS
BURL__SECURITY__CORS__ALLOWED_ORIGIN_REGEXES
BURL__SECURITY__CORS__ALLOW_ALL_ORIGINS
BURL__SECURITY__JWT__ACCESS_LIFETIME
BURL__SECURITY__JWT__REFRESH_LIFETIME
BURL__SECURITY__SECRET_KEY
BURL__SECURITY__SENDGRID_API_KEY

Of course, per the django convention, you can always set the DJANGO_SETTINGS_MODULE environment variable to a python module of your choice, to further extend or bypass all of burl’s settings and configuration mechanisms if needed.

Configuration Notes

Email

If you want working email (e.g. for password resets) the only supported option at this time is to use sendgrid. Set the security.sendgrid_api_key setting (BURL__SECURITY__SENDGRID_API_KEY environment variable) to enable sendgrid support. Otherwise all email is printed to the console and never sent.

Development

Implementation

burl is a reference implementation of django-burl, which implements most of the functionality found in burl. Please review django-burl’s documentation for details.

burl adds JWT authentication to django-burl via Simple JWT.

The current Swagger UI (api documentation) can be found at /api/v2/swagger of the running service.

The django admin can be found as usual at /admin.

code requirements

burl requires python 3.7 or newer. Python 2 is not supported.

burl should run anywhere python will run, most easily on a unix-like system.

database requirements

burl strongly recommends using a postgresql database via python’s psycopg2 library.

You will need a C compiler, python header files, and postgres development libraries on your system to build the postgres psycopg2 module needed for postgresql.

Installation

burl is made to be installed via the standard python installation methods. You can install it as simply as running:

pip install burl

It is recommended, however, that you install burl in a virtualenv or Docker container. For development, in particular, the easiest way to set everything up is to use pipenv (see below).

Once you have installed burl, you will need to create a database for its use. The default configuration expects a database called burl, owned by a user named burl, with a password of burl. You should alter these settings by using the configuration mechanisms described above.

Once your database is configured, run the database migrations to create:

burl-manager migrate

Then create a new superuser:

burl-manager createsuperuser

Now you should be ready to run burl! You can run a test/development server by running burl-manager runserver to ensure that everything is working. In production, you should deploy behind a WSGI server.

Deployment

burl is a straightforward django app, with nothing fancy.

You can deploy burl with any WSGI-compliant web server. Running gunicorn as the backend WSGI server, with an nginx reverse proxy in front of it, is a common and well-supported configuration.

Deploying Django has some generic information about deploying django applications that you may find useful if you are new to this stack.

Docker

The included Dockerfile builds a container that bundles burl with gunicorn and exposes gunicorn on port 8000. It builds with uid 65432 by default, which you can change on the docker build command line, e.g.:

docker build --build-arg uid=23456 -t burl .

This container does not include postgres or nginx. You will need postgres to run burl, and you will want to put nginx in front of the container.

Once you have a built container, it can be activated as follows:

docker run -dit -p 8000:8000 --env-file /etc/burl/env --add-host=dbhost:10.0.0.10 \
    --restart unless-stopped burl:latest burl

Tooling

burl uses a modern python toolchain, consisting of:

• pipenv for managing dependencies,

• pbr build system,

• docker support,

• semantic version numbers,

• git flow branching scheme.

To start coding, first install pipenv, then clone this repo and run pipenv install -d. This will set up a virtualenv, install all of the dependencies, and install burl in editable mode. You should now be able to run commands like pipenv shell, pipenv run burl-manager test, etc.

When using pipenv you can make use of a .env file in the source root, and set the requisite environment variables (above) there. This file is ignored in .gitignore and local to your environment.

See:

• Why Python devs should use Pipenv

Tests

burl was not developed using TDD, but has reasonable test coverage. Tests are located in the standard places for django applications. New PRs should include relevant tests whenever possible.