Email client for the Opwen project
This repository contains the source code for the Opwen Lokole web-application.
The Lokole is a simple email client that has functionality like:
- Self-service creation of user accounts
- Read emails sent to the account
- Write emails including rich formatting
- Send attachments
All emails are stored in a local SQLite database. Once per day, the emails that were written during the past 24 hours get exported from the database, stored in a JSON file, compressed and uploaded to a location on Azure Blob Storage. The Opwen Server picks up these JSON files, manages the actual mailboxes for the users on the Lokole and sends new emails back to the Lokole by using the same compressed file exchange format.
The Lokole web-application is intended to run on low-spec Raspberry Pi 3 hardware (or similar). Read the “Production setup” section below for further information on how to set up the client devices.
Why is this useful?
Email is at the core of our modern life, letting us keep in touch with friends and family, connecting us to our businesses partners and fostering innovation through exchange of information.
However, in many parts of the developing world, email access is not very wide-spread, usually because bandwidth costs are prohibitively high compared to local purchasing power. For example, in the Democratic Republic of the Congo (DRC) only 3% of the population have access to emails which leaves 75 million people unconnected.
The Opwen Lokole is a project by the Canadian-Congolese NGO Ascoderu that aims to address this problem by tackling it from three perspectives:
- The Lokole is an email client that only uses bandwidth on a schedule. This reduces the cost of service as bandwidth can now be purchased when the cost is lowest. For example, in the DRC, $1 purchases only 65 MB of data during peak hours. At night, however, the same amount of money buys 1 GB of data.
- The Lokole uses an efficient data exchange format plus compression so that it uses minimal amounts of bandwidth, reducing the cost of service. All expensive operations (e.g. creating and sending of emails with headers, managing mailboxes, etc.) are performed on a server in a country where bandwidth is cheap.
- The Lokole only uses bandwidth in batches. This means that the cost of service can be spread over many people and higher savings from increased compression ratios can be achieved. For example, individually purchasing bandwidth for $1 to check emails is economically un-viable for most people in the DRC. However, the same $1 can buy enough bandwidth to provide email for hundreds of people via the Lokole. Spreading the cost in this way makes email access sustainable for local communities.
System overview & Data exchange format
Can be found in the opwen-cloudserver readme.
First, get the source code.
git clone https://github.com/ascoderu/opwen-webapp.git cd opwen-webapp
Second, install the system-level dependencies using your package manager, e.g. on Ubuntu:
sudo apt-get install -y python3-venv npm shellcheck sudo ln -s /usr/bin/nodejs /usr/bin/node
Third, use the makefile to verify your installation by running the tests and starting up the server. The makefile will automatically install all required dependencies into a virtual environment and set up some dummy environment variables for local development. The server will automatically reload whenever any of the Flask code or Jinja templates are changed.
make tests make server
The routes of the app are defined in views.py so take a look there for an overview of the entrypoints into the code.
For local development, you can set the following additional environment variables:
With these environment variables set, when the Lokole exchanges data with the server, it will not make any calls to Azure and instead depend on the files in the AZURE_ROOT directory. Any files uploaded to the server will be written to a subdirectory of AZURE_ROOT so that they can be inspected. To test sending emails from the server to the Lokole, simply create a file at $AZURE_ROOT/to-lokole/emails.pack and the Lokole device will ingest the emails in that file during the data exchange.
There is a script to set up a new Lokole device. The script will install the web app in this repository as well as standard web infrastructure like nginx and gunicorn. The script will also make ready peripherals like the USB modem used for data exchange, and set up any required background jobs such as the email synchronization cron job.
The setup script assumes that you have already set up:
- 3 Azure Storage Accounts, general purpose: for the cloudserver to manage its queues, tables and blobs
- 1 Azure Storage Account, blob storage: for the cloudserver and webapp to exchange email packages
- 1 Application Insights account: to collect logs from the cloudserver and monitor its operations
- 1 SendGrid account: to send and receive emails in the cloudserver
The setup script is tested with hardware:
- Raspberry Pi 3 running Raspbian Jessie lite v2016-05-27, v2017-01-11, and v2017-04-10
- Orange Pi Zero running Armbian Ubuntu Xenial
The setup script is also tested with USB modems:
The setup script installs the latest version of the webapp published to PyPI. New versions get automatically published to PyPI (via Travis) whenever a new release is created on Github.
You can run the script on your client device like so:
curl https://raw.githubusercontent.com/ascoderu/opwen-webapp/master/setup/setup-lokole.sh \ | bash -s \ <client-name> <storage-account-name> <storage-account-key> <sim-type> \ <email-key> <server-tables-name> <server-tables-key> <cloudflare-user> \ <cloudflare-key> <cloudflare-zone> <local-password> <sync-schedule>
Adding a new language
export LANG=ln make prepare-translations poedit "opwen_email_client/webapp/translations/$LANG/LC_MESSAGES/messages.po" make compile-translations
Or via Transifex.