Silence 1.0.4

An educational API+Web framework.

---

Introduction

Silence is a framework that allows for a rapid deployment of a RESTful API and a Web application around a relational database. Silence revolves around the concept of projects, which contain everything needed to deploy the previously mentioned elements and can be moved and/or shared between computers or systems with ease.

Silence has been built as an educational and teaching tool for its use in several courses of the Degree in Computer Science at the University of Seville.

Installation

Silence is available in the Python Package Index (PyPI) for Python ≥ 3.6. To install Silence, run:

pip install Silence

Silence also requires a connection to a MySQL/MariaDB database.

Creating a new project

Once Silence has been installed via pip, the silence command becomes available. To create a new project, run silence new <name>, where <name> is the name of the new project.

This will download a sample, working project that you can adapt to your needs. If you have a repo with your own Silence project template that you wish to download instead, you can run silence new <name> <repo_url.git>.

Configuring your project

The project settings can be found in settings.py. The available configuration parameters are:

• DEBUG_ENABLED Controls whether debug messages and Flask's debug mode are active (bool, default: False)
• LISTEN_ADDRESS IP address in which the web server will listen to requests (str, default: "0.0.0.0", all interfaces)
• HTTP_PORT Port in which the web server will listen to requests (int, default: 8080)
• SQL_SCRIPTS Sequence of files inside the sql/ folder to run when the silence createdb command is issued (list[str], default: [])
• API_PREFIX URL prefix for API requests (str, default: /api, do not set empty)
• DB_CONN Connection details to the MySQL/MariaDB database
  • host IP or name of the server (str, default: '127.0.0.1')
  • port Port the SQL server is listening to (int, default: 3306)
  • username User in the SQL server to use (str, default: default_username)
  • password Password for the previous user (str, default: default_password)
  • database Name of the schema to use, the user should have privileges to create and destroy tables in this schema (str, default: default_database)
• RUN_API Deploy the API endpoints (bool, default: True)
• RUN_WEB Deploy the static web server (bool, default: True)
• ENABLE_LOGIN Enables the /login endpoint (bool, default: True)
• ENABLE_REGISTER Enables the /register endpoint (bool, default: True)
• ENABLE_SUMMARY Enables the API summary endpoint (GET API_PREFIX) (bool, default: True)
• COLORED_OUTPUT Enables colors in the console output (bool, default: True)
• USER_AUTH_DATA Configures which information to use for login and register
  • table Name of the table containing your users (str, default: users)
  • identifier Column of this table containing the unique identifiers used for login (str, default: username)
  • password Column of this table containing the hashed passwords (str, default: password)
• SECRET_KEY Random string used for signing session tokens and Flask security. Generated automatically upon project creation when using silence new.
• MAX_TOKEN_AGE Time in seconds during which a session token is valid after it has been issued (int, default: 86400)

Creating the database

Silence provides the silence createdb command to automatically execute any number of SQL scripts to create your database and/or set it to a controlled initial state.

To use this command, put the .sql files that you want it to run inside the project's sql/ folder, and change the SQL_SCRIPTS configuration parameter accordingly in settings.py to reflect which files you want it to run and in which order, for example:

SQL_SCRIPTS = ["create_tables.sql", "populate_database.sql"]

Once this has been configured, silence createdb will execute the specified files in the desired order, using the connection and database/schema specified in DB_CONN.

Defining your API endpoints

The Silence philosophy is that API endpoints are wrappers around SQL operations, and thus they can be defined in a relatively simple manner. As an example, we will demonstrate how to create basic CRUD endpoints for a Department table whose columns are (departmentId, name, city).

We begin by creating a department.py file inside the project's api/ folder. The name of this file is not relevant, and all files inside the api/ folder containing endpoint declarations will be automatically detected and imported.

Endpoint declarations are done through the silence.decorators.endpoint decorator, so we have to import it:

from silence.decorators import endpoint

Endpoints require an URL route, an associated HTTP verb and the SQL query to execute, and they are bound to a Python function that will be executed every time it is called. Optionally, they can also include a description and be limited only to logged users.

GET operations

We define an endpoint to retrieve all existing Departments like this:

@endpoint(
    route="/departments",
    method="GET",
    sql="SELECT * FROM Departments",
    description="Shows all departments"
)
def get_all():
    pass

For GET operations, the associated function is empty, as there is nothing to check.

Note that the endpoint route is defined without the global /api prefix, it is automatically added later by Silence. In this case, whenever anyone performs an HTTP GET request against /api/departments, the SQL query will be executed and the results will be displayed as JSON.

GET /api/departments

(200 OK)
[
  {
    "city": "C\u00e1diz",
    "departmentId": 1,
    "name": "Arte"
  },
  {
    "city": null,
    "departmentId": 2,
    "name": "Historia"
  },
  {
    "city": "Sevilla",
    "departmentId": 3,
    "name": "Inform\u00e1tica"
  }
]

URL parameters can be defined and passed on to the SQL query. For example, we can also define an endpoint to retrieve a certain Department by its departmentId:

@endpoint(
    route="/departments/$departmentId",
    method="GET",
    sql="SELECT * FROM Departments WHERE departmentId = $departmentId",
    description="Shows one department by ID"
)
def get_by_id():
    pass

In this case, the URL parameter is defined using the $param syntax so Silence knows that it is a variable part of the URL pattern. It is passed to the SQL query using the same syntax by specifying the same parameter name. You can capture and pass as many parameters as you want, in any order, in this manner. Silence checks that all parameters in the SQL query can be obtained through the URL pattern.

All parameters in a SQL query, whether received via URL pattern or request body, are passed in a safe manner using SQL placeholders.

A user may thus request a specific Department by its ID using this endpoint:

GET /api/departments/3

(200 OK)
[
  {
    "city": "Sevilla",
    "departmentId": 3,
    "name": "Inform\u00e1tica"
  }
]

Endpoints whose routes end with a URL parameter automatically return a 404 HTTP code if the SQL query returns an empty result:

GET /api/departments/-1

(404 NOT FOUND)
{
    "code": 404,
    "message": "Not found"
}

POST/PUT operations

Editing data through POST and PUT requests follows the same guidelines, however, most commonly they will receive aditional parameters through the HTTP request body.

An endpoint to create a new Department is associated with a SQL INSERT operation:

from silence.exceptions import HTTPError

@endpoint(
    route="/departments",
    method="POST",
    sql="INSERT INTO Departments (name, city) VALUES ($name, $city)",
    description="Creates a new department"
)
def create(name, city):
    if len(name) < 3:
        raise HTTPError(400, "The department's name should have at least 3 characters")

Note that, in this case, the SQL query expects the parameters $name and $city, but they are not defined in the URL pattern. Instead, they are expected in the HTTP POST body. We declare this by setting them as parameters of the associated Python function: create(name, city).

By doing so, Silence knows that you're expecting to receive them in the body of the received HTTP request and can check your SQL query for unexpected parameters, and you can perform validations on the received parameters. You should declare these parameters as arguments of the associated function even if you don't want to validate them.

Silence can extract parameters from request bodies encoded in application/x-www-form-urlencoded or application/json:

POST /api/departments
Content-Type: application/json
{"name": "New department", "city": "Seville"}

(200 OK)
{
  "lastId": 4
}

For SQL operations other than SELECT, Silence returns the ID of the last modified row (in this case, it represents the ID assigned to the newly created resource).

The previously shown example also displays how the HTTPError exception can be used to validate received requests:

POST /api/departments
Content-Type: application/json
{"name": "..", "city": "Seville"}

(400 BAD REQUEST)
{
  "code": 400,
  "message": "The department's name should have at least 3 characters"
}

PUT requests are similar and combine both URL and body parameters, as udpate requests are aimed towards an already existing resource:

@endpoint(
    route="/departments/$departmentId",
    method="PUT",
    sql="UPDATE Departments SET name = $name, city = $city WHERE departmentId = $departmentId",
    description="Updates an existing department"
)
def update(name, city):
    if len(name) < 3:
        raise HTTPError(400, "The department's name should have at least 3 characters")

Silence makes sure that all parameters included in your SQL string can be obtained from either the request URL (declared in your route pattern) or the request body (declared in your function arguments).

DELETE operations

An example of an endpoint to delete a Department by its departmentId is as follows:

@endpoint(
    route="/departments/$departmentId",
    method="DELETE",
    sql="DELETE FROM Departments WHERE departmentId = $departmentId",
    description="Removes an existing department",
)
def delete():
    pass

In this case only the URL parameter is needed, and thus nothing is validated in the associated function.

Default endpoints and utilities

Unless explicitly disabled in the project's settings, Silence provides endpoints to register a new user and log in with an existing one. Additionaly, Silence enhances user-defined endpoints with some additional utilities:

Summary endpoint

A HTTP GET request to the API prefix will result in a list of the currently enabled endpoints, along with their descriptions if they have been provided:

GET /api

(200 OK)
[
  {
    "desc": "Returns the data regarding the API endpoints",
    "method": "GET",
    "route": "/api"
  },
  {
    "desc": "Starts a new session, returning a session token and the user data if the login is successful",
    "method": "POST",
    "route": "/api/login"
  },
  {
    "desc": "Creates a new user, returning a session token and the user data if the register is successful",
    "method": "POST",
    "route": "/api/register"
  },
  {
    "description": "Shows all departments",
    "method": "GET",
    "route": "/api/departments"
  },
  {
    "description": "Shows one department by ID",
    "method": "GET",
    "route": "/api/departments/$departmentId"
  },
  {
    "description": "Creates a new department",
    "method": "POST",
    "route": "/api/departments"
  },
  {
    "description": "Updates an existing department",
    "method": "PUT",
    "route": "/api/departments/$departmentId"
  },
  {
    "description": "Removes an existing department",
    "method": "DELETE",
    "route": "/api/departments/$departmentId"
  }
]

/register endpoint

If ENABLE_REGISTER is enabled, which it is by default, Silence will deploy a /register endpoint to enable users to register in your application. This endpoint must be accessed through a HTTP POST request, including the desired fields in the request body, and it takes care of:

• Ensuring that the request contains at least the identifier and password fields specified in USER_AUTH_DATA.identifier and USER_AUTH_DATA.password
• Ensuring that the identifier does not exist already in the users table specified in USER_AUTH_DATA.table
• Hashing the submitted password
• Starting a new session if the register is successful, returning a session token and the user data

For example, let us assume that you have a Users table containing the columns (userId, username, password, email) and that you have set up your project to use username as the identifier field and password as the password field:

USER_AUTH_DATA = {
    "table": "Users",
    "identifier": "username",
    "password": "password",
}

Then, in order to register a new user, you need to submit a POST request to /register with the username, password and any additional fields that you wish to insert:

POST /api/register
Content-Type: application/json
{"username": "new_user", "password": "123456", "email": "[e-mail]"}

(200 OK)
{
  "sessionToken": ".eJwl[...]7qMo",
  "user": {
    "email": "[e-mail]",
    "userId": 5,
    "username": "new_user"
  }
}

You have successfully registered, and Silence returns a session token under the response field "sessionToken" and the currently logged in user data under "user".

Future attempts to register with the same identifier will be met with an error:

POST /api/register
Content-Type: application/json
{"username": "new_user", "password": "123456", "email": "[e-mail]"}

(400 BAD REQUEST)
{
  "code": 400,
  "message": "There already exists another user with that username"
}

Please note that, in order to be flexible and support any possible table, /register assumes that all fields received in the request exist in the users table, and it will try to insert values in them. Any received fields that have no matching columns in the specified table will be ignored.

/login endpoint

If ENABLE_LOGIN is enabled, which it is by default, Silence will deploy a /login endpoint to enable users to log in in your application. This endpoint must be accessed through a HTTP POST request, including the identifier and password fields in the request body.

Like /register, the /login endpoint uses the table and columns specified in the USER_AUTH_DATA setting and expects to receive the identifier and password fields in the POST body. /login receives a clear-text password and compares it against the hashed version in the database:

POST /api/login
Content-Type: application/json
{"username": "new_user", "password": "123456"}

(200 OK)
{
  "sessionToken": ".eJwl[...]WKXc",
  "user": {
    "email": "[e-mail]",
    "userId": 5,
    "username": "new_user"
  }
}

The response data is the same as in /register.

Restricting endpoints to logged users

All @endpoint() declarations accept an optional auth_required argument that can be set to True if the endpoint is intended to be used only by logged users:

@endpoint(
    route="/departments",
    method="GET",
    sql="SELECT * FROM Departments",
    description="Shows all departments",
    auth_required=True  # <=======
)
def get_all():
    pass

When an endpoint is protected in this manner, the user has to prove that they have a current session by sending their session token as an HTTP header, under the key Token:

GET /api/departments
Token: .eJwl[...]WKXc

(200 OK)
[
  {
    "city": "C\u00e1diz",
    "departmentId": 1,
    "name": "Arte"
  },
  {
    "city": null,
    "departmentId": 2,
    "name": "Historia"
  },
  {
    "city": "Sevilla",
    "departmentId": 3,
    "name": "Inform\u00e1tica"
  }
]

Session tokens are provided by the /register and /login endpoints under the "sessionToken" response field, and they remain valid for the duration specified in the MAX_TOKEN_AGE setting. If the user tries to access a restricted endpoint without providing a valid session token, the server responds with a 401 HTTP code:

GET /api/departments
(No "Token" header)

(401 UNAUTHORIZED)
{
  "code": 401,
  "message": "Unauthorized"
}

URL query parameters in GET requests

Silence provides all user-defined GET endpoints with automatic filtering, paging and ordering via URL query params:

• _sort determines the field by which the results will be ordered (default: order determined by the database)
• _order determines the sorting order ("asc" or "desc", default: "asc")
• _limit determines the maximum amount of results to show (default: no limit)
• _page determines the page of results to show (to be combined with _limit for easy paging, default: 0)
• All other query parameters will be interpreted as a field that has to be filtered on

For example:

Get all departments whose city is Sevilla:

GET /api/departments?city=Sevilla

(200 OK)
[
  {
    "city": "Sevilla",
    "departmentId": 3,
    "name": "Inform\u00e1tica"
  }
]

Get all departments whose city is Sevilla AND their name is Arte:

GET /api/departments?city=Sevilla&name=Arte

(200 OK)
[]

Get all departments ordered by decreasing departmentId:

GET /api/departments?_sort=departmentId&_order=desc

(200 OK)
[
    {
        "city": "Sevilla",
        "departmentId": 3,
        "name": "Inform\u00e1tica"
    },
    {
        "city": null,
        "departmentId": 2,
        "name": "Historia"
    },
    {
        "city": "C\u00e1diz",
        "departmentId": 1,
        "name": "Arte"
    }
]

Get the second page of the previous query, with 2 results per page:

GET /api/departments?_sort=departmentId&_order=desc&_limit=2&_page=1

(200 OK)
[
  {
    "city": "C\u00e1diz",
    "departmentId": 1,
    "name": "Arte"
  }
]

These parameters can be combined in any way and work for all GET endpoints.

Static web server

Silence also serves as a web server for static files (unless explicitly disabled via the RUN_WEB setting). The docs/ folder inside your project is the web root, and thus you can place your web application there to be deployed by Silence.

The web server has no prefix. Accessing http://<address>/ will hit the index.html file located in the root of the docs/ folder. Any subfolders will work as expected, with the only exception of a route that creates a conflict with the API prefix (for example, if your API prefix is /api, do not create an api/ folder directly in the root of docs/).

Running the server

Once you have configured your project, database and defined your endpoints, you can launch the web server with:

silence run

Access logs, debug messages (if allowed) and uncontained exceptions will be logged directly to the console.

Changelog

See CHANGELOG.md

Contributions

All contributions are welcome provided that they follow our Code of Conduct.

We keep a TO-DO with the changes that we'd like to implement in the future. Feel free to open an issue if you need clarifications with any of its items.

Disclaimer

Silence has been designed and built with educational-only use in mind. It makes no specific efforts to ensure efficiency, security, or fitness to purposes other than educational ones. We do NOT recommend its use in a production environment of any kind.

License

MIT License