resc-backend 3.3.1

Repository Scanner - Backend

Repository Scanner Backend (RESC-Backend)

[!NOTE]

This component is part of Repository Scanner - resc

Table of contents

1. About the component
2. Getting started
   • Prerequisites
   • Run RESC Web service locally from source
   • Run RESC Web service locally through Make
   • Run locally using docker
3. Testing
   • Run unit tests, linting and import checks locally
   • Run Newman tests locally
   • Run OWASP ZAP API Security tests locally
4. Database migrations
5. Documentation

About the component

The RESC-backend component includes database models, RESC Web service, Alembic scripts for database migration, RabbitMQ users, and queue creation.

Getting started

These instructions will help you to get a copy of the project up and running on your local machine for development and testing purposes.

Prerequisites

• Git
• Docker Desktop
• Python (v3.9.0 or higher)
• Install odbc 17 sql server driver for your OS
  • Download for Windows
  • Download for Mac
  • Download for Linux sudo apt install unixodbc-dev

Run RESC Web service locally

Run RESC Web service locally from source

Preview

Ensure resc database is up and running locally.
You can connect RESC web service to database, if you have already deployed RESC through helm in Kubernetes.
Open the Git Bash terminal from /components/resc-backend folder and run below commands.

Create virtual environment:

pip install virtualenv
virtualenv venv
source venv/Scripts/activate

Install resc_backend package:

pip install pyodbc==4.0.32
pip install -e .

Set environment variables:

source db.env
export MSSQL_SCHEMA=master
export MSSQL_DB_PORT=30880
export MSSQL_PASSWORD="<enter password for local database>"

Run Web service:

uvicorn resc_backend.resc_web_service.api:app --workers 1

Open http://127.0.0.1:8000 in a browser to access the API.

Run RESC Web service locally through make

Note: This procedure has been only tested in Linux and Mac. It may not work in machines running the Apple M1 chip due to lack of support from MSSQL docker image.

Prerequisites:

• Install Make on your system.
• Update MSSQL_PASSWORD (password you want to set for local database) in db.env file.

Preview

1. Create Python virtual environment and install resc_backend package:

make env

2. Run database locally:

make db

This target will run a local MSSQL instance in a container called resc-db. It creates and populates the resc database schema using alembic and the sql script located in test_data/database_dummy_data.sql

Note:: This target will also try to remove the DB container if it already exists.

If you want to remove this container, run: make cleandb

3. Run Web service:

make rws

Open http://127.0.0.1:1234 in a browser to access the API.

4. Clean up:

make clean

Run locally using docker

Preview

Run the RESC-Backend docker image locally with the following commands:

• Pull the docker image from registry:

docker pull rescabnamro/resc-backend:latest

• Alternatively, build the docker image locally by running following command: Ensure resc database is up and running locally.
  You can connect RESC web service to database, if you have already deployed RESC through helm in Kubernetes.
  

  Open the Git Bash terminal from /components/resc-backend folder and run below commands.
  Update MSSQL_PASSWORD value in the docker run command.

docker build -t rescabnamro/resc-backend:latest .

• Use the following command to run the RESC backend:

source db.env
docker run -p 8000:8000 -e DB_CONNECTION_STRING -e MSSQL_ODBC_DRIVER -e MSSQL_USERNAME -e RESC_REDIS_CACHE_ENABLE -e AUTHENTICATION_REQUIRED -e MSSQL_DB_HOST="host.docker.internal" -e MSSQL_PASSWORD="<enter password for local database>" -e MSSQL_SCHEMA="master" -e MSSQL_DB_PORT=30880 --name resc-backend rescabnamro/resc-backend:latest uvicorn resc_backend.resc_web_service.api:app --workers 1 --host 0.0.0.0 --port 8000

Open http://127.0.0.1:8000 in a browser to access the API.

Testing

(Back to top)

Run unit tests, linting and import checks locally:

See below commands for running various (unit/linting) tests locally. To run these tests you need to install tox. This can be done on Linux and Windows with Git Bash.

Run below commands to make sure that the unit tests are running and that the code matches quality standards:

pip install tox      # install tox locally

tox -v -e sort       # Run this command to validate the import sorting
tox -v -e lint       # Run this command to lint the code according to this repository's standard
tox -v -e pytest     # Run this command to run the unit tests
tox -v               # Run this command to run all of the above tests

Run Newman tests locally:

If you don't provide any argument to the script, then the default image value will be used

cd tests/newman_tests
./run_newman_tests.sh

If you can override the images by providing below arguments to the script.

cd tests/newman_tests
./run_newman_tests.sh -b <resc-backend image:tag> -d <resc-database image:tag>  -n <newman image:tag> 

Example: ./run_newman_tests.sh -b 'rescabnamro/resc-backend:latest' -d 'mcr.microsoft.com/azure-sql-edge:1.0.7' -n 'postman/newman:5.3.1-alpine'

Run OWASP ZAP API Security tests locally:

If you don't provide any argument to the script, then the default image value will be used

cd tests/zap_tests
./run_run_zap_api_tests.sh

If you can override the images by providing below arguments to the script.

cd tests/zap_tests
./run_run_zap_api_tests.sh -b <resc-backend image:tag> -d <resc-database image:tag>  -z <zap image:tag>

Example: ./run_newman_tests.sh -b 'rescabnamro/resc-backend:latest' -d 'mcr.microsoft.com/azure-sql-edge:1.0.7' -n 'owasp/zap2docker-weekly'

Documentation

(Back to top)

• Alembic documentation

• Connect to database using Azure Data Studio

• How to create a connection string

• Connect and view the database tables from Intellij IDEA Ultimate edition