scrilla 1.3.11

a financial optimization program

scrilla: A Financial Optimization Application

Documentation

Table of Contents

• Setup
  • Installation
  • Dependencies
  • Configuration
• Environment
  • Required Configuration
  • Optional Configuration
• Usage
  • Command Line
  • Programmatic

This is a financial application that calculates asset correlations, statistics and produces various graphical summaries of data. It was originally designed to optimize portfolio allocations using data it retrieves from external services (currently: AlphaVantage, IEX and Quandl). Its scope has since expanded as way for the author to test the limits of his knowledge, as it were.

Statistical calculations are informed using the results of Ito Calculus, i.e. the sampling distributions for returns on assets are derived through stochastic differential equations. The parameters of these distributions are estimated with statistical techniques. scrilla can estimate financial statistics in a variety of ways; it can be configured to estimate using the method of moment matching, method of percentile matching(section 4.1.3.2) or maximum likelihood estimation. The estimation method can be stored in an environment variable to set a default for calculations, or explicitly passed into statistical functions. See the documentation for more information.

In terms of its original purpose, portfolios are optimized, using the procedures of Modern Portfolio Theory, by minimizing the portfolio's variance/volatility, i.e. by finding the optimal spot on the portfolio's efficient frontier as defined by the CAPM model. Alternatively, portfolios can be optimized by maximizing the portfolio's Sharpe ratio or by minimizing the portfolio's Conditional Value at Risk.

This application optimizes across asset classes, i.e. the theoretical portfolio being constructed can be composed of equities, cryptocurrencies or both. In a future release, I would like to include fixed income assets, volatility assets (VIX futures, options, etc.) and other derivatives, but for now, only those two asset types are supported. I am looking for a good API that provides historical data on the other types of financial instruments before I bring them into the optimization algorithm, so if you know of one, contact me.

The program's functions are wrapped in PyQt5 widgets which provide a user interface (this feature is still in development and may explode). In addition, visualizations are created by matplotlib.

The links below will take you to the registration pages for each API service Key,

AlphaVantage API Key Registration
Quandl API Key Registration
IEX API Key Registration

Setup

Installation

Install the package with the Python package manager,

pip install scrilla

This will install a command line interface on your path under the name scrilla. Confirm your installation with with the -version flag,

scrilla --version

If you are on Windows, you may need to add your Python scripts bin to the $PATH. To keep the installation as minimal as possible, the base package does not include the GUI libraries. You can install the optional GUI dependency with,

pip install scrilla[gui]

Note, the GUI has a different CLI entrypoint, namely,

scrilla-gui

If you prefer, you can build from source. git clone the repository and then from the root directory install the project dependencies and build the library,

pip3 install -r requirements
python3 -m build

cd into the generated /dist/ to manually install the packaged code,

pip install scrilla-<major>.<minor>.<micro>-py3-none-any.whl

Dependencies

You will need Python3.8 or greater. This application depends on the following Python libraries:

Required

• dateutil>=2.8.2
  
• holidays>=0.10.4
  
• matplotlib>=3.3.3
  
• numpy>=1.19.3
  
• python-dotenv>=0.17.0
  
• requests>=2.25.0
  
• scipy>=1.5.4
  

Optional

• PySide6>=6.2.0
  

Configuration

In order to use this application, you will need to register for API keys. The program will need to be made aware of these keys somehow. The best option is storing these credentials in environment variables. See Required Configuration for more information. You can also invoke the CLI function store to store the credentials in the local installation /data/common/ directory. To do so,

scrilla store --key <key> --value <value>

where <key> is one of the values: ALPHA_VANTAGE_KEY, QUANDL_KEY or IEX_KEY. <value> is the corresponding key itself given to you after registration. <value> is case-sensitive!

Keep in mind if using this method to store the API keys, the keys will be stored unencrypted in the local /data/common/ directory.

Environment

A sample environment file is located here, along with comments describing the purpose of each variable. The application sets sensible defaults for most of the environment variable configurations, but there are several required environment variables you will need to set yourself.

Required Configuration

As mentioned, you will need to register for API keys at AlphaVantage, IEX and Quandl. One way of passing API keys to the program is by storing these in your session's environment. scrilla will search for environment variables named ALPHA_VANTAGE_KEY, QUANDL_KEY and IEX_KEY. You can add the following lines to your .bashrc profile or corresponding configuration file for whatever shell you are using,

export ALPHA_VANTAGE_KEY=<key goes here>
export QUANDL_KEY=<key goes here>
export IEX_KEY=<key goes here>

If no API keys are found in these variables, the application will not function properly; be sure to load these variables into your shell session before using scrilla.

Optional Configuration

scrilla can be configured with the following optional environment variables. Each variable in this list has a suitable default set and so does not need changed unless the user prefers a different setting. Make sure you export these values from your current session, i.e.

export RISK_FREE=ONE_YEAR
scrilla risk-free # returns one year risk free rate
export RISK_FREE=THREE_YEAR
scrilla risk-free # returns three year risk free rate

• ANALYSIS_MODE

The asset price model assumed during the estimation of statistics. A value ofgeometric corresponds geometric brownian motion model where the return is proportional to the asset price. The constant of proportionality is a random variable with a constant mean and volatility. A value of reversion refers to a mean reverting model where the return is proportional to the difference between the asset price and a stationary mean. The unit of proportionality is a random variable with constant mean and volatility.

Note, it is highly recommended that if you change this value, you should clear the cache, as the cache stores frequent statistical calculations to speed up the program. The previous values of the statistics calculated under the prior model will be referenced and result in incorrect calculations.

• DEFAULT_ESTIMATION_METHOD

Determines the method used to calculate risk-return profiles. If set to moments, the return and volatility will be estimated by setting them equal to the first and second sample moments. If set to percents, the return and volatilty will be estimated by setting the 25th percentile and 75th percentile of the assumed distribution (see above ANALYSIS_MODE) equal to the 25th and 75th percentile from the sample of data. If set to likely, the likelihood function calculated from the assumed distribution (see ANALYSIS_MODE again) will be maximized with respect to the return and volatility; the values which maximize will be used as the estimates.

• RISK_FREE

Determines which annualized US-Treasury yield is used as stand-in for the risk free rate. This variable will default to a value of ONE_YEAR, but can be modified to any of the following: ONE_MONTH, THREE_MONTH, SIX_MONTH, ONE_YEAR, THREE_YEAR, FIVE_YEAR, TEN_YEAR, THIRTY_YEAR.

• MARKET_PROXY

Determines which ticker symbol is used as a proxy for the overall market return. This variable will default to a value of SPY, but can be set to any ticker on the stock market. Recommended values: SPY, QQQ, DJI or VTI.

• FRONTIER_STEPS

Determines the number of data points in a portfolio's efficient frontier. This variable will default to a value of 5, but can be set equal to any integer.

• MA_1, MA_2, MA_3

Determines the number of days used in the sample for moving average series and plots. These variables default to the values of 20, 60 and 100. In other words, by default, moving average plots will display the 20-day moving average, the 60-day moving average and the 100-day moving average. These variables can be set equal to any integer, as long as MA_1 < MA_2 < MA_3.

• FILE_EXT

Determines the type of files that are output by scrilla. This variable is currently only defined for an argument of json. A future release will include csv.

• LOG_LEVEL

Determines the amount of output. Defaults to info. Allowable values: none, info, debug or verbose. Be warned, verbose is extremely verbose.

Usage

Command Line

Most functions have been wired into command line arguments. For a full list of scrilla's functionality,

scrilla help

The main usage of scrilla is detailed below.

Synatx

scrilla [COMMAND] [TICKERS] [OPTIONS]

Commands: asset,cvar,var,capm-equity,capm-beta,clear-cache,clear-static,clear-common,close,correlation,correlations,discount-dividend-model,dividends,efficient-frontier,help,interest,watchlist,max-return,mov-averages,optimize-portfolio,optimize-cvar,plot-correlations,plot-dividends,plot-efficient-frontier,plot-moving-averages,plot-returns,plot-risk-profile,plot-yield-curve,prices,purge,risk-free,risk-profile,screen,sharpe-ratio,stat,stats,store,version,watch,yield-curve

Tickers: space-separated list of asset tickers/statistic symbols/interest maturities (depending on the command)

Options: command-specific flags and configuration.

Optimization

1. Volatility Minimization & Sharpe-Ratio Maximization

A portfolio of consisting of the equities ALLY, BX and SONY can be optimized with the following command,

scrilla optimize-portfolio ALLY BX SONY

By default, scrilla will optimize over the last 100 trading days. If you wish to optimize over a different time period, you may use the -start and -end argument flags to provide starting and ending dates in the YYYY-MM-DD format.

Also by default, the optimization function will minimize the portfolio variance. You can also specify the portfolio should be maximized with respect to the Sharpe ratio,

scrilla optimize-portfolio ALLY BX SONY -sh

There are several other arguments you may use to configure your optimization program. The full list of arguments is shown below,

scrilla optimize-portfolio [TICKERS] --sh \
                                     --start <YYYY-MM-DD> \
                                     --end <YYYY-MM-DD> \
                                     --save <absolute path to json file> \
                                     --target <float> --invest <float>

--target will optimize the portfolio with the additional constraint that its rate of return must equal target. Note the target return must be between the minimum rate of return and maximum rate of return in a basket of equities. For example, if ALLY had a rate of return of 10%, BX 15%, SONY 20%, the frontier of possible rates of returns resides in the range [10%, 20%]. It is impossible to combine the equities in such a way to get a rate of return less than 10% or one greater than 20%. Note, this assumes shorting is not possible. A future release will relax this assumption and allow portfolio weights to be negative.

--invest represents the total amount of money invested in a portfolio.

For example, the following command,

scrilla optimize-portfolio ALLY BX SONY --sh \
                                        --save <path-to-json-file> \
                                        --target 0.25 \
                                        --invest 10000 \
                                        --start 2020-01-03 \
                                        --end 2021-05-15

Will optimize a portfolio consisting of ALLY, BX and SONY using historical data between the dates of January 1st, 2020 and May 15th, 2021. The portfolio will be constrained to return a rate of 25%. A total $10,000 will be invested into this portfolio (to the nearest whole share). The output of this command will look like this,

---------------------------------------------- Results ----------------------------------------------
----------------------------------- Optimal Percentage Allocation -----------------------------------
ALLY = 22.83 %
BX = 19.26 %
SONY = 57.91 %
-------------------------------------- Optimal Share Allocation --------------------------------------
ALLY = 42
BX = 15
SONY = 56
-------------------------------------- Optimal Portfolio Value --------------------------------------
>> Total = $ 9893.98
---------------------------------------- Risk-Return Profile ----------------------------------------
>> Return = 0.25
>> Volatility = 0.201
----------------------------------------------------------------------------------------------------

Note the optimal share allocation does not allow fractional shares. scrilla will attempt to get as close to the total investment inputted without going over using only whole shares. Also note the return of this portfolio is 25%, as this was inputted into the target return constraint.

2. Conditional Value at Risk Minimization

The portfolio optimization can also be done by minimizing its conditional value at risk. Because the underlying calculations are a bit different, this function is accessed through a different command and requires different arguments.

The two new arguments are prob and expiry. prob, in essence, represents the percentile of the portfolio's distribution on which the value at risk will be conditioned. In other words, if the portfolio value is represented by a random variable P, for a given value of P=p, the prob is the probability such that Probability(P<p)=prob.

expiry represents the time horizon over which the value at risk will be calculated, i.e. the point in time in which the hypothetical loss occurs.

With these two new arguments, a portfolio's conditional value at risk can be optimized using the following,

scrilla optimize-cvar ALLY BX SONY --prob 0.05 --expiry 0.5

The command given above will optimize the portfolio's value at risk consisting of ALLY, BX and SONY over the next half year (expiry = 0.5) conditioned the value at risk being in the 5th percentile.

Other Notable Features

1. Distribution Modes

scrilla will assume an asset price process and therefore a probability distribution for the population of asset returns. The model scrilla assumes is determined by the environment variable ANALYSIS_MODE. Currently, only one model is available: geometric, which corresponds to an assume price process that follows geometric brownian motion and thus a probability distribution for the asset returns that is log-normal.

In the near future, a mean reversion model will implemented.

2. Estimation Modes

scrilla can estimate model parameters in a number of ways. The default estimation method is defined by the environment variable DEFAULT_ESTIMATION_METHOD, but all statistical functions can have their estimation overridden with a flag. scrilla supports three estimation modes: moments, percents and likely.

moments will use the method of moment matching, where the moments of a sample of data are equated to the moments of the assumed distribution in order to determine the distribution parameters. percents will use the method of percentile matching, where the first and third quartile of the sample are equated to the theoretical distribution percentiles to determine the distribution parameters. likely will use maximum likelihood estimation, where the probability of each observation given the assumed distribution is calculated and then the intersection of the probabilities is minimized with respect to the distribution parameters. (Note: the underlying distribution can be configured through the ANALYSIS_MODE environment variable; see Environment for more information)

For example, the following command will return the risk profile of ACI using the method of moment matching,

scrilla risk-profile ACI --moments

Where as the following command will return the risk profile of ACI using maximum likelihood estimation,

scrilla risk-profile ACI --likelihood

And the following command will return the risk profile of ACI using the method of percentile matching,

scrilla risk-profile ACI --percentiles

Note, the following command,

scrilla risk-profile ACI

will return the risk profile of ACI using the method set in the DEFAULT_ESTIMATION_METHOD environment variable. If this variable is not set, it will default to a value of moments.

2. Discount Dividend Model

scrilla will pull an equity's dividend payment history, regress the payment amount against its date and infer a simple linear regression model from this time series. It will use this model to project future dividend payments and then calculate the current cost of equity and use that to discount the sum of dividend payments back to the present. The following command will perform this action,

scrilla ddm ALLY

Alternatively, you can visualize the dividend payments against the regression model with a matplotlib graphic,

scrilla plot-divs ALLY

3. Financial Statistics

   • Beta: scrilla capm-beta [TICKERS] [OPTIONS]
   • Correlation Matrix: scrilla cor [TICKERS] [OPTIONS]
   • Conditional Value At Risk scrilla cvar [TICKERS] -prob PROB -expiry EXP [OPTIONS]
   • Cost Of Equity: scrilla capm-equity [TICKERS] [OPTIONS]
   • Risk-Return Profile: scrilla risk-profile [TICKERS] [OPTIONS]
   • Sharpe Ratio: scrilla sharpe-ratio [TICKERS] [OPTIONS]
   • Value At Risk: scrilla var [TICKERS] -prob PROB -expiry EXP [OPTIONS]

4. Stock Watchlist and Screening

Stocks can be added to your watchlist with,

scrilla watch [TICKERS]

You can then screen stocks according to some criteria. For example, the following command will search your watchlist for stock prices that are less than their Discount Dividend Model (very rare this happens...),

scrilla screen --criteria DDM

5. Visualizations
   • Discount Dividend Model: scrilla plot-divs [TICKER] [OPTIONS]
     • NOTE: THIS FUNCTION ONLY ACCEPTS ONE TICKER AT A TIME.
   • Efficient Fronter: scrilla plot-ef [TICKERS] [OPTIONS]
   • Moving Averages: scrilla plot-mas [TICKERS] [OPTIONS]
   • Risk Return Profile: scrilla plot-rp [TICKERS] [OPTIONS]
   • Yield Curve: scrilla plot-yield
   • QQ Plot of Returns: scrilla plot-rets [TICKER] [OPTIONS]
     • NOTE: THIS FUNCTION ONLY ACCEPTS ONE TICKER AT A TIME
   • Correlation Time Series scrilla plot-cors [TICKERS] [OPTIONS]
     • NOTE: THIS FUNCTION ACCEPTS EXACTLY TWO TICKERS

Notes

1. The following symbols have both equity and crypto assets trading on exchanges:

['ABT', 'AC', 'ADT', 'ADX', 'AE', 'AGI', 'AI', 'AIR', 'AMP', 'AVT', 'BCC', 'BCD', 'BCH', 'BCX', 'BDL', 'BFT', 'BIS', 'BLK', 'BQ', 'BRX', 'BTA', 'BTG', 'CAT', 'CMP', 'CMT', 'CNX', 'CTR', 'CURE', 'DAR', 'DASH', 'DBC', 'DCT', 'DDF', 'DFS', 'DTB', 'DYN', 'EBTC', 'ECC', 'EFL', 'ELA', 'ELF','EMB', 'ENG', 'ENJ', 'EOS', 'EOT', 'EQT', 'ERC', 'ETH', 'ETN', 'EVX', 'EXP', 'FCT', 'FLO', 'FLT', 'FTC', 'FUN', 'GAM', 'GBX', 'GEO', 'GLD', 'GNT', 'GRC', 'GTO', 'INF', 'INS', 'INT', 'IXC', 'KIN', 'LBC', 'LEND', 'LTC', 'MAX', 'MCO', 'MEC', 'MED', 'MGC', 'MINT', 'MLN', 'MNE', 'MOD', 'MSP', 'MTH', 'MTN', 'MUE', 'NAV', 'NEO', 'NEOS', 'NET', 'NMR', 'NOBL', 'NXC', 'OCN', 'OPT', 'PBT', 'PING', 'PPC', 'PPT', 'PRG', 'PRO', 'PST', 'PTC', 'QLC', 'QTUM','R', 'RDN', 'REC', 'RVT', 'SALT', 'SAN', 'SC', 'SKY', 'SLS', 'SPR', 'SNX', 'STK', 'STX', 'SUB', 'SWT', 'THC', 'TKR', 'TRC', 'TRST', 'TRUE', 'TRX', 'TX', 'UNB', 'VERI', 'VIVO', 'VOX', 'VPN', 'VRM', 'VRS', 'VSL', 'VTC', 'VTR', 'WDC', 'WGO', 'WTT', 'XEL', 'NEM', 'ZEN']

Since there is no way good way to distinguish whether or not the asset is an equity or a cryptocurrency based on the value of the ticker alone, the module functions scrilla.files.get_asset_type and scrilla.errors.validate_asset_type will always default to the equity ticker for the above symbols.

This is not the greatest solution, as all the crypto symbols given above are inaccessible to analysis. In particular, ETH represents a popular crypto that cannot be analyzed, which represents a major failing of the current application.

The way the service module works, PriceManager can be forced to retrieve the crypto asset's prices instead of the equity asset's through the services.PriceManager.get_prices method by providing the method an argument of asset_type='crypto'; However, the service module function services.get_daily_price_history, which is the point of contact between the PriceManager and the rest of the application, wraps calls to the PriceManager.get_prices method in a cache persistence layer (meaning, get_daily_price_history checks if prices exist in the cache before passing the request off to an external service query). The cache doesn't distinguish asset types currently. The PriceCache stores prices based on the inputs (ticker, date, open close, close price). So, even if the PriceManager is forced to get crypto prices on the first call, subsequent calls to the same get_daily_price_history function will likely break the application, or at least lead to misleading results, since the cache will contain a set of prices that doesn't necessarily map one-to-one with its ticker symbol.

If the above problem is to be solved, the cache needs modified to separate prices based on asset type.

2. There is a slight discrepancy between the results of maximum likelihood estimation and moment matching when the underyling distribution of the price process is log-normal. The likelihood algorithm in this library relies on the generalized idea of likelihood estimation; it will compute the log-likelihood function for a given vector of parameters and then optimize that function by varying the vector until the input that produces the maximum output; the usual matter of course is to derive a formula using calculus that can then be analytically solved. Both operations should be equivalent. Moreover, theoretically, it can be shown the maximization operation should be equivalent to the results obtained by the moment matching operation, i.e the maximum likelihood estimator for the mean is the sample mean, etc. However, the results between maximum likelihood estimation and moment matching are off by a few decimal points. It may be due to some vagary of floating point arithmetic, but something else may be going on. See comments in `scrilla.analysis.models.geometric.statistics'

Documentation

• dateutil
• holidays
• matplotlib
• numpy
• pyqt
• requests
• scipy
• sqlite3