Skip to main content

Tools for scraping personal financial data.

Project description

Python package for scraping personal financial data from financial institutions.

License: GPL v2 Build Status

This package may be useful on its, but is specifically designed to be used with beancount-import.

Supported data sources

Setup

To install the most recent published package from PyPi, simply type:

pip install finance-dl

To install from a clone of the repository, type:

python setup.py install

or for development:

python setup.py develop

Configuration

Create a Python file like example_finance_dl_config.py.

Refer to the documentation of the individual scraper modules for details.

Usage

You can run a scraping configuration named myconfig as follows:

python -m finance_dl.cli --config-module example_finance_dl_config --config myconfig

The configuration myconfig refers to a function named CONFIG_myconfig in the configuration module.

Make sure that your configuration module is accessible in your Python sys.path. Since sys.path includes the current directory by default, you can simply run this command from the directory that contains your configuration module.

By default, the scrapers run fully automatically, and the ones based on selenium and chromedriver run in headless mode. If the initial attempt for a selenium-based scraper fails, it is automatically retried again with the browser window visible. This allows you to manually complete the login process and enter any multi-factor authentication code that is required.

To debug a scraper, you can run it in interactive mode by specifying the -i command-line arugment. This runs an interactive IPython shell that lets you manually invoke parts of the scraping process.

License

Copyright (C) 2014-2018 Jeremy Maitin-Shepard.

Distributed under the GNU General Public License, Version 2.0 only. See LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for finance-dl, version 1.0.0
Filename, size File type Python version Upload date Hashes
Filename, size finance-dl-1.0.0.zip (32.9 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page