Skip to main content

Scrapes the NM Dept of Game and Fish fishing report

Project description

NM Fishing Report

master: master branch build status

Scrapes the NM Dept of Game and Fish fishing report

Features

  • Scrapes fishing report into sqlite database
  • Writes to a text file the reports for a specified list of spots
    • Basic Markdown format
    • I use this to output to a Dropbox folder for easy access on mobile
  • Monitors for keywords and can optionally use a custom notification script when those keywords appear in a report

Introduction

The NM Dept of Game and Fish publishes a biweekly fishing report. This script scrapes the report into a database so I can see how various spots fare throughout the year. It send me a Pushover notification whenever the report for one of my favorite spots has any of the buzzwords I've selected.

Dependencies

  • Python3
  • OS X or Linux
  • See requirements.txt

Quickstart

  1. Copy config-sample.ini (recommended: rename to config.ini, which is .gitignored)
  2. Modify config (see section below)
  3. pip3 install nmfishingreport
  4. python3 -m nmfishingreport -c /path/to/your-config.ini

Development Setup

  1. Clone the repo: git clone https://github.com/n8henrie/nmfishingreport && cd nmfishingreport
  2. Make a virtualenv: python3 -m venv .venv
  3. Install dev setup: ./.venv/bin/pip install .[dev]

Configuration

I recommend you start with config-sample.ini. I've tried to add comments to make it somewhat self-explanatory. A few notes:

  • You'll need to make sure your spelling matches NMDGF for fav_spots.
  • I've included my database file with some reports going back to 2015
    • There are likely several holes from times when the NMDGF updated their website and broke the script or times that my computer wasn't running
    • If you want to want to continue with my existing database, copy it to a more reasonable filename (recommended: fishing_reports.db), and use it as db in your config
    • The filename fishing_reports.db is .gitignore-d
    • I'll try to update the provided file from time to time

Notification config

If you know a bit of Python, you can optionally provide a notification script if you want to get a push notification (or email or what have you) when certain keywords show up in the report for one of your fav_spots. The file should expose a bare function notify that accepts two arguemnts:

  1. A dictionary containing the following keys:
    • spot: the fishing spot triggering the notification
    • report: the text of that spot's fishing report
    • url: the URL for the fishing report
  2. The path to your config file

Because it accepts a path to your config file, you can add a section to your config file to include usernames and passwords for the notification script if needed. For example, in the extras/ directory I've included my (working) notify.py for Pushover -- it pulls my Pushover credentials from my config file, which is kept out of version control.

Be forewarned that nmfishingreport loads the notification script to execing its contents, which I'm sure could have security ramifications or lead to data loss if you aren't careful. Leave the NOTIFY section out of your config to avoid this entirely.

Acknowledgements

  • NM Dept of Game and Fish!

Troubleshooting / FAQ / Examples

  • How can I get info out of an sqlite database?
    • I'm not terribly good at sqlite either, so here's how to convert it to a csv file that you can open in your spreadsheet app of choice:
      • sqlite3 -header -csv fishing_reports.db "SELECT * FROM nm_fishing_reports;" > fishingreport.csv
  • Some other ideas on looking at the contents:
    • Dump all reports to your screen:
      • sqlite3 fishing_reports.db 'SELECT * FROM nm_fishing_reports;'
    • Show the 10 most recent reports:
      • sqlite3 fishing_reports.db 'SELECT * FROM nm_fishing_reports ORDER BY date DESC LIMIT 10;'
    • Show the 5 most recent reports for the Jemez waters:
      • sqlite3 fishing_reports.db 'SELECT date, report FROM nm_fishing_reports WHERE spot LIKE "%Jemez%" ORDER BY date DESC LIMIT 5;'
    • Show the most recent spot to have had the word "excellent" in the report:
      • sqlite3 fishing_reports.db 'SELECT date, spot FROM nm_fishing_reports WHERE report LIKE "% excellent %" ORDER BY date DESC LIMIT 1;'
    • Show what times of year the Jemez fishing has been "very good" or "excellent":
      • sqlite3 fishing_reports.db 'SELECT date FROM nm_fishing_reports WHERE (report LIKE "% very good %" OR report LIKE "% excellent %") AND spot LIKE "%Jemez%";'
  • How can I run nmfishingreport automatically?
    • If you're on OS X, I've included an example launchd plist in extras/

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nmfishingreport-0.1.3.tar.gz (7.5 kB view details)

Uploaded Source

Built Distribution

nmfishingreport-0.1.3-py3-none-any.whl (7.6 kB view details)

Uploaded Python 3

File details

Details for the file nmfishingreport-0.1.3.tar.gz.

File metadata

  • Download URL: nmfishingreport-0.1.3.tar.gz
  • Upload date:
  • Size: 7.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.6

File hashes

Hashes for nmfishingreport-0.1.3.tar.gz
Algorithm Hash digest
SHA256 d6ccc1095066627f3f65c0f4ed6501f2ff1a12fa0b5055167d8baf069da6f931
MD5 984639ff022c9c040ba398143e806189
BLAKE2b-256 1f8772e83923c8ea496441573f8eb4fa0c82e0feede45fcf80183bc045dd55db

See more details on using hashes here.

File details

Details for the file nmfishingreport-0.1.3-py3-none-any.whl.

File metadata

  • Download URL: nmfishingreport-0.1.3-py3-none-any.whl
  • Upload date:
  • Size: 7.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.4.2 importlib_metadata/4.8.1 pkginfo/1.7.1 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.2 CPython/3.9.6

File hashes

Hashes for nmfishingreport-0.1.3-py3-none-any.whl
Algorithm Hash digest
SHA256 8350f6111311fdc8c34c29ed19027af412543fd974c9b40ae8dc5698fc0d6c53
MD5 58a431a2b26c25cc9fb0f1c92a646e1f
BLAKE2b-256 a8632165a7f643e3fc959c8009c2d6f46fe692a0259355e9fbbadc8f36df7e00

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page