A simple python module that scrapes XML data from the government of Western Australia's FuelWatch website that makes parsing a breeze.
Project description
______ __ __ __
/ ____/_ _____ / / ______ _/ /______/ /_ ___ _____
/ /_ / / / / _ \/ / | /| / / __ `/ __/ ___/ __ \/ _ \/ ___/
/ __/ / /_/ / __/ /| |/ |/ / /_/ / /_/ /__/ / / / __/ /
/_/ \__,_/\___/_/ |__/|__/\__,_/\__/\___/_/ /_/\___/_/
Fuelwatcher
A simple python module that scrapes XML data from the government of Western Australia's FuelWatch website that makes parsing a breeze.
Fuelwatch.wa.gov.au provides information on fuel prices by fuel type, location, brand and region within Western Australia. Fuelwatcher will parse the XML from the fuelwatch.wa.gov.au RSS feed giving the developer an easy way to manipulate the information.
Installation
Requires pip
to be installed or pip3
dependent on system, or environment.
pip install fuelwatcher
Usage example
Basic Usage
from fuelwatcher import FuelWatch
api = FuelWatch()
# returns byte string of xml.
api.query(product=2, region=25, day='yesterday')
# iterates over each fuel station entry in the byte string
# and returns list of dictionaries in human-readable text
xml_query = api.get_xml
print(xml_query)
>>>> [{'title': '138.5: Puma Bayswater', 'description': 'Address: 502 Guildford Rd, BAYSWATER, Phone: (08) 9379 1322, Open 24 hours', 'brand': 'Puma', 'date': '2018-04-05', 'price': '138.5', 'trading-name': 'Puma Bayswater', 'location': 'BAYSWATER', 'address': '502 Guildford Rd', 'phone': '(08) 9379 1322', 'latitude': '-31.919556', 'longitude': '115.929069', 'site-features': ', Open 24 hours'} ..snip... '}]
# python dictionary parsing
print(xml_query[0]['title'])
>>>> '138.5: Puma Bayswater'
Fuelwatcher can also transform the XML into JSON format. It is as simple as calling the get_json
method.
api = FuelWatch()
api.query(region=1)
json_response = api.get_json
>>>> [
>>>> {
>>>> "title": "143.9: United Boulder Kalgoorlie",
>>>> "description": "Address: Cnr Lane St & Davis St, BOULDER, Phone: (08) 9093 1543",
>>>> "brand": "United",
>>>> "date": "2018-04-13",
>>>> "price": "143.9",
>>>> ... snip ...
>>>> "longitude": "121.433746",
>>>> "site-features": "Unmanned Station, "
>>>> }
>>>> ]
For most operations the get_xml()
or get_json()
method will be sufficient. If the developer wants to parse the raw RSS XML then the get_raw()
method is available.
This will return bytes.
get_raw = api.get_raw
print(get_raw)
>>>> (b'<?xml version="1.0" encoding="UTF-8"?>\r\n<rss version="2.0"><channel><title>FuelWatch Prices For North of River</title><ttl>720</ttl><link>http://www.fuelwatch.wa.gov.au</link><description>05/04/2018 - North of River</description><language>en-us</language><copyright>Copyright 2005 FuelWatch... snip...</item></channel></rss>\r\n')
print(type(get_raw))
>>>> <class 'bytes'>
The query method takes several keyword arguments. A query without any arguments will return all of today's Unleaded stations in Western Australia.
As guide query takes the following kwargs
def query(self, product: int = None, suburb: str = None, region: int = None,
brand: int = None, surrounding: str = None, day: str = None):
Note
If suburb
is set then surrounding
will default to yes
. To get only the suburb, and not surrounding areas an explicit surrounding='no'
must be called.
Setting region
with suburb
and surrounding
will have unexpected results and are best not mixed together.
Simply put, if you want just one suburb
then set surrounding='no'
, else leave the default. Only one suburb
can be set per query. If a region
is selected, do not set surrounding
or suburb
.
A list of valid suburbs, brands, regions and products (fuel types) can be found in constants.py
Fuelwatcher will run validation on the query
method and throw AssertionError if an invalid integer or string is input
api.query(product=20) # product=20 is invalid
>>> .... error snippet....
>>> AssertionError: Invalid Product Integer.
Release History
- 0.2.1
- README.md updated
- 0.2.0
- Breaking Change!
- @property added raw, xml and json methods
- json output now supported
- 0.1.1
- Include correct packages in setup.py
- 0.1.0
- First release live to PyPi
- 0.1.0rc2
- Minor formatting fixes
- 0.1.0rc1
- The first release candidate
- 0.0.1
- Work in progress
Meta
Daniel Michaels – https://www.danielms.site
Distributed under the MIT license. See LICENSE
for more information.
Contributing
All requests, ideas or improvements are welcomed!
- Fork it
- Create your feature branch (
git checkout -b feature/fooBar
) - Commit your changes (
git commit -am 'Add some fooBar'
) - Push to the branch (
git push origin feature/fooBar
) - Create a new Pull Request
Inspired by..
A local python meetup group idea that turned into a PyPi package for anyone to use!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for fuelwatcher-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9a3130b4256d7f04a3ea2ae682fbf695999984323b8722492a85ef8e2b33e2f7 |
|
MD5 | 258760235c3b876430018c5c7815aa8b |
|
BLAKE2b-256 | 2bbb56bc1d891085dd8a2b70167431aecad3749568325c1db33a4e55b45003ef |