A webscraper for Daft.ie
Project description
daft-scraper
A python library for scraping housing data from Daft.ie.
Inspired by the wonderful daftlistings library.
Install
Via Pip
You can install the library using pip:
pip install daft-scraper
Via Git
The project uses poetry, so you'll need poetry to install the dependencies and setup the project.
git clone git@github.com:TheJokersThief/daft-scraper.git
cd daft-scraper
make install
Real-life Usage
- Daft2BigQuery: A project that pulls housing information from Daft and puts it in GCP BigQuery for data modelling.
Example Usage
from daft_scraper.search import DaftSearch, SearchType
from daft_scraper.search.options import (
PropertyType, PropertyTypesOption, Facility, FacilitiesOption,
PriceOption, BedOption
)
from daft_scraper.search.options_location import LocationsOption, Location
options = [
PropertyTypesOption([PropertyType.APARTMENT]),
FacilitiesOption([Facility.PARKING, Facility.SERVICED_PROPERTY]),
LocationsOption([Location.SWORDS_DUBLIN]),
PriceOption(0, 999999),
BedOption(1, 4),
]
api = DaftSearch(SearchType.RENT)
listings = api.search(options)
for listing in listings:
print(listing.title)
Using the CLI
The CLI is included as an easy way for me test things and get some quick results :) Let me know if you have any suggestions!
To install the CLI, clone the repo and install the dependencies with make install
.
$ poetry run daft search --max-pages 1 property-for-rent --location cork --location galway
id price title propertyType url
------- ------- ----------------------------------------------------------------------- -------------- --------------------------------------------------------------------------------------------------
2315059 3328 The Elysian, Eglinton Road, Co. Cork Apartments https://daft.ie/for-rent/the-elysian-eglinton-road-co-cork/2315059
2588837 570 Parchment Square, Model Farm Road, Cork, Co. Cork Apartments https://daft.ie/for-rent/parchment-square-model-farm-road-cork-co-cork/2588837
2310295 759.5 Nido Curraheen Point, Farranlea Road, Co. Cork Apartments https://daft.ie/for-rent/nido-curraheen-point-farranlea-road-co-cork/2310295
2292251 954.8 From Here - Student Living, Galway Central, Fairgreen Road, Co. Galway Apartments https://daft.ie/for-rent/from-here-student-living-galway-central-fairgreen-road-co-galway/2292251
2590894 495 BUNK CO LIVING, Kiltartan house Forster Street, Co. Galway Apartments https://daft.ie/for-rent/bunk-co-living-kiltartan-house-forster-street-co-galway/2590894
2575994 650 Steelworks, 9/10 Copley Street, Ballintemple, Cork City, Cork, Co. Cork Apartments https://daft.ie/for-rent/steelworks-9-10-copley-street-ballintemple-cork-city-cork-co-cork/2575994
2327420 1028.58 Lee Point, South Main Street, Co. Cork Apartments https://daft.ie/for-rent/lee-point-south-main-street-co-cork/2327420
2751036 2400 16A The Long Walk, Co. Galway House https://daft.ie/for-rent/house-16a-the-long-walk-co-galway/2751036
2745585 1588 Wellington Road, Co. Cork Apartment https://daft.ie/for-rent/apartment-wellington-road-co-cork/2745585
2626561 2800 3 Saint Joseph's Terrace, Gould Street, Co. Cork House https://daft.ie/for-rent/house-3-saint-josephs-terrace-gould-street-co-cork/2626561
2737101 1800 CHURCHFIELDS SALTHILL, Salthill, Co. Galway House https://daft.ie/for-rent/house-churchfields-salthill-salthill-co-galway/2737101
2759058 1400 24 Rutland Place, South Terrace, Co. Cork Apartment https://daft.ie/for-rent/apartment-24-rutland-place-south-terrace-co-cork/2759058
2629695 1750 56 Caiseal Cam, Roscam, Co. Galway House https://daft.ie/for-rent/house-56-caiseal-cam-roscam-co-galway/2629695
2737848 1500 Dark Rd, Kilcolgan, Co. Galway House https://daft.ie/for-rent/house-dark-rd-kilcolgan-co-galway/2737848
2758935 1200 Hollyville, Turners Cross, Co. Cork House https://daft.ie/for-rent/house-hollyville-turners-cross-co-cork/2758935
2737834 1800 11 Shangort Park, Knocknacarra, Co. Galway House https://daft.ie/for-rent/house-11-shangort-park-knocknacarra-co-galway/2737834
2757337 950 Apartment 3, 13 Harbour Row, Cobh, Co. Cork House https://daft.ie/for-rent/house-apartment-3-13-harbour-row-cobh-co-cork/2757337
2756288 4500 Meizelljob, Coast Road, Fountainstown, Co. Cork House https://daft.ie/for-rent/house-meizelljob-coast-road-fountainstown-co-cork/2756288
2756231 1500 Garrai De Brun, Fort Lorenzo, Taylor's Hill, Co. Galway House https://daft.ie/for-rent/house-garrai-de-brun-fort-lorenzo-taylors-hill-co-galway/2756231
2632714 1650 3 Bothar An tSléibhe, Moycullen, Co. Galway House https://daft.ie/for-rent/house-3-bothar-an-tsl-ibhe-moycullen-co-galway/2632714
search
command
argument | description |
---|---|
search_type | The type of search you want to initiate. For the possible values, check out the SearchType Enum. |
For any flag that can take [multiple]
arguments, you can supply the flag multiple times.
flag | description |
---|---|
--headers | The attributes to print out for each listing. [multiple] |
--location | Which location you want to search for. For all the possible values, check out the Location Enum [multiple] |
--max-pages | Each page is 20 results, this sets the limit on the number of pages fetched. |
--min-price | Minimum price. |
--max-price | Maximum price. |
--min-beds | Minimum number of bedrooms. |
--max-beds | Maximum number of bedrooms. |
--min-lease | Minimum term on the lease (in months). |
--max-lease | Maximum term on the lease (in months). |
--property-type | The type of property to search for. For all possible values, checkout the PropertyType Enum |
--facility | Which facilities must the listing include. [multiple] |
--media-type | Which media types must the listing include. [multiple] |
--sort | How should the results be sorted. For all possible views, check out the Sort Enum. |
--furnishing | Should the listing be furnished or unfurnished. |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file daft_scraper-1.3.1.tar.gz
.
File metadata
- Download URL: daft_scraper-1.3.1.tar.gz
- Upload date:
- Size: 60.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.12.6 Darwin/24.0.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 54535e48c300df4dfbadcb60b354cda18aad8411e7e3ef15b3fb58ee51cd304a |
|
MD5 | 5abee975da2f78491e666cd9f4c888a9 |
|
BLAKE2b-256 | 6d7ce714bf7cf6a18e4d066c4df77c6207e7105e1332738535013c2ec9f4f03b |
File details
Details for the file daft_scraper-1.3.1-py3-none-any.whl
.
File metadata
- Download URL: daft_scraper-1.3.1-py3-none-any.whl
- Upload date:
- Size: 59.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.3 CPython/3.12.6 Darwin/24.0.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6792ec9e007ba0c0291d49cc55603f816209d6453143b5ffb70bc1d9156be3f |
|
MD5 | bf7df6f24bbf5ee6cceb4364aabc987f |
|
BLAKE2b-256 | 6238475783c6220a483765d93087b71094b1fe33faf091979db66d7225b4eefa |