Skip to main content

A webscraper for Daft.ie

Project description

daft-scraper

A python library for scraping housing data from Daft.ie.

Inspired by the wonderful daftlistings library.

TheJokersThief

Install

Via Pip

You can install the library using pip:

pip install daft-scraper

Via Git

The project uses poetry, so you'll need poetry to install the dependencies and setup the project.

git clone git@github.com:TheJokersThief/daft-scraper.git
cd daft-scraper
make install

Real-life Usage

  • Daft2BigQuery: A project that pulls housing information from Daft and puts it in GCP BigQuery for data modelling.

Example Usage

from daft_scraper.search import DaftSearch, SearchType
from daft_scraper.search.options import (
    PropertyType, PropertyTypesOption, Facility, FacilitiesOption,
    PriceOption, BedOption
)
from daft_scraper.search.options_location import LocationsOption, Location

options = [
    PropertyTypesOption([PropertyType.APARTMENT]),
    FacilitiesOption([Facility.PARKING, Facility.SERVICED_PROPERTY]),
    LocationsOption([Location.SWORDS_DUBLIN]),
    PriceOption(0, 999999),
    BedOption(1, 4),
]

api = DaftSearch(SearchType.RENT)
listings = api.search(options)

for listing in listings:
    print(listing.title)

Using the CLI

The CLI is included as an easy way for me test things and get some quick results :) Let me know if you have any suggestions!

To install the CLI, clone the repo and install the dependencies with make install.

$ poetry run daft search --max-pages 1 property-for-rent --location cork --location galway
     id    price  title                                                                    propertyType    url
-------  -------  -----------------------------------------------------------------------  --------------  --------------------------------------------------------------------------------------------------
2315059  3328     The Elysian, Eglinton Road, Co. Cork                                     Apartments      https://daft.ie/for-rent/the-elysian-eglinton-road-co-cork/2315059
2588837   570     Parchment Square, Model Farm Road, Cork, Co. Cork                        Apartments      https://daft.ie/for-rent/parchment-square-model-farm-road-cork-co-cork/2588837
2310295   759.5   Nido Curraheen Point, Farranlea Road, Co. Cork                           Apartments      https://daft.ie/for-rent/nido-curraheen-point-farranlea-road-co-cork/2310295
2292251   954.8   From Here - Student Living, Galway Central, Fairgreen Road, Co. Galway   Apartments      https://daft.ie/for-rent/from-here-student-living-galway-central-fairgreen-road-co-galway/2292251
2590894   495     BUNK CO LIVING, Kiltartan house Forster Street, Co. Galway               Apartments      https://daft.ie/for-rent/bunk-co-living-kiltartan-house-forster-street-co-galway/2590894
2575994   650     Steelworks, 9/10 Copley Street, Ballintemple, Cork City, Cork, Co. Cork  Apartments      https://daft.ie/for-rent/steelworks-9-10-copley-street-ballintemple-cork-city-cork-co-cork/2575994
2327420  1028.58  Lee Point, South Main Street, Co. Cork                                   Apartments      https://daft.ie/for-rent/lee-point-south-main-street-co-cork/2327420
2751036  2400     16A The Long Walk, Co. Galway                                            House           https://daft.ie/for-rent/house-16a-the-long-walk-co-galway/2751036
2745585  1588     Wellington Road, Co. Cork                                                Apartment       https://daft.ie/for-rent/apartment-wellington-road-co-cork/2745585
2626561  2800     3 Saint Joseph's Terrace, Gould Street, Co. Cork                         House           https://daft.ie/for-rent/house-3-saint-josephs-terrace-gould-street-co-cork/2626561
2737101  1800     CHURCHFIELDS SALTHILL, Salthill, Co. Galway                              House           https://daft.ie/for-rent/house-churchfields-salthill-salthill-co-galway/2737101
2759058  1400     24 Rutland Place, South Terrace, Co. Cork                                Apartment       https://daft.ie/for-rent/apartment-24-rutland-place-south-terrace-co-cork/2759058
2629695  1750     56 Caiseal Cam, Roscam, Co. Galway                                       House           https://daft.ie/for-rent/house-56-caiseal-cam-roscam-co-galway/2629695
2737848  1500     Dark Rd, Kilcolgan, Co. Galway                                           House           https://daft.ie/for-rent/house-dark-rd-kilcolgan-co-galway/2737848
2758935  1200     Hollyville, Turners Cross, Co. Cork                                      House           https://daft.ie/for-rent/house-hollyville-turners-cross-co-cork/2758935
2737834  1800     11 Shangort Park, Knocknacarra, Co. Galway                               House           https://daft.ie/for-rent/house-11-shangort-park-knocknacarra-co-galway/2737834
2757337   950     Apartment 3, 13 Harbour Row, Cobh, Co. Cork                              House           https://daft.ie/for-rent/house-apartment-3-13-harbour-row-cobh-co-cork/2757337
2756288  4500     Meizelljob, Coast Road, Fountainstown, Co. Cork                          House           https://daft.ie/for-rent/house-meizelljob-coast-road-fountainstown-co-cork/2756288
2756231  1500     Garrai De Brun, Fort Lorenzo, Taylor's Hill, Co. Galway                  House           https://daft.ie/for-rent/house-garrai-de-brun-fort-lorenzo-taylors-hill-co-galway/2756231
2632714  1650     3 Bothar An tSléibhe, Moycullen, Co. Galway                             House           https://daft.ie/for-rent/house-3-bothar-an-tsl-ibhe-moycullen-co-galway/2632714

search command

argument description
search_type The type of search you want to initiate. For the possible values, check out the SearchType Enum.

For any flag that can take [multiple] arguments, you can supply the flag multiple times.

flag description
--headers The attributes to print out for each listing. [multiple]
--location Which location you want to search for. For all the possible values, check out the Location Enum [multiple]
--max-pages Each page is 20 results, this sets the limit on the number of pages fetched.
--min-price Minimum price.
--max-price Maximum price.
--min-beds Minimum number of bedrooms.
--max-beds Maximum number of bedrooms.
--min-lease Minimum term on the lease (in months).
--max-lease Maximum term on the lease (in months).
--property-type The type of property to search for. For all possible values, checkout the PropertyType Enum
--facility Which facilities must the listing include. [multiple]
--media-type Which media types must the listing include. [multiple]
--sort How should the results be sorted. For all possible views, check out the Sort Enum.
--furnishing Should the listing be furnished or unfurnished.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

daft_scraper-1.3.1.tar.gz (60.5 kB view details)

Uploaded Source

Built Distribution

daft_scraper-1.3.1-py3-none-any.whl (59.6 kB view details)

Uploaded Python 3

File details

Details for the file daft_scraper-1.3.1.tar.gz.

File metadata

  • Download URL: daft_scraper-1.3.1.tar.gz
  • Upload date:
  • Size: 60.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.12.6 Darwin/24.0.0

File hashes

Hashes for daft_scraper-1.3.1.tar.gz
Algorithm Hash digest
SHA256 54535e48c300df4dfbadcb60b354cda18aad8411e7e3ef15b3fb58ee51cd304a
MD5 5abee975da2f78491e666cd9f4c888a9
BLAKE2b-256 6d7ce714bf7cf6a18e4d066c4df77c6207e7105e1332738535013c2ec9f4f03b

See more details on using hashes here.

File details

Details for the file daft_scraper-1.3.1-py3-none-any.whl.

File metadata

  • Download URL: daft_scraper-1.3.1-py3-none-any.whl
  • Upload date:
  • Size: 59.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.3 CPython/3.12.6 Darwin/24.0.0

File hashes

Hashes for daft_scraper-1.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 c6792ec9e007ba0c0291d49cc55603f816209d6453143b5ffb70bc1d9156be3f
MD5 bf7df6f24bbf5ee6cceb4364aabc987f
BLAKE2b-256 6238475783c6220a483765d93087b71094b1fe33faf091979db66d7225b4eefa

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page