A PCPartPicker data extractor for Python.
Project description
pypartpicker
A PCPartPicker data extractor for Python.
Features:
- Fetch product information, specs, pricing and reviews
- Fetch part lists
- Utilise PCPartPicker's built in search functionality
- Scraping countermeasures out of the box via requests-html
- Support for all regions
- Customisable scraping
Table of Contents
Installation
$ pip install pypartpicker
Note
Due to pyppeteer your first use of the library may install a chromium browser for JS rendering.
This is only done once. If you would like to disable this feature entirely, use the no_js=True option in the Client constructor.
Examples
Fetch a product:
import pypartpicker
pcpp = pypartpicker.Client()
part = pcpp.get_part("https://pcpartpicker.com/product/fN88TW")
for spec, value in part.specs.items():
print(f"{spec}: {value}")
print(part.cheapest_price)
Search parts with pagination:
import pypartpicker
pcpp = pypartpicker.Client()
page = 1
while True:
result = pcpp.get_part_search("ryzen 5", region="uk", page=page)
for part in result.parts:
print(part.name)
page += 1
if page > result.total_pages:
break
Fetch a product (async):
import pypartpicker
import asyncio
async def get_parts():
async with pypartpicker.AsyncClient() as pcpp:
part = await pcpp.get_part("https://pcpartpicker.com/product/fN88TW")
for spec, value in part.specs.items():
print(f"{spec}: {value}")
asyncio.run(get_parts())
Proxy rotation w/ response_retriever override:
import pypartpicker
import requests_html
from itertools import cycle
# replace with own list of proxies
list_proxy = [
"socks5://Username:Password@IP1:20000",
"socks5://Username:Password@IP2:20000",
"socks5://Username:Password@IP3:20000",
"socks5://Username:Password@IP4:20000",
]
proxy_cycle = cycle(list_proxy)
session = requests_html.HTMLSession()
def response_retriever(url):
proxy = next(proxy_cycle)
return session.get(url, proxies={"http": proxy, "https": proxy})
client = pypartpicker.Client(response_retriever=response_retriever)
res = client.get_part_search("cpu")
for result in res.parts:
part = client.get_part(result.url)
print(part.specs)
Documentation
Client
Represents a client for interacting with parts-related data and making HTTP requests.
Options
max_retries:int– The maximum number of retries for requests. Default is3.retry_delay:int– The delay between retries in seconds. Default is0.cookies:Optional[dict]– Cookies to include in requests.response_retriever:Optional[Callable]– A custom function to perform a request, overriding the default one. Can be used to implement proxy rotation and custom scraping measures.no_js:bool– Disables pyppeteer JS rendering. Default isFalse.
Methods
get_part(id_url: str, region: str = None) -> Part
Fetches a single part by its URL/ID and region.
-
Parameters:
id_url:str– The part ID or URL of the part to retrieve.region:Optional[str]– The region for the part data.
-
Returns:
Part– The part details.
get_part_list(id_url: str, region: str = None) -> PartList
Fetches a part list by its URL/ID and region.
-
Parameters:
id_url:str– The part list ID or URL of the part list to retrieve.region:Optional[str]– The region for the part list data.
-
Returns:
PartList– The part list details.
get_part_search(query: str, page: int = 1, region: Optional[str] = None) -> PartSearchResult
Searches for parts using PCPartPicker's search functionality.
-
Parameters:
query:str– The search query string.page:int– The page number to fetch. Default is1.region:Optional[str]– The region for the search results.
-
Returns:
PartSearchResult– The search results for parts.
get_part_reviews(id_url: str, page: int = 1, rating: Optional[int] = None) -> PartReviewsResult
Fetches reviews for a specific part.
-
Parameters:
id_url:str– The part ID or URL of the part to retrieve reviews for.page:int– The page number to fetch. Default is1.rating:Optional[int]– Filter reviews by a specific star rating.
-
Returns:
PartReviewsResult– The reviews for the specified part.
Exceptions
CloudflareException– Raised when the request fails due to Cloudflare protection after the maximum retries.RateLimitException– Raised when the request encounters a PCPartPicker rate limit issue.
AsyncClient
Same methods and options as Client except called with await.
Types
Price
Represents the pricing details of a product.
base:Optional[float]– The base price of the item.discounts:Optional[float]– Any discounts applied to the item.shipping:Optional[float]– Shipping costs associated with the item.tax:Optional[float]– Taxes applied to the item.total:Optional[float]– The total price after applying all factors.currency:Optional[str]– The currency of the price.
Vendor
Represents a vendor offering a product.
name:str– The name of the vendor.logo_url:str– The URL to the vendor's logo image.in_stock:bool– Whether the product is in stock.price:Price– The price details for the product.buy_url:str– The URL to purchase the product from the vendor.
Rating
Represents the rating of a product.
stars:int– The number of stars given by reviewers.count:int– The total number of ratings received.average:float– The average rating value.
User
Represents a user who interacts with reviews.
username:str– The username of the user.avatar_url:str– The URL to the user's avatar image.profile_url:str– The URL to the user's profile.
Review
Represents a review for a product.
author:User– The user who wrote the review.points:int– The number of points given to the review.stars:int– The star rating given in the review.created_at:str– The timestamp when the review was created.content:str– The textual content of the review.build_name:Optional[str]– The name of the build associated with the review.build_url:Optional[str]– The URL to the build associated with the review.
PartReviewsResult
Represents the result of a paginated query for part reviews.
reviews:listofReview– A list of reviews for a product.page:int– The current page of results.total_pages:int– The total number of pages available.
Part
Represents an individual part of a system or build.
name:str– The name of the part.type:str– The type or category of the part.image_urls:Optional[list[str]]– A list of URLs to images of the part.url:Optional[str]– The URL for more details about the part.cheapest_price:OptionalofPrice– The cheapest price for the part.in_stock:Optional[bool]– Whether the part is currently in stock.vendors:Optional[listofVendor]– A list of vendors offering the part.rating:OptionalofRating– The rating details for the part.specs:Optional[dict[str, str]]– A dictionary of specifications for the part.reviews:Optional[listofReview]– A list of reviews for the part.
PartList
Represents a list of parts for a system or build.
parts:listofPart– A list of parts in the build.url:str– The URL for the part list.estimated_wattage:float– The power consumption of the build, measured in watts.total_price:float– The total price of the build.currency:str– The currency used for pricing.
PartSearchResult
Represents the result of a paginated query for parts.
parts:listofPart– A list of parts matching the search query.page:int– The current page of results.total_pages:int– The total number of pages available.
Supported Regions
- Australia:
au - Austria:
at - Belgium:
be - Canada:
ca - Czech Republic:
cz - Denmark:
dk - Finland:
fi - France:
fr - Germany:
de - Hungary:
hu - Ireland:
ie - Italy:
it - Netherlands:
nl - New Zealand:
nz - Norway:
no - Portugal:
pt - Romania:
ro - Saudi Arabia:
sa - Slovakia:
sk - Spain:
es - Sweden:
se - United Kingdom:
uk - United States:
us
Supported Product Types
PRODUCT_KEYBOARD_PATH = "keyboard"
PRODUCT_SPEAKERS_PATH = "speakers"
PRODUCT_MONITOR_PATH = "monitor"
PRODUCT_THERMAL_PASTE_PATH = "thermal-paste"
PRODUCT_VIDEO_CARD_PATH = "video-card"
PRODUCT_CASE_FAN_PATH = "case-fan"
PRODUCT_OS_PATH = "os"
PRODUCT_CPU_COOLER_PATH = "cpu-cooler"
PRODUCT_FAN_CONTROLLER_PATH = "fan-controller"
PRODUCT_UPS_PATH = "ups"
PRODUCT_WIRED_NETWORK_CARD_PATH = "wired-network-card"
PRODUCT_MEMORY_PATH = "memory"
PRODUCT_HEADPHONES_PATH = "headphones"
PRODUCT_SOUND_CARD_PATH = "sound-card"
PRODUCT_INTERNAL_HARD_DRIVE_PATH = "internal-hard-drive"
PRODUCT_MOUSE_PATH = "mouse"
PRODUCT_WIRELESS_NETWORK_CARD_PATH = "wireless-network-card"
PRODUCT_POWER_SUPPLY_PATH = "power-supply"
PRODUCT_WEBCAM_PATH = "webcam"
PRODUCT_MOTHERBOARD_PATH = "motherboard"
PRODUCT_EXTERNAL_HARD_DRIVE_PATH = "external-hard-drive"
PRODUCT_OPTICAL_DRIVE_PATH = "optical-drive"
PRODUCT_CASE_PATH = "case"
PRODUCT_CPU_PATH = "cpu"
FAQs
Chromium Errors
If [INFO]: Downloading Chromium errors are encountered, find your __init__.py file located in C:\Users\yourusername\AppData\Local\Programs\Python\Python3XX\Lib\site-packages\pyppeteer, and edit line 20 from __chromium_revision__ = '1181205' to __chromium_revision__ = '1263111'
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file pypartpicker-2.0.5.tar.gz.
File metadata
- Download URL: pypartpicker-2.0.5.tar.gz
- Upload date:
- Size: 12.6 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.5 CPython/3.10.7 Windows/10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
27c30e50d0f581bdef82c3966901ae9cacc15c9f0748a480dc70a56feb936bde
|
|
| MD5 |
02f9879034127380196b587f884a27cc
|
|
| BLAKE2b-256 |
53cc6f459cb48534601babfa757d9179b3e6402f81b7ad6fa4477505667670ba
|
File details
Details for the file pypartpicker-2.0.5-py3-none-any.whl.
File metadata
- Download URL: pypartpicker-2.0.5-py3-none-any.whl
- Upload date:
- Size: 13.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.8.5 CPython/3.10.7 Windows/10
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
738fbec9f0dc1226fd6926694b8f189f20bd794d2473765aa4205e7f6015b2c3
|
|
| MD5 |
5ab8800f8450160e9500d34346da108b
|
|
| BLAKE2b-256 |
061ce0e2a251bebc426014180b67350bdae4f0c1e673a711387668e55c7b70fb
|