Key Features
A web scraping framework used to crawl web pages
Data extraction tools used to extract structured data from web pages
Spider Example
以下是我们的一个爬虫类示例，其作用为爬取 百度新闻 的热点要闻:
from xpaw import Spider, HttpRequest, Selector, run_spider
class BaiduNewsSpider(Spider):
def start_requests(self):
yield HttpRequest("http://news.baidu.com/", callback=self.parse)
def parse(self, response):
selector = Selector(response.text)
hot = selector.css("div.hotnews a").text
self.log("Hot News:")
for i in range(len(hot)):
self.log("%s: %s", i + 1, hot[i])
if __name__ == '__main__':
run_spider(BaiduNewsSpider)在爬虫类中我们定义了一些方法：
start_requests: 返回爬虫初始请求。
parse: 处理请求得到的页面，这里借助 Selector 及CSS Selector语法提取到了我们所需的数据。
Documentation
1 maintainer
Release files for xpaw 0.12.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| xpaw-0.12.0.tar.gz | 172.2 kB | Details |
Release files / xpaw-0.12.0.tar.gz
| Download URL | xpaw-0.12.0.tar.gz |
|---|---|
| Size | 172.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e2fb006259034f7ccc29e5c771b7c68ce796fc58426f546c4ea1d476f28ddbbf
|
|
BLAKE2b-256 checksum How to use checksums |
09890bc209cafb6e68c97ce01c6eb27954abb024c3804c2b264cd445e6926e37
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
Python-urllib/3.6
|
