data-extractor 0.5.4

Combine XPath, CSS Selectors and JSONPath for Web data extracting.

Combine XPath, CSS Selectors and JSONPath for Web data extracting.

Quickstarts

Installation

Install the stable version from PYPI.

pip install data-extractor

Or install the latest version from Github.

pip install git+https://github.com/linw1995/data_extractor.git@master

Usage

from data_extractor import Field, Item, JSONExtractor


class Count(Item):
    followings = Field(JSONExtractor("countFollowings"))
    fans = Field(JSONExtractor("countFans"))


class User(Item):
    name_ = Field(JSONExtractor("name"), name="name")
    age = Field(JSONExtractor("age"), default=17)
    count = Count()


assert User(JSONExtractor("data.users[*]"), is_many=True).extract(
    {
        "data": {
            "users": [
                {
                    "name": "john",
                    "age": 19,
                    "countFollowings": 14,
                    "countFans": 212,
                },
                {
                    "name": "jack",
                    "description": "",
                    "countFollowings": 54,
                    "countFans": 312,
                },
            ]
        }
    }
) == [
    {"name": "john", "age": 19, "count": {"followings": 14, "fans": 212}},
    {"name": "jack", "age": 17, "count": {"followings": 54, "fans": 312}},
]

Changelog

v0.5.4

• 9552c79 Fix:Simplified item’s extract_first method fail to raise ExtractError

• 08167ab Fix:Simplified item’s extract_first method should support param default

• 6e4c269 New:More unittest for testing the simplified items

• a35b85a Chg:Update poetry.lock

• e5ff37b Docs,Chg:Update travis-ci status source in the README.rst