Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Combine XPath, CSS Selectors and JSONPath for Web data extracting.

Project description

license Pypi Status Python version Package version PyPI - Downloads GitHub last commit Code style: black Build Status codecov Documentation Status

Combine XPath, CSS Selectors and JSONPath for Web data extracting.

Quickstarts

Installation

Install the stable version from PYPI.

pip install data-extractor

Or install the latest version from Github.

pip install git+https://github.com/linw1995/data_extractor.git@master

Usage

from data_extractor import Field, Item, JSONExtractor


class Count(Item):
    followings = Field(JSONExtractor("countFollowings"))
    fans = Field(JSONExtractor("countFans"))


class User(Item):
    name_ = Field(JSONExtractor("name"), name="name")
    age = Field(JSONExtractor("age"), default=17)
    count = Count()


assert User(JSONExtractor("data.users[*]"), is_many=True).extract(
    {
        "data": {
            "users": [
                {
                    "name": "john",
                    "age": 19,
                    "countFollowings": 14,
                    "countFans": 212,
                },
                {
                    "name": "jack",
                    "description": "",
                    "countFollowings": 54,
                    "countFans": 312,
                },
            ]
        }
    }
) == [
    {"name": "john", "age": 19, "count": {"followings": 14, "fans": 212}},
    {"name": "jack", "age": 17, "count": {"followings": 54, "fans": 312}},
]

Changelog

v0.5.3

  • 6a26be5 Chg:Wrap the single return value as a list
  • 0b63927 Fix:Item can not extract the data is list type
  • 9deeb5f Chg:Update poetry.lock

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for data-extractor, version 0.5.3
Filename, size File type Python version Upload date Hashes
Filename, size data_extractor-0.5.3-py3-none-any.whl (12.7 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size data_extractor-0.5.3.tar.gz (10.9 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page