Skip to main content

Parsel

Build Status PyPI Version Coverage report

Parsel is a BSD-licensed Python library to extract and remove data from HTML and XML using XPath and CSS selectors, optionally combined with regular expressions.

Find the Parsel online documentation at https://parsel.readthedocs.org.

Example (open online demo):

>>> from parsel import Selector
>>> selector = Selector(text=u"""<html>
        <body>
            <h1>Hello, Parsel!</h1>
            <ul>
                <li><a href="http://example.com">Link 1</a></li>
                <li><a href="http://scrapy.org">Link 2</a></li>
            </ul>
        </body>
        </html>""")
>>> selector.css('h1::text').get()
'Hello, Parsel!'
>>> selector.xpath('//h1/text()').re(r'\w+')
['Hello', 'Parsel']
>>> for li in selector.css('ul > li'):
...     print(li.xpath('.//@href').get())
http://example.com
http://scrapy.org

History

1.6.0 (2020-05-07)

  • Python 3.4 is no longer supported

  • New Selector.remove() and SelectorList.remove() methods to remove selected elements from the parsed document tree

  • Improvements to error reporting, test coverage and documentation, and code cleanup

1.5.2 (2019-08-09)

  • Selector.remove_namespaces received a significant performance improvement

  • The value of data within the printable representation of a selector (repr(selector)) now ends in ... when truncated, to make the truncation obvious.

  • Minor documentation improvements.

1.5.1 (2018-10-25)

  • has-class XPath function handles newlines and other separators in class names properly;

  • fixed parsing of HTML documents with null bytes;

  • documentation improvements;

  • Python 3.7 tests are run on CI; other test improvements.

1.5.0 (2018-07-04)

  • New Selector.attrib and SelectorList.attrib properties which make it easier to get attributes of HTML elements.

  • CSS selectors became faster: compilation results are cached (LRU cache is used for css2xpath), so there is less overhead when the same CSS expression is used several times.

  • .get() and .getall() selector methods are documented and recommended over .extract_first() and .extract().

  • Various documentation tweaks and improvements.

One more change is that .extract() and .extract_first() methods are now implemented using .get() and .getall(), not the other way around, and instead of calling Selector.extract all other methods now call Selector.get internally. It can be backwards incompatible in case of custom Selector subclasses which override Selector.extract without doing the same for Selector.get. If you have such Selector subclass, make sure get method is also overridden. For example, this:

class MySelector(parsel.Selector):
    def extract(self):
        return super().extract() + " foo"

should be changed to this:

class MySelector(parsel.Selector):
    def get(self):
        return super().get() + " foo"
    extract = get

1.4.0 (2018-02-08)

  • Selector and SelectorList can’t be pickled because pickling/unpickling doesn’t work for lxml.html.HtmlElement; parsel now raises TypeError explicitly instead of allowing pickle to silently produce wrong output. This is technically backwards-incompatible if you’re using Python < 3.6.

1.3.1 (2017-12-28)

  • Fix artifact uploads to pypi.

1.3.0 (2017-12-28)

  • has-class XPath extension function;

  • parsel.xpathfuncs.set_xpathfunc is a simplified way to register XPath extensions;

  • Selector.remove_namespaces now removes namespace declarations;

  • Python 3.3 support is dropped;

  • make htmlview command for easier Parsel docs development.

  • CI: PyPy installation is fixed; parsel now runs tests for PyPy3 as well.

1.2.0 (2017-05-17)

  • Add SelectorList.get and SelectorList.getall methods as aliases for SelectorList.extract_first and SelectorList.extract respectively

  • Add default value parameter to SelectorList.re_first method

  • Add Selector.re_first method

  • Add replace_entities argument on .re() and .re_first() to turn off replacing of character entity references

  • Bug fix: detect None result from lxml parsing and fallback with an empty document

  • Rearrange XML/HTML examples in the selectors usage docs

  • Travis CI:

    • Test against Python 3.6

    • Test against PyPy using “Portable PyPy for Linux” distribution

1.1.0 (2016-11-22)

  • Change default HTML parser to lxml.html.HTMLParser, which makes easier to use some HTML specific features

  • Add css2xpath function to translate CSS to XPath

  • Add support for ad-hoc namespaces declarations

  • Add support for XPath variables

  • Documentation improvements and updates

1.0.3 (2016-07-29)

  • Add BSD-3-Clause license file

  • Re-enable PyPy tests

  • Integrate py.test runs with setuptools (needed for Debian packaging)

  • Changelog is now called NEWS

1.0.2 (2016-04-26)

  • Fix bug in exception handling causing original traceback to be lost

  • Added docstrings and other doc fixes

1.0.1 (2015-08-24)

  • Updated PyPI classifiers

  • Added docstrings for csstranslator module and other doc fixes

1.0.0 (2015-08-22)

  • Documentation fixes

0.9.6 (2015-08-14)

  • Updated documentation

  • Extended test coverage

0.9.5 (2015-08-11)

  • Support for extending SelectorList

0.9.4 (2015-08-10)

  • Try workaround for travis-ci/dpl#253

0.9.3 (2015-08-07)

  • Add base_url argument

0.9.2 (2015-08-07)

  • Rename module unified -> selector and promoted root attribute

  • Add create_root_node function

0.9.1 (2015-08-04)

  • Setup Sphinx build and docs structure

  • Build universal wheels

  • Rename some leftovers from package extraction

0.9.0 (2015-07-30)

  • First release on PyPI.

Metadata

Release files for parsel 1.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for parsel 1.6.0
File Size Uploaded Provenance Actions
parsel-1.6.0.tar.gz 41.8 kB Without provenance Details for parsel-1.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for parsel 1.6.0
File Interpreter ABI Platform Actions
parsel-1.6.0-py2.py3-none-any.whl Python 2, Python 3 none any Details for parsel-1.6.0-py2.py3-none-any.whl

Total release size: 54.8 kB

Release files / parsel-1.6.0.tar.gz

Download URL parsel-1.6.0.tar.gz
Size 41.8 kB
Tags Source
SHA-256 checksum
How to use checksums
70efef0b651a996cceebc69e55a85eb2233be0890959203ba7c3a03c72725c79
BLAKE2b-256 checksum
How to use checksums
57208e7aef69de46de1c991d7880ffb6c046e0cb94ad41e20dcd6a74d02c1c1a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.7.1

Release files / parsel-1.6.0-py2.py3-none-any.whl

Download URL parsel-1.6.0-py2.py3-none-any.whl
Size 13.0 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
9e1fa8db1c0b4a878bf34b35c043d89c9d1cbebc23b4d34dbc3c0ec33f2e087d
BLAKE2b-256 checksum
How to use checksums
231e9b39d64cbab79d4362cdd7be7f5e9623d45c4a53b3f7522cd8210df52d8e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.7.1

Security for parsel 1.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page