Skip to main content

Screen-scraping library

Project description

Beautiful Soup is a library that makes it easy to scrape information from web pages. It sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying the parse tree.

Quick start

>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup("<p>Some<b>bad<i>HTML")
>>> print(soup.prettify())
<html>
 <body>
  <p>
   Some
   <b>
    bad
    <i>
     HTML
    </i>
   </b>
  </p>
 </body>
</html>
>>> soup.find(text="bad")
'bad'
>>> soup.i
<i>HTML</i>
#
>>> soup = BeautifulSoup("<tag1>Some<tag2/>bad<tag3>XML", "xml")
#
>>> print(soup.prettify())
<?xml version="1.0" encoding="utf-8"?>
<tag1>
 Some
 <tag2/>
 bad
 <tag3>
  XML
 </tag3>
</tag1>

To go beyond the basics, comprehensive documentation is available.

Links

Note on Python 2 sunsetting

Since 2012, Beautiful Soup has been developed as a Python 2 library which is automatically converted to Python 3 code as necessary. This makes it impossible to take advantage of some features of Python 3.

For this reason, I plan to discontinue Beautiful Soup's Python 2 support at some point after December 31, 2020: one year after the sunset date for Python 2 itself. Beyond that point, new Beautiful Soup development will exclusively target Python 3. Of course, older releases of Beautiful Soup, which support both versions, will continue to be available.

Supporting the project

If you use Beautiful Soup as part of your professional work, please consider a Tidelift subscription. This will support many of the free software projects your organization depends on, not just Beautiful Soup.

If you use Beautiful Soup for personal projects, the best way to say thank you is to read Tool Safety, a zine I wrote about what Beautiful Soup has taught me about software development.

Building the documentation

The bs4/doc/ directory contains full documentation in Sphinx format. Run make html in that directory to create HTML documentation.

Running the unit tests

Beautiful Soup supports unit test discovery from the project root directory:

$ nosetests
$ python -m unittest discover -s bs4

If you checked out the source tree, you should see a script in the home directory called test-all-versions. This script will run the unit tests under Python 2, then create a temporary Python 3 conversion of the source and run the unit tests again under Python 3.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

beautifulsoup4-4.9.3.tar.gz (376.0 kB view details)

Uploaded Source

Built Distributions

beautifulsoup4-4.9.3-py3-none-any.whl (115.8 kB view details)

Uploaded Python 3

beautifulsoup4-4.9.3-py2-none-any.whl (115.9 kB view details)

Uploaded Python 2

File details

Details for the file beautifulsoup4-4.9.3.tar.gz.

File metadata

  • Download URL: beautifulsoup4-4.9.3.tar.gz
  • Upload date:
  • Size: 376.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.2

File hashes

Hashes for beautifulsoup4-4.9.3.tar.gz
Algorithm Hash digest
SHA256 84729e322ad1d5b4d25f805bfa05b902dd96450f43842c4e99067d5e1369eb25
MD5 57fd468ae3eb055f6871106e8f7813e2
BLAKE2b-256 6bc3d31704ae558dcca862e4ee8e8388f357af6c9d9acb0cad4ba0fbbd350d9a

See more details on using hashes here.

File details

Details for the file beautifulsoup4-4.9.3-py3-none-any.whl.

File metadata

  • Download URL: beautifulsoup4-4.9.3-py3-none-any.whl
  • Upload date:
  • Size: 115.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.2

File hashes

Hashes for beautifulsoup4-4.9.3-py3-none-any.whl
Algorithm Hash digest
SHA256 fff47e031e34ec82bf17e00da8f592fe7de69aeea38be00523c04623c04fb666
MD5 bc7da27acae18d1907995f602ee572fe
BLAKE2b-256 d141e6495bd7d3781cee623ce23ea6ac73282a373088fcd0ddc809a047b18eae

See more details on using hashes here.

File details

Details for the file beautifulsoup4-4.9.3-py2-none-any.whl.

File metadata

  • Download URL: beautifulsoup4-4.9.3-py2-none-any.whl
  • Upload date:
  • Size: 115.9 kB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/44.0.0 requests-toolbelt/0.9.1 tqdm/4.49.0 CPython/3.8.2

File hashes

Hashes for beautifulsoup4-4.9.3-py2-none-any.whl
Algorithm Hash digest
SHA256 4c98143716ef1cb40bf7f39a8e3eec8f8b009509e74904ba3a7b315431577e35
MD5 7131ff441c704c172ae022881d9d5998
BLAKE2b-256 1cd98c507915ff962b9e854b477b203c171074f59cce9465dac9f71c2b57ebd6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page