Skip to main content

Screen-scraping library

Project description

Beautiful Soup is a library that makes it easy to scrape information from web pages. It sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying the parse tree.

Quick start

>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup("<p>Some<b>bad<i>HTML")
>>> print soup.prettify()
<html>
<body>
<p>
Some
<b>
bad
<i>
HTML
</i>
</b>
</p>
</body>
</html>
>>> soup.find(text="bad")
u'bad'
>>> soup.i
<i>HTML</i>
#
>>> soup = BeautifulSoup("<tag1>Some<tag2/>bad<tag3>XML", "xml")
#
>>> print soup.prettify()
<?xml version="1.0" encoding="utf-8">
<tag1>
Some
<tag2 />
bad
<tag3>
XML
</tag3>
</tag1>

To go beyond the basics, comprehensive documentation is available.

Links

Note on Python 2 sunsetting

Since 2012, Beautiful Soup has been developed as a Python 2 library which is automatically converted to Python 3 code as necessary. This makes it impossible to take advantage of some features of Python 3.

For this reason, I plan to discontinue Beautiful Soup's Python 2 support at some point after January 1, 2021: one year after the sunset date for Python 2 itself. Beyond that point, new Beautiful Soup development will exclusively target Python 3. Of course, older releases of Beautiful Soup, which support both versions, will continue to be available.

Supporting the project

If you use Beautiful Soup as part of your professional work, please consider a Tidelift subscription. This will support many of the free software projects your organization depends on, not just Beautiful Soup.

If you use Beautiful Soup for personal projects, the best way to say thank you is to read Tool Safety, a zine I wrote about what Beautiful Soup has taught me about software development.

Building the documentation

The bs4/doc/ directory contains full documentation in Sphinx format. Run make html in that directory to create HTML documentation.

Running the unit tests

Beautiful Soup supports unit test discovery from the project root directory:

$ nosetests
$ python -m unittest discover -s bs4

If you checked out the source tree, you should see a script in the home directory called test-all-versions. This script will run the unit tests under Python 2, then create a temporary Python 3 conversion of the source and run the unit tests again under Python 3.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

beautifulsoup4-4.8.2.tar.gz (298.6 kB view details)

Uploaded Source

Built Distributions

beautifulsoup4-4.8.2-py3-none-any.whl (106.9 kB view details)

Uploaded Python 3

beautifulsoup4-4.8.2-py2-none-any.whl (107.0 kB view details)

Uploaded Python 2

File details

Details for the file beautifulsoup4-4.8.2.tar.gz.

File metadata

  • Download URL: beautifulsoup4-4.8.2.tar.gz
  • Upload date:
  • Size: 298.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.7.0

File hashes

Hashes for beautifulsoup4-4.8.2.tar.gz
Algorithm Hash digest
SHA256 05fd825eb01c290877657a56df4c6e4c311b3965bda790c613a3d6fb01a5462a
MD5 5dbdb56c009e4632bae7bed1b385804b
BLAKE2b-256 52ba0e121661f529e7f456e903bf5c4d255b8051d8ce2b5e629c5212efe4c3f1

See more details on using hashes here.

File details

Details for the file beautifulsoup4-4.8.2-py3-none-any.whl.

File metadata

  • Download URL: beautifulsoup4-4.8.2-py3-none-any.whl
  • Upload date:
  • Size: 106.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.7.0

File hashes

Hashes for beautifulsoup4-4.8.2-py3-none-any.whl
Algorithm Hash digest
SHA256 9fbb4d6e48ecd30bcacc5b63b94088192dcda178513b2ae3c394229f8911b887
MD5 9e5e0ec6ec40a59c5935681b3c1458bd
BLAKE2b-256 cba1c698cf319e9cfed6b17376281bd0efc6bfc8465698f54170ef60a485ab5d

See more details on using hashes here.

File details

Details for the file beautifulsoup4-4.8.2-py2-none-any.whl.

File metadata

  • Download URL: beautifulsoup4-4.8.2-py2-none-any.whl
  • Upload date:
  • Size: 107.0 kB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.7.0

File hashes

Hashes for beautifulsoup4-4.8.2-py2-none-any.whl
Algorithm Hash digest
SHA256 e1505eeed31b0f4ce2dbb3bc8eb256c04cc2b3b72af7d551a4ab6efd5cbe5dae
MD5 63806c599f3987ea180bdc925ecf317e
BLAKE2b-256 c548c88b0b390ae1f785942fc83413feb1268a1eb696f343d4d55db735b9bb39

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page