Beautiful Soup is a library that makes it easy to scrape information from web pages. It sits atop an HTML or XML parser, providing Pythonic idioms for iterating, searching, and modifying the parse tree.
Quick start
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup("<p>Some<b>bad<i>HTML")
>>> print soup.prettify()
<html>
<body>
<p>
Some
<b>
bad
<i>
HTML
</i>
</b>
</p>
</body>
</html>
>>> soup.find(text="bad")
u'bad'
>>> soup.i
<i>HTML</i>
#
>>> soup = BeautifulSoup("<tag1>Some<tag2/>bad<tag3>XML", "xml")
#
>>> print soup.prettify()
<?xml version="1.0" encoding="utf-8">
<tag1>
Some
<tag2 />
bad
<tag3>
XML
</tag3>
</tag1>
To go beyond the basics, comprehensive documentation is available.
Links
Note on Python 2 sunsetting
Since 2012, Beautiful Soup has been developed as a Python 2 library which is automatically converted to Python 3 code as necessary. This makes it impossible to take advantage of some features of Python 3.
For this reason, I plan to discontinue Beautiful Soup's Python 2 support at some point after December 31, 2020: one year after the sunset date for Python 2 itself. Beyond that point, new Beautiful Soup development will exclusively target Python 3. Of course, older releases of Beautiful Soup, which support both versions, will continue to be available.
Supporting the project
If you use Beautiful Soup as part of your professional work, please consider a Tidelift subscription. This will support many of the free software projects your organization depends on, not just Beautiful Soup.
If you use Beautiful Soup for personal projects, the best way to say thank you is to read Tool Safety, a zine I wrote about what Beautiful Soup has taught me about software development.
Building the documentation
The bs4/doc/ directory contains full documentation in Sphinx
format. Run make html in that directory to create HTML
documentation.
Running the unit tests
Beautiful Soup supports unit test discovery from the project root directory:
$ nosetests
$ python -m unittest discover -s bs4
If you checked out the source tree, you should see a script in the home directory called test-all-versions. This script will run the unit tests under Python 2, then create a temporary Python 3 conversion of the source and run the unit tests again under Python 3.
1 maintainer
Release files for beautifulsoup4 4.9.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| beautifulsoup4-4.9.0.tar.gz | 368.8 kB | Details |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| beautifulsoup4-4.9.0-py3-none-any.whl | Python 3 | none | any | Details |
| beautifulsoup4-4.9.0-py2-none-any.whl | Python 2 | none | any | Details |
Total release size: 587.7 kB
Release files / beautifulsoup4-4.9.0.tar.gz
| Download URL | beautifulsoup4-4.9.0.tar.gz |
|---|---|
| Size | 368.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
594ca51a10d2b3443cbac41214e12dbb2a1cd57e1a7344659849e2e20ba6a8d8
|
|
BLAKE2b-256 checksum How to use checksums |
3be47cfc641f11e0eef60123912611a5c9ee7d4638da7325878b695b9ae4bb6f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.7.0
|
Release files / beautifulsoup4-4.9.0-py3-none-any.whl
| Download URL | beautifulsoup4-4.9.0-py3-none-any.whl |
|---|---|
| Size | 109.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
d4e96ac9b0c3a6d3f0caae2e4124e6055c5dcafde8e2f831ff194c104f0775a0
|
|
BLAKE2b-256 checksum How to use checksums |
e8b57bb03a696f2c9b7af792a8f51b82974e51c268f15e925fc834876a4efa0b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.7.0
|
Release files / beautifulsoup4-4.9.0-py2-none-any.whl
| Download URL | beautifulsoup4-4.9.0-py2-none-any.whl |
|---|---|
| Size | 109.5 kB |
| Tags | Python 2 |
|
SHA-256 checksum How to use checksums |
a4bbe77fd30670455c5296242967a123ec28c37e9702a8a81bd2f20a4baf0368
|
|
BLAKE2b-256 checksum How to use checksums |
2d3e8b2fc5d3c31c84d7209313f4059858f502f2e4a9d986693eca03fe325565
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.0.0 requests-toolbelt/0.8.0 tqdm/4.24.0 CPython/3.7.0
|
