A Python library for automating interaction with websites
Project description
Home page
Overview
A Python library for automating interaction with websites. MechanicalSoup automatically stores and sends cookies, follows redirects, and can follow links and submit forms. It doesn’t do JavaScript.
MechanicalSoup was created by M Hickford, who was a fond user of the Mechanize library. Unfortunately, Mechanize is incompatible with Python 3 and its development stalled for several years. MechanicalSoup provides a similar API, built on Python giants Requests (for HTTP sessions) and BeautifulSoup (for document navigation). Since 2017 it is a project actively maintained by a small team including @hemberger and @moy.
Installation
PyPy and PyPy3 are also supported (and tested against).
Download and install the latest released version from PyPI:
pip install MechanicalSoup
Download and install the development version from GitHub:
pip install git+https://github.com/MechanicalSoup/MechanicalSoup
Installing from source (installs the version in the current working directory):
python setup.py install
(In all cases, add --user to the install command to install in the current user’s home directory.)
Documentation
The full documentation is available on https://mechanicalsoup.readthedocs.io/. You may want to jump directly to the automatically generated API documentation.
Example
From examples/expl_duck_duck_go.py, code to get the results from a DuckDuckGo search:
"""Example usage of MechanicalSoup to get the results from
DuckDuckGo."""
import mechanicalsoup
# Connect to duckduckgo
browser = mechanicalsoup.StatefulBrowser()
browser.open("https://duckduckgo.com/")
# Fill-in the search form
browser.select_form('#search_form_homepage')
browser["q"] = "MechanicalSoup"
browser.submit_selected()
# Display the results
for link in browser.get_current_page().select('a.result__a'):
print(link.text, '->', link.attrs['href'])
More examples are available in examples/.
For an example with a more complex form (checkboxes, radio buttons and textareas), read tests/test_browser.py and tests/test_form.py.
Development
Instructions for building, testing and contributing to MechanicalSoup: see CONTRIBUTING.rst.
Common problems
Read the FAQ.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for MechanicalSoup-0.11.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | bf53141b956ecf2c79d932a6b92f3df0dcf057019efd8e3fd7d4008def45e74a |
|
MD5 | 364b2688b1ac5c1e6099b55cc9c4b664 |
|
BLAKE2b-256 | f66a263f3e12d50e3272abf3842e13a3c991cda4af0f253e9c73a41d0b8387c3 |