Project description

web2preview

For a given URL web2preview extracts its title, description, and image url using Open Graph, Twitter Card, or Schema meta tags, or, as an alternative, parses it as a generic webpage.

This is a fork of an excellent webpreview library and it maintains complete and absolute compatibility with the original while fixing several bugs, enhancing parsing, and adding a new convenient APIs.

Main differences between web2preview and webpreview:

Enhanced parsing for generic web pages
No unnecessary GET request is ever made if content of the page is supplied
Complete fallback mechanism which continues to parse until all methods are exhausted
Python Typings are added across the entire library (better syntax highlighting)
New dict-like WebPreview result object makes it easier to read parsing results
Command-line utility to extract title, description, and image from URL

Installation

pip install web2preview

Usage

Use the generic web2preview method to parse the page independent of its nature. It tries to extract the values from Open Graph properties, then it falls back to Twitter Card format, then Schema. If none of them can extract all three of the title, description, and preview image, then webpage's content is parsed using a generic extractor.

>>> from web2preview import web2preview

>>> p = web2preview("https://en.wikipedia.org/wiki/Enrico_Fermi")
>>> p.title
'Enrico Fermi - Wikipedia'
>>> p.description
'Italian-American physicist (1901–1954)'
>>> p.image
'https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Enrico_Fermi_1943-49.jpg/1200px-Enrico_Fermi_1943-49.jpg'

# Access the parsed fields both as attributes and items
>>> p["url"] == p.url
True

# Check if all three of the title, description, and image are in the parsing result
>>> p.is_complete()
True

# Provide page content from somewhere else
>>> content = """
<html>
    <head>
        <title>The Dormouse's story</title>
        <meta property="og:description" content="A Mad Tea-Party story" />
    </head>
    <body>
        <p class="title"><b>The Dormouse's story</b></p>
        <a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>
    </body>
</html>
"""

# This function call won't make any external calls,
# only relying on the supplied content, unlike the example above
>>> web2preview("aa.com", content=content)
WebPreview(url="http://aa.com", title="The Dormouse's story", description="A Mad Tea-Party story")

Using the command line

When web2preview is installed via pip the accompanying command-line tool is intalled alongside.

$ web2preview https://en.wikipedia.org/wiki/Enrico_Fermi
title: Enrico Fermi - Wikipedia
description: Italian-American physicist (1901–1954)
image: https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Enrico_Fermi_1943-49.jpg/1200px-Enrico_Fermi_1943-49.jpg

$ web2preview https://github.com/ --absolute-url
title: GitHub: Where the world builds software
description: GitHub is where over 83 million developers shape the future of software, together.
image: https://github.githubassets.com/images/modules/site/social-cards/github-social.png

Note: For the Original webpreview API please check the official docs.

Run with Docker

The docker image can be built and ran similarly to the command line. The default entry point is the web2preview command-line function.

$ docker build -t web2preview .
$ docker run -it --rm web2preview "https://en.m.wikipedia.org/wiki/Enrico_Fermi"
title: Enrico Fermi - Wikipedia
description: Enrico Fermi (Italian: [enˈriːko ˈfermi]; 29 September 1901 – 28 November 1954) was an Italian (later naturalized American) physicist and the creator of the world's first nuclear reactor, the Chicago Pile-1. He has been called the "architect of the nuclear age"[1] and the "architect of the atomic bomb".
image: https://upload.wikimedia.org/wikipedia/commons/thumb/d/d4/Enrico_Fermi_1943-49.jpg/1200px-Enrico_Fermi_1943-49.jpg

Note: built docker image weighs around 210MB.

Project details

These details have not been verified by PyPI

Project links

Release history Release notifications | RSS feed

1.1.3

Jun 8, 2022

1.1.1

May 24, 2022

This version

1.1.0

May 22, 2022

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

web2preview-1.1.0.tar.gz (13.7 kB view details)

Uploaded May 22, 2022 Source

Built Distribution

web2preview-1.1.0-py3-none-any.whl (16.1 kB view details)

Uploaded May 22, 2022 Python 3

File details

Details for the file web2preview-1.1.0.tar.gz.

File metadata

Download URL: web2preview-1.1.0.tar.gz
Upload date: May 22, 2022
Size: 13.7 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: poetry/1.1.13 CPython/3.10.2 Darwin/21.4.0

File hashes

Hashes for web2preview-1.1.0.tar.gz
Algorithm	Hash digest
SHA256	`38cb5ce6036eb8437a7813db7413dd0fff9b225399520441879659471395f138`
MD5	`0ada0e7ff6ef7a1cf8d9e03357320718`
BLAKE2b-256	`1b36fe42ddc6ec911cf0eb7829915dc43bccffeb0599f1e4dda042599107abf5`

See more details on using hashes here.

File details

Details for the file web2preview-1.1.0-py3-none-any.whl.

File metadata

Download URL: web2preview-1.1.0-py3-none-any.whl
Upload date: May 22, 2022
Size: 16.1 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: poetry/1.1.13 CPython/3.10.2 Darwin/21.4.0

File hashes

Hashes for web2preview-1.1.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`b123fd47360a810427e94b806e90591550b8aca487ac94fff74b98258a45dac1`
MD5	`ab8a0f363f710d370277445f0cfb9757`
BLAKE2b-256	`07645f757720e068014ee9d8db3fb671bb0c7a1b0495e5203a276d8b67cf3df5`