Htmldate: Find the Publication Date of Web Pages
Find original and updated publication dates of any web page. It is often not possible to do it using just the URL or the server response.
On the command-line or with Python, all the steps needed from web page download to HTML parsing, scraping, and text analysis are included.
The package is used in production on millions of documents and integrated into thousands of projects.
In a nutshell
With Python
>>> from htmldate import find_date
>>> find_date('http://blog.python.org/2016/12/python-360-is-now-available.html')
'2016-12-23'
On the command-line
$ htmldate -u http://blog.python.org/2016/12/python-360-is-now-available.html
'2016-12-23'
Features
- Flexible input: URLs, HTML files, or HTML trees can be used as input (including batch processing).
- Customizable output: Any date format (defaults to ISO 8601 YMD).
- Detection of both original and updated dates.
- Multilingual.
- Compatible with Python 3.10 and later.
How it works
Htmldate operates by sifting through HTML markup and if necessary text elements. It features the following heuristics:
- Markup in header: Common patterns are used to identify relevant
elements (e.g.
linkandmetaelements) including Open Graph protocol attributes. - HTML code: The whole document is searched for structural markers
like
abbrortimeelements and a series of attributes (e.g.postmetadata). - Bare HTML content: Heuristics are run on text and markup:
- In
fastmode the HTML page is cleaned and precise patterns are targeted. - In
extensivemode all potential dates are collected and a disambiguation algorithm determines the best one.
- In
Finally, the output is validated and converted to the chosen format.
Performance
1000 web pages containing identifiable dates (as of 2026-06-01 on Python 3.13)
| Python Package | Precision | Recall | Accuracy | F-Score | Time |
|---|---|---|---|---|---|
| articleDateExtractor 0.20 | 0.846 | 0.745 | 0.656 | 0.792 | 3x |
| date_guesser 2.1.4 | 0.832 | 0.611 | 0.544 | 0.705 | 11x |
| goose3 3.1.21 | 0.930 | 0.568 | 0.545 | 0.706 | 14x |
| htmldate[all] 1.10.0 (fast) | 0.924 | 0.927 | 0.861 | 0.925 | 1x |
| htmldate[all] 1.10.0 (extensive) | 0.908 | 0.993 | 0.903 | 0.949 | 1.8x |
| newspaper4k 0.9.5 | 0.912 | 0.728 | 0.680 | 0.810 | 2.5x |
| news-please 1.6.16 | 0.845 | 0.777 | 0.680 | 0.810 | 29x |
For the complete results and explanations see evaluation page.
Installation
Htmldate is tested on Linux, macOS and Windows systems, it is compatible
with Python 3.10 upwards. It can notably be installed with pip (pip3
where applicable) from the PyPI package repository:
pip install htmldate- (optionally)
pip install htmldate[speed]
The last version to support Python 3.6 and 3.7 is htmldate==1.8.1; for
Python 3.8 and 3.9 use the 1.9.x series.
Documentation
For more details on installation, Python & CLI usage, please refer to the documentation: htmldate.readthedocs.io
License
This package is distributed under the Apache 2.0 license.
Versions prior to v1.8.0 are under GPLv3+ license.
Context and contributions
Initially launched to create text databases for research purposes at the Berlin-Brandenburg Academy of Sciences (DWDS and ZDL units), this project continues to be maintained but its future development depends on community support.
If you value this software or depend on it for your product, consider sponsoring it and contributing to its codebase. Your support will help maintain and enhance this package. Visit the Contributing page for more information.
Reach out via the software repository or the contact page for inquiries, collaborations, or feedback.
@article{barbaresi-2020-htmldate,
title = {{htmldate: A Python package to extract publication dates from web pages}},
author = "Barbaresi, Adrien",
journal = "Journal of Open Source Software",
volume = 5,
number = 51,
pages = 2439,
url = {https://doi.org/10.21105/joss.02439},
publisher = {The Open Journal},
year = 2020,
}
- Barbaresi, A. "htmldate: A Python package to extract publication dates from web pages", Journal of Open Source Software, 5(51), 2439, 2020. DOI: 10.21105/joss.02439
- Barbaresi, A. "Generic Web Content Extraction with Open-Source Software", Proceedings of KONVENS 2019, Kaleidoscope Abstracts, 2019.
- Barbaresi, A. "Efficient construction of metadata-enhanced web corpora", Proceedings of the 10th Web as Corpus Workshop (WAC-X), 2016.
Acknowledgements
Kudos to the following software libraries:
- lxml, dateparser
- A few patterns are derived from the python-goose, metascraper, newspaper and articleDateExtractor libraries. This module extends their coverage and robustness significantly.
1 maintainer
Metadata
1 maintainer
Release files for htmldate 1.10.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| htmldate-1.10.0.tar.gz | 44.5 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| htmldate-1.10.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 76.0 kB
Release files / htmldate-1.10.0.tar.gz
| Download URL | htmldate-1.10.0.tar.gz |
|---|---|
| Size | 44.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a38df10772ab5d7dbb11896e3f6a852a8491fb1b0965465bc174e23fc2baae58
|
|
BLAKE2b-256 checksum How to use checksums |
ad1fe7cf83e23d7b68105de8b874a8b36ba23b450d6f71388583e4ca3ce475ca
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.1
|
Release files / htmldate-1.10.0-py3-none-any.whl
| Download URL | htmldate-1.10.0-py3-none-any.whl |
|---|---|
| Size | 31.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9211dae35ab94147c8ed9e5fc2c9287a5cf31d2394cb7857e7f5dd814eb2aad6
|
|
BLAKE2b-256 checksum How to use checksums |
f717d3356233c826c641f940983d9479eab27faec59d49f4070bc58e80fcc021
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.13.1
|
