🐶 OhMyScrapper - v0.11.0
OhMyScrapper scrapes texts and urls looking for links and jobs-data to create a final report with general information about job positions.
Scope
- Read texts;
- Extract and load urls;
- Scrapes the urls looking for og:tags and titles;
- Export a list of links with relevant information;
Installation
You can install directly in your pip:
pip install ohmyscrapper
I recomend to use the uv, so you can just use the command bellow and everything is installed:
uv add ohmyscrapper
uv run ohmyscrapper --version
But you can use everything as a tool, for example:
uvx ohmyscrapper --version
How to use and test (development only)
OhMyScrapper works in 3 stages:
- It collects and loads urls from a text in a database;
- It scraps/access the collected urls and read what is relevant. If it finds new urls, they are collected as well;
- Export a list of urls in CSV files;
You can do 3 stages with the command:
ohmyscrapper start
Remember to add your text file in the folder
/inputwith the name that finishes with.txt!
You will find the exported files in the folder /output like this:
/output/report.csv/output/report.csv-preview.html/output/urls-simplified.csv/output/urls-simplified.csv-preview.html/output/urls.csv/output/urls.csv-preview.html
BUT: if you want to do step by step, here it is:
First we load a text file you would like to look for urls. It it works with any txt file.
The default folder is /input. Put one or more text (finished with .txt) files
in this folder and use the command load:
ohmyscrapper load
or, if you have another file in a different folder, just use the argument -input like this:
ohmyscrapper load -input=my-text-file.txt
In this case, you can add an url directly to the database, like this:
ohmyscrapper load -input=https://cesarcardoso.cc/
That will append the last url in the database to be scraped.
That will create a database if it doesn't exist and store every url the oh-my-scrapper
find. After that, let's scrap the urls with the command scrap-urls:
ohmyscrapper scrap-urls --recursive --ignore-type
That will scrap only the linkedin urls we are interested in. For now they are:
- linkedin_post: https://%.linkedin.com/posts/%
- linkedin_redirect: https://lnkd.in/%
- linkedin_job: https://%.linkedin.com/jobs/view/%
- linkedin_feed" https://%.linkedin.com/feed/%
- linkedin_company: https://%.linkedin.com/company/%
But we can use every other one generically using the argument --ignore-type:
ohmyscrapper scrap-urls --ignore-type
And we can ask to make it recursively adding the argument --recursive:
ohmyscrapper scrap-urls --recursive
!!! important: we are not sure about blocks we can have for excess of requests
And we can finally export with the command:
ohmyscrapper export
ohmyscrapper export --file=output/urls-simplified.csv --simplify
ohmyscrapper report
To monitor recent scraping jobs locally, start the dashboard:
ohmyscrapper dashboard
Then open http://127.0.0.1:8765. Use --host and --port to bind a different local address.
That's the basic usage! But you can understand more using the help:
ohmyscrapper --help
See Also
License
This package is distributed under the MIT license.
Metadata
Release files for ohmyscrapper 0.11.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| ohmyscrapper-0.11.0.tar.gz | 24.8 kB | With provenance | Details for ohmyscrapper-0.11.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| ohmyscrapper-0.11.0-py3-none-any.whl | Python 3 | none | any | Details for ohmyscrapper-0.11.0-py3-none-any.whl |
Total release size: 58.4 kB
Release files / ohmyscrapper-0.11.0.tar.gz
| Download URL | ohmyscrapper-0.11.0.tar.gz |
|---|---|
| Size | 24.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c20a479f3ada2ee7028de14abcdef21bd5c0afdf18c1bc86b63340c36dd89ea6
|
|
BLAKE2b-256 checksum How to use checksums |
d472e209ef89ab5df8f34fac378735d0ab31bc11934579dc4ef22e985f4701ca
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jun 17, 2026.
Transparency logRelease files / ohmyscrapper-0.11.0-py3-none-any.whl
| Download URL | ohmyscrapper-0.11.0-py3-none-any.whl |
|---|---|
| Size | 33.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
1e17615fb29c8d3607e91770a5626279a65a28f0b89736c744464dffd5ee250e
|
|
BLAKE2b-256 checksum How to use checksums |
5f29c2b6636df1f1e50eff540f88144904ddf8baa448d11c797fe2cb2c9ada4c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jun 17, 2026.
Transparency logSecurity for ohmyscrapper 0.11.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for ohmyscrapper-0.11.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jun 17, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.