Skip to main content

🐶 OhMyScrapper - v0.11.0

OhMyScrapper scrapes texts and urls looking for links and jobs-data to create a final report with general information about job positions.

Scope

  • Read texts;
  • Extract and load urls;
  • Scrapes the urls looking for og:tags and titles;
  • Export a list of links with relevant information;

Installation

You can install directly in your pip:

pip install ohmyscrapper

I recomend to use the uv, so you can just use the command bellow and everything is installed:

uv add ohmyscrapper
uv run ohmyscrapper --version

But you can use everything as a tool, for example:

uvx ohmyscrapper --version

How to use and test (development only)

OhMyScrapper works in 3 stages:

  1. It collects and loads urls from a text in a database;
  2. It scraps/access the collected urls and read what is relevant. If it finds new urls, they are collected as well;
  3. Export a list of urls in CSV files;

You can do 3 stages with the command:

ohmyscrapper start

Remember to add your text file in the folder /input with the name that finishes with .txt!

You will find the exported files in the folder /output like this:

  • /output/report.csv
  • /output/report.csv-preview.html
  • /output/urls-simplified.csv
  • /output/urls-simplified.csv-preview.html
  • /output/urls.csv
  • /output/urls.csv-preview.html

BUT: if you want to do step by step, here it is:

First we load a text file you would like to look for urls. It it works with any txt file.

The default folder is /input. Put one or more text (finished with .txt) files in this folder and use the command load:

ohmyscrapper load

or, if you have another file in a different folder, just use the argument -input like this:

ohmyscrapper load -input=my-text-file.txt

In this case, you can add an url directly to the database, like this:

ohmyscrapper load -input=https://cesarcardoso.cc/

That will append the last url in the database to be scraped.

That will create a database if it doesn't exist and store every url the oh-my-scrapper find. After that, let's scrap the urls with the command scrap-urls:

ohmyscrapper scrap-urls --recursive --ignore-type

That will scrap only the linkedin urls we are interested in. For now they are:

  • linkedin_post: https://%.linkedin.com/posts/%
  • linkedin_redirect: https://lnkd.in/%
  • linkedin_job: https://%.linkedin.com/jobs/view/%
  • linkedin_feed" https://%.linkedin.com/feed/%
  • linkedin_company: https://%.linkedin.com/company/%

But we can use every other one generically using the argument --ignore-type:

ohmyscrapper scrap-urls --ignore-type

And we can ask to make it recursively adding the argument --recursive:

ohmyscrapper scrap-urls --recursive

!!! important: we are not sure about blocks we can have for excess of requests

And we can finally export with the command:

ohmyscrapper export
ohmyscrapper export --file=output/urls-simplified.csv --simplify
ohmyscrapper report

To monitor recent scraping jobs locally, start the dashboard:

ohmyscrapper dashboard

Then open http://127.0.0.1:8765. Use --host and --port to bind a different local address.

That's the basic usage! But you can understand more using the help:

ohmyscrapper --help

See Also

License

This package is distributed under the MIT license.

Metadata

Release files for ohmyscrapper 0.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for ohmyscrapper 0.11.0
File Size Uploaded Provenance Actions
ohmyscrapper-0.11.0.tar.gz 24.8 kB With provenance Details for ohmyscrapper-0.11.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for ohmyscrapper 0.11.0
File Interpreter ABI Platform Actions
ohmyscrapper-0.11.0-py3-none-any.whl Python 3 none any Details for ohmyscrapper-0.11.0-py3-none-any.whl

Total release size: 58.4 kB

Release files / ohmyscrapper-0.11.0.tar.gz

Download URL ohmyscrapper-0.11.0.tar.gz
Size 24.8 kB
Tags Source
SHA-256 checksum
How to use checksums
c20a479f3ada2ee7028de14abcdef21bd5c0afdf18c1bc86b63340c36dd89ea6
BLAKE2b-256 checksum
How to use checksums
d472e209ef89ab5df8f34fac378735d0ab31bc11934579dc4ef22e985f4701ca
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 17, 2026.

Transparency log

Release files / ohmyscrapper-0.11.0-py3-none-any.whl

Download URL ohmyscrapper-0.11.0-py3-none-any.whl
Size 33.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1e17615fb29c8d3607e91770a5626279a65a28f0b89736c744464dffd5ee250e
BLAKE2b-256 checksum
How to use checksums
5f29c2b6636df1f1e50eff540f88144904ddf8baa448d11c797fe2cb2c9ada4c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 17, 2026.

Transparency log

Security for ohmyscrapper 0.11.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for ohmyscrapper-0.11.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 17, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.11.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page