# default-scraper
Python Web Scraper
## Features
Scrap all search results for a keyword entered as an argument.
Can be saved as .csv and .json.
Also collect user data who uploaded contents included in search results.
## Usage
### Install
`bash pip install default-scraper `
or
`bash pip install git+https://github.com/Seongbuming/crawler.git `
### Scrap Instagram contents in python script
`python from default_scraper.instagram.parser import InstagramParser USERNAME = "" PASSWORD = "" KEYWORD = "" parser = InstagramParser(USERNAME, PASSWORD, KEYWORD, False) parser.run() `
### Scrap Instagram contents using bash command
Run following command to scrap contents from Instagram:
`bash python main.py --platform instagram --keyword {KEYWORD} [--output_file OUTPUT_FILE] [--all] `
Use –all or -a option to also scrap unstructured fields.
## Data description
Structured fields - pk - id - taken_at - media_type - code - comment_count - user - like_count - caption - accessibility_caption - original_width - original_height - images
Some fields may be missing depending on Instagram’s response data.
## Future works
Will support scraping from more platform services.
1 maintainer
Metadata
1 maintainer
Release files for default-scraper 1.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| default-scraper-1.1.1.tar.gz | 7.3 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| default_scraper-1.1.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 7.7 MB
Release files / default-scraper-1.1.1.tar.gz
| Download URL | default-scraper-1.1.1.tar.gz |
|---|---|
| Size | 7.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
46b571959ab4c15d89aa92e70e97ecd846f0c4722d883fe4ee2d233a0934f0b7
|
|
BLAKE2b-256 checksum How to use checksums |
9d0f0c3fcaf79384d1d2014dd0103919a7c3729ab2c0b4daa216817346b69109
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.7.9
|
Release files / default_scraper-1.1.1-py3-none-any.whl
| Download URL | default_scraper-1.1.1-py3-none-any.whl |
|---|---|
| Size | 7.7 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8e8d24e061aad5dd32a99022336b0e562174980f41b62d15a8a8ad77ef97bc7e
|
|
BLAKE2b-256 checksum How to use checksums |
8029834e4a1355e810adf328373ff7717f1b9e2dbc8150b26e30d246299c1b29
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.7.9
|
Security for default-scraper 1.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
