Israel Supermarket Parsers: Clients to parser the data published by the supermarkets.
This is a parser for ALL the supermarket chains listed in the GOV.IL site.
שקיפות מחירים (השוואת מחירים) - https://www.gov.il/he/departments/legalInfo/cpfta_prices_regulations הגדרת הקבצים שהיו צריכים להיות זמינים באתרים - https://www.nevo.co.il/law_html/law01/501_131.htm
Daily Automatic Testing
The test suite is scheduled to run daily, so you can see if the supermarket chains have changed something in their interface and the package will not work properly.
Status:
🤗 Want to support my work?
Got a question?
You can email me at erlichsefi@gmail.com
If you think you've found a bug:
- Create issue in issue tracker to see if it's already been reported
- Please consider solving the issue by yourself and creating a pull request.
What is il_supermarket_parsers?
A simple access layer to the data the supermarkets publish. You can download the data using https://github.com/OpenIsraeliSupermarkets/israeli-supermarket-scarpers.
You only need to run the following code to parse all the data currently shared by the supermarkets and was downloaded using the package above.
from il_supermarket_parsers import ConvertingTask
task = ConvertingTask(
source_configuration={"folder": "dumps"},
output_configuration=[{"output_mode": "csv", "output_folder": "outputs"}],
status_configuration={"database_type": "json", "base_path": "outputs"},
)
task.start()
task.join()
Optional filters (same names as Docker): enabled_parsers, files_types, and multiprocessing. Pass a maximum number of files with task.start(limit=10).
Quick start
il_supermarket_parsers can be installed using pip:
python3 pip install il_supermarket_parsers
If you want to run the latest version of the code, you can install it from the repo directly:
python3 -m pip install -U git+https://github.com/OpenIsraeliSupermarkets/israeli-supermarket-parsers.git
# or if you don't have 'git' installed
python3 -m pip install -U https://github.com/OpenIsraeliSupermarkets/israeli-supermarket-parsers/main
Running Docker
Build yourself:
docker build -t erlichsefi/israeli-supermarket-parsers --target prod .
or pull the existing image from docker hub:
docker pull erlichsefi/israeli-supermarket-parsers:latest
Then running it using:
docker run -v "./dumps:/usr/src/app/dumps" \
-v "./outputs:/usr/src/app/outputs" \
-e ENABLED_PARSERS="BAREKET,YAYNO_BITAN_AND_CARREFOUR" \
-e ENABLED_FILE_TYPES="STORE_FILE" \
-e NUMBER_OF_PROCESSES=4 \
-e LIMIT=1 \
erlichsefi/israeli-supermarket-parsers
Optional paths (defaults match the volumes above): DATA_FOLDER (input), OUTPUT_FOLDER (CSV output), STATUS_FOLDER (JSON status; defaults to OUTPUT_FOLDER). Optional tuning: NUMBER_OF_PROCESSES, ENABLED_PARSERS, and ENABLED_FILE_TYPES (comma-separated enum names), LIMIT.
Contributing
Help in testing, development, documentation and other tasks is highly appreciated and useful to the project. There are tasks for contributors of all experience levels.
If you need help getting started, don't hesitate to contact me.
Development status
IL SuperMarket Parser is beta software, as far as i see devlopment stoped until new issues will be found.
1 maintainer
Metadata
1 maintainer
Release files for il-supermarket-parser 1.0.14
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| il_supermarket_parser-1.0.14.tar.gz | 50.7 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| il_supermarket_parser-1.0.14-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 125.2 kB
Release files / il_supermarket_parser-1.0.14.tar.gz
| Download URL | il_supermarket_parser-1.0.14.tar.gz |
|---|---|
| Size | 50.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f2036b8d9d31298c21465805111faea8cc24b7965afa6dd4da9f3110c5cf2556
|
|
BLAKE2b-256 checksum How to use checksums |
abd8a61eb5f2890dc5b59b813b1cac99cbf19c3ef8cae75f1036e156be5065c0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Release files / il_supermarket_parser-1.0.14-py3-none-any.whl
| Download URL | il_supermarket_parser-1.0.14-py3-none-any.whl |
|---|---|
| Size | 74.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8beac1119a2b9537f1e8d80648d77c551ce7874e58cf2ce58da7daca40ff2734
|
|
BLAKE2b-256 checksum How to use checksums |
c2490a9552c4899ed023a5175b718ec5af8d7c1f087334657c3356d15c40eb54
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.9.25
|
Security for il-supermarket-parser 1.0.14
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
