Skip to main content

Israel Supermarket Parsers: Clients to parser the data published by the supermarkets.

This is a parser for ALL the supermarket chains listed in the GOV.IL site.

שקיפות מחירים (השוואת מחירים) - https://www.gov.il/he/departments/legalInfo/cpfta_prices_regulations הגדרת הקבצים שהיו צריכים להיות זמינים באתרים - https://www.nevo.co.il/law_html/law01/501_131.htm

Unit & Integration Tests CodeQL Pylint Publish Docker image Upload Python Package

Daily Automatic Testing

The test suite is scheduled to run daily, so you can see if the supermarket chains have changed something in their interface and the package will not work properly.

Status: Scheduled Tests

🤗 Want to support my work?

Buy Me A Coffee

Got a question?

You can email me at erlichsefi@gmail.com

If you think you've found a bug:

  • Create issue in issue tracker to see if it's already been reported
  • Please consider solving the issue by yourself and creating a pull request.

What is il_supermarket_parsers?

A simple access layer to the data the supermarkets publish. You can download the data using https://github.com/OpenIsraeliSupermarkets/israeli-supermarket-scarpers.

You only need to run the following code to parse all the data currently shared by the supermarkets and was downloaded using the package above.

from il_supermarket_parsers import ConvertingTask

task = ConvertingTask(
    source_configuration={"folder": "dumps"},
    output_configuration=[{"output_mode": "csv", "output_folder": "outputs"}],
    status_configuration={"database_type": "json", "base_path": "outputs"},
)
task.start()
task.join()

Optional filters (same names as Docker): enabled_parsers, files_types, and multiprocessing. Pass a maximum number of files with task.start(limit=10).

Quick start

il_supermarket_parsers can be installed using pip:

python3 pip install il_supermarket_parsers

If you want to run the latest version of the code, you can install it from the repo directly:

python3 -m pip install -U git+https://github.com/OpenIsraeliSupermarkets/israeli-supermarket-parsers.git
# or if you don't have 'git' installed
python3 -m pip install -U https://github.com/OpenIsraeliSupermarkets/israeli-supermarket-parsers/main

Running Docker

Build yourself:

docker build -t erlichsefi/israeli-supermarket-parsers --target prod .

or pull the existing image from docker hub:

docker pull erlichsefi/israeli-supermarket-parsers:latest

Then running it using:

docker run  -v "./dumps:/usr/src/app/dumps" \
            -v "./outputs:/usr/src/app/outputs" \
            -e ENABLED_PARSERS="BAREKET,YAYNO_BITAN_AND_CARREFOUR" \
            -e ENABLED_FILE_TYPES="STORE_FILE" \
            -e NUMBER_OF_PROCESSES=4 \
            -e LIMIT=1 \
            erlichsefi/israeli-supermarket-parsers

Optional paths (defaults match the volumes above): DATA_FOLDER (input), OUTPUT_FOLDER (CSV output), STATUS_FOLDER (JSON status; defaults to OUTPUT_FOLDER). Optional tuning: NUMBER_OF_PROCESSES, ENABLED_PARSERS, and ENABLED_FILE_TYPES (comma-separated enum names), LIMIT.

Contributing

Help in testing, development, documentation and other tasks is highly appreciated and useful to the project. There are tasks for contributors of all experience levels.

If you need help getting started, don't hesitate to contact me.

Development status

IL SuperMarket Parser is beta software, as far as i see devlopment stoped until new issues will be found.

Metadata

Release files for il-supermarket-parser 1.0.12

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for il-supermarket-parser 1.0.12
File Size Uploaded
il_supermarket_parser-1.0.12.tar.gz 48.1 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for il-supermarket-parser 1.0.12
File Interpreter ABI Platform
il_supermarket_parser-1.0.12-py3-none-any.whl Python 3 none any Details

Total release size: 119.3 kB

Release files / il_supermarket_parser-1.0.12.tar.gz

Download URL il_supermarket_parser-1.0.12.tar.gz
Size 48.1 kB
Tags Source
SHA-256 checksum
How to use checksums
a48af36c45664182ff89acbcd0a93192ecdc34ce81761f008dfa3ff34630d17b
BLAKE2b-256 checksum
How to use checksums
ef5a8e638ef947a39e283c552bfd4fb6b3b259202080c339b186d69e5b301047
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Release files / il_supermarket_parser-1.0.12-py3-none-any.whl

Download URL il_supermarket_parser-1.0.12-py3-none-any.whl
Size 71.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1a187860bd3489e6b159d5ca1beea05e4d5c298e9ab133d5af5d80a2fbf321f6
BLAKE2b-256 checksum
How to use checksums
a9c8fb1c642d34c6c804ea9faef15a613525761d73bc4df70f75a0fa3b53fdbf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.9.25

Release history Release notifications | RSS feed

This release

1.0.12 This release

2 release files

1.0.11

2 release files

1.0.9

2 release files

1.0.8

2 release files

1.0.7

2 release files

1.0.6

2 release files

1.0.5

2 release files

1.0.4

2 release files

1.0.3

2 release files

1.0.2

2 release files

1.0.1

2 release files

1.0.0

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.9

2 release files

0.1.8

2 release files

0.1.7

2 release files

0.1.6

2 release files

0.1.5

2 release files

0.1.4

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.1.0

2 release files

0.0.9

2 release files

0.0.8

2 release files

0.0.7

2 release files

0.0.6

2 release files

0.0.5

2 release files

0.0.4

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page