Mozilla Bugzilla Bug Version ETL
Python version of Metric’s Bugzilla ETL (https://github.com/mozilla-metrics/bugzilla_etl)
Motivation and Details
PyPy and SetupTools are required. If you are installing on Windows please follow instructions to get these installed. When done, installation is easy:
pip install Bugzilla-ETL
You must prepare a settings.json file to reference the resources, and it’s filename must be provided as an argument in the command line. Examples of settings files can be found in resources/settings
Bugzilla-ETL keeps local run state in the form of two files: first_run_time and last_run_time. These are both parameters in the ``settings.json``` file.
- first_run_time is written only if it does not exist, and triggers a full ETL refresh. Delete this file if you want to create a new ES index and start ETL from the beginning.
- last_run_time is recorded whenever there has been a successful ETL. This file will not exist until the initial full ETL has completed successfully. Deleteing this file should have no net effect, other than making the program work harder then it should.
Asuming your settings.json file is in ~/Bugzilla_ETL:
cd ~/Bugzilla_ETL bzetl --settings=settings.json
Use --help for more options, and see example command line script
Got it working?
The initial ETL will take over two hours. If you want something quicker to confirm your configuration is correct, use --reset --quick arguments on the command line. This will limit ETL to the first 1000, and last 1000 bugs.
bzetl --settings=settings.json --reset --quick
If you plan to help improve this software, or if you enjoy working from source, you can clone from Github:
git clone https://github.com/klahnakoski/Bugzilla-ETL.git
pip install -e
It is best you install on Linux, but if you do install on Windows you can find further Windows-specific Python installation instructions at one of my other projects: https://github.com/klahnakoski/pyLibrary/blob/master/README.md
The Git clone will include test code. You can run those tests, but you must…
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|File Name & Checksum SHA256 Checksum Help||Version||File Type||Upload Date|
|Bugzilla_ETL-0.3.13353-py2.7.egg (286.2 kB) Copy SHA256 Checksum SHA256||2.7||Egg||Dec 19, 2013|
|Bugzilla-ETL-0.3.13353.win32-py2.7.exe (463.0 kB) Copy SHA256 Checksum SHA256||2.7||Windows Installer||Dec 19, 2013|
|Bugzilla-ETL-0.3.13353.zip (195.0 kB) Copy SHA256 Checksum SHA256||–||Source||Dec 19, 2013|