Bamboo
Bamboo is a library intended to facilitate the process of creating extract, transform, and load (ETL) data pipelines. Bamboo includes many features such as the ability to download and cache HTTP assets, copy files from remote servers, run commands on remote servers, handle zipped archives, perform bulk database ingests and more.
Installation
pip install bamboo-lib
Additional Steps
If you will need to use the distributed locking functionality, you will need to install some additional software. Below are the instructions for macOS
Installing Sherlock on macOS.
brew install libmemcached
pip install pylibmc --install-option="--with-libmemcached=/usr/local/Cellar/libmemcached/1.0.18_2"
pip install sherlock
Running tests
To run the tests, simply run:
pytest
Alternatively, if you would like to display all log/print statements run:
pytest -s
Configuration
To change the default folder where Bamboo will store HTTP downloads, set the BAMBOO_DOWNLOAD_FOLDER environment variable. By default, downloads will go to /tmp.
To change the default logger settings, set BAMBOO_LOGGER_CONF to point to a logging configuration file (see example in example/logging.conf).
Troubleshooting tips
Mutli-processing issues on macOS
If you are running High Sierra or later and get an error like:
objc[30911]: +[__NSPlaceholderDate initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug
Try setting:
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
1 maintainer
Release files for bamboo-lib 0.0.39
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| bamboo_lib-0.0.39-py3-none-any.whl | Python 3 | none | any | Details |
Release files / bamboo_lib-0.0.39-py3-none-any.whl
| Download URL | bamboo_lib-0.0.39-py3-none-any.whl |
|---|---|
| Size | 59.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
23a65cfc129a6a579e49b485c13b3fc753897b58ef28944f5535aa98222576de
|
|
BLAKE2b-256 checksum How to use checksums |
932fbb2c7160a7de0b997f167d4761f21d336a1f249f5783c614ac6c27787182
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.9
|
