💧 Osmosis Streaming Driver Implementation
Project description
osmosis-streaming-driver
💧 Osmosis Streaming Driver Implementation oceanprotocol.com
Table of Contents
- Overview
- Setup with Barge (Recommended)
- Standalone Install (Alternative)
- Code Style
- Testing
- Integration
- New Version
- License
Overview
This Osmosis Driver demonstrates how to expand Brizo, and go beyond static data files. It enables it to download live data from Websocket streams. It achieves this through a proxy server, which is responsible for issuing access tokens.
Introduction Video:
Setup
These steps show how to get this driver up and running with Barge.
Due to a technical limitation, this driver does not currently work with the standard osmosis-driver-interface. It relies on a forked version. This repo includes a script to extend the Brizo container with the custom interface and patch Barge (i.e. patch_barge.sh ).
We also piggyback on Brizo's container, and start our proxy server alongside the Brizo service. By default we expose port 3580 for the proxy.
Proxy Server Settings
If you open ./Dockerfile
, you will see a section containing the Proxy Server configuration. It resembles some of the settings used by the Brizo service.
You may need to customize some of the following settings:
#ENV PROXY_SERVER_HOSTNAME='myproxyhost.com'
#ENV PROXY_SERVER_TOKEN_EXPIRATION_MIN=2
PROXY_SERVER_HOSTNAME
is used for generating signed URLs. Let us assume you have a WebSocket stream pointing to
wss://websocket-api-host.com/stream-1
The proxy will generate a signed URL in the format
http://<HOST>:<PORT>/proxy?token=1234f654
By default, PORT
is 3580
and HOST
is the eth0
IP address. For our purposes, this is most likely the IP of the Brizo Docker container. This IP may not be directly available to external consumers. You can use this property to override HOST
when generating the signed URL.
PROXY_SERVER_TOKEN_EXPIRATION_MIN
is also involved in the process of generating signed URLs. It determines how long should a token be valid. By default this value is 2 minutes.
Other settings, related to timeout, default port and number of workers, may also be overriden. However, it is recommended that you leave them at their default values
Patching
Assuming you cloned barge into <BARGE DIR>
, simply call
./patch_barge.sh <BARGE DIR>
If the command was successful, you can start Barge as usual:
<BARGE DIR>/start_ocean.sh
Standalone Install
If can also install it in your local machine, without Barge. You just need to keep in mind that you need to clone and install the custom interface first
git clone https://github.com/marcojrfurtado/osmosis-driver-interface
cd osmosis-driver-interface
make dist && pip install --force dist/osmosis_driver_interface-<VERSION>-py2.py3-none-any.whl
Then install the osmosis streaming driver
make dist && pip install dist/osmosis_streaming_driver-<VERSION>-py2.py3-none-any
Instantiate an Osmosis instance and resolve an ipfs url
from osmosis_driver_interface.osmosis import Osmosis
url = "wss://stream.binance.com:9443/ws/bnbbtc@depth"
osm = Osmosis(url) # the proper osmosis driver is loaded automatically to match the url
# Resolve the url
download_url = osm.data_plugin.generate_url(url)
Code Style
Information about our Python code style is documented in the python-developer-guide and the python-style-guide.
Testing
Automatic tests are setup via Travis, executing tox
.
Our tests use the pytest framework.
Integration
This driver interface also includes integration tests, based on the squid-js setup.
Please refer to the Integration README for instructions on how to run this test and requirements.
New Version
The bumpversion.sh
script helps to bump the project version. You can execute the script using as first argument {major|minor|patch} to bump accordingly the version.
License
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for osmosis-streaming-driver-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | a90b91caefd283b7655961c9901427772a9d176d13787d08b6394cbbe6d3f34d |
|
MD5 | 4b59d2e71bf09ab6f4371680569c5944 |
|
BLAKE2b-256 | 219d1ac9d25aee9f7ec3f35eb27e5e1ac87b0ab439b0b765c4285947b55f52e3 |