Skip to main content
Donate to the Python Software Foundation or Purchase a PyCharm License to Benefit the PSF! Donate Now

IBM Streams Cloud Object Storage integration

Project description

Overview

Provides functions to read objects from Cloud Object Storage as a stream and submit tuples to create objects in Cloud Object Storage (COS).

This package exposes the com.ibm.streamsx.objectstorage toolkit as Python methods for use with Streaming Analytics service on IBM Cloud and IBM Streams including IBM Cloud Private for Data.

Sample

A simple hello world example of a Streams application writing string messages to an object. Scan for created object on COS and read the content:

from streamsx.topology.topology import *
from streamsx.topology.schema import CommonSchema
from streamsx.topology.context import submit
import streamsx.objectstorage as cos

topo = Topology('ObjectStorageHelloWorld')

to_cos = topo.source(['Hello', 'World!'])
to_cos = to_cos.as_string()

# sample bucket with resiliency "regional" and location "us-south"
bucket = 'streamsx-py-sample'
# US-South region private endpoint
endpoint='s3.private.us-south.cloud-object-storage.appdomain.cloud'

# Write a stream to COS
cos.write(to_cos, bucket, endpoint, '/sample/hw%OBJECTNUM.txt')

scanned = cos.scan(topo, bucket=bucket, endpoint=endpoint, directory='/sample')

# read text file line by line
r = cos.read(scanned, bucket=bucket, endpoint=endpoint)

# print each line (tuple)
r.print()

submit('STREAMING_ANALYTICS_SERVICE', topo)
# Use for IBM Streams including IBM Cloud Private for Data
# submit ('DISTRIBUTED', topo)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
streamsx.objectstorage-1.1.0-py2.py3-none-any.whl (13.8 kB) Copy SHA256 hash SHA256 Wheel 3.5
streamsx.objectstorage-1.1.0.tar.gz (6.5 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page