An utility library for separating individual messages in a stream
Project description
DataChunk is an useful library for proessing messages in a stream where messages are sent in a stream as shown below: <msg-size><msg-bytes><msg-size><msg-bytes><msg-size>…. msg-size is a 4 byte integer (little-endian) whenever a complete message is read, DataChunk calls the handle_msg method of the handler.
A message may come in fragments. Sometimes many messages may come in a single data packet. All we need to do is to call process_chunk method of DataChunk and DataChunk object will take care of re-assembling the messages as and when needed.
Basic example
from __future__ import print_function from struct import pack import random from datachunkpy.datachunk import DataChunk, MessageHandler # A message handler class class MyMsgHandler(MessageHandler): def __init__(self): self.msg_processed = 0 self.bytes_processed = 0 def handle_msg(self, data): print('I am handling', data) self.msg_processed += 1 self.bytes_processed += len(data) datach = DataChunk(MyMsgHandler()) i = 0 random.seed() mydata = b'' total_bytes = 0 while i < 10000: i += 1 j = random.randint(1,50) mydata += pack('i', j) mydata += b'a' * j total_bytes += j datach.process_chunk(mydata)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
datachunkpy-1.0.0.tar.gz
(3.1 kB
view hashes)