File Delivery over Unidirectional Transport (FLUTE)
Project description
FLUTE - File Delivery over Unidirectional Transport
Massively scalable multicast distribution solution
The library implements a unidirectional file delivery, without the need of a return channel.
RFC
This library implements the following RFCs
RFC | Title | Link |
---|---|---|
RFC 6726 | FLUTE - File Delivery over Unidirectional Transport | https://www.rfc-editor.org/rfc/rfc6726.html |
RFC 5775 | Asynchronous Layered Coding (ALC) Protocol Instantiation | https://www.rfc-editor.org/rfc/rfc5775.html |
RFC 5661 | Layered Coding Transport (LCT) Building Block | https://www.rfc-editor.org/rfc/rfc5651 |
RFC 5052 | Forward Error Correction (FEC) Building Block | https://www.rfc-editor.org/rfc/rfc5052 |
RFC 5510 | Reed-Solomon Forward Error Correction (FEC) Schemes | https://www.rfc-editor.org/rfc/rfc5510.html |
UDP/IP Multicast files sender
Transfer files over a UDP/IP network
use flute::sender::Sender;
use flute::sender::ObjectDesc;
use flute::sender::Cenc;
use std::net::UdpSocket;
use std::time::SystemTime;
// Create UDP Socket
let udp_socket = UdpSocket::bind("0.0.0.0:0").unwrap();
udp_socket.connect("224.0.0.1:3400").expect("Connection failed");
// Create FLUTE Sender
let tsi = 1;
let oti = Default::default();
let config = Default::default();
let mut sender = Sender::new(tsi, &oti, &config);
// Add object(s) (files) to the FLUTE sender
let obj = ObjectDesc::create_from_buffer(b"hello world", "text/plain",
&url::Url::parse("file:///hello.txt").unwrap(), 1, None, None, Cenc::Null, true, None, true).unwrap();
sender.add_object(obj);
// Always call publish after adding objects
sender.publish(SystemTime::now());
// Send FLUTE packets over UDP/IP
while let Some(pkt) = sender.read(SystemTime::now()) {
udp_socket.send(&pkt).unwrap();
std::thread::sleep(std::time::Duration::from_millis(1));
}
UDP/IP Multicast files receiver
Receive files from a UDP/IP network
use flute::receiver::{writer, MultiReceiver, UDPEndpoint};
use std::net::UdpSocket;
use std::time::SystemTime;
use std::rc::Rc;
// Create UDP/IP socket to receive FLUTE pkt
let endpoint = UDPEndpoint::new(None, "224.0.0.1".to_string(), 3400);
let udp_socket = UdpSocket::bind(format!("{}:{}", endpoint.destination_group_address, endpoint.port)).expect("Fail to bind");
// Create a writer able to write received files to the filesystem
let writer = Rc::new(writer::ObjectWriterFSBuilder::new(&std::path::Path::new("./flute_dir"))
.unwrap_or_else(|_| std::process::exit(0)));
// Create a multi-receiver capable of de-multiplexing several FLUTE sessions
let mut receiver = MultiReceiver::new(writer, None, false);
// Receive pkt from UDP/IP socket and push it to the FLUTE receiver
let mut buf = [0; 2048];
loop {
let (n, _src) = udp_socket.recv_from(&mut buf).expect("Failed to receive data");
let now = SystemTime::now();
receiver.push(&endpoint, &buf[..n], now).unwrap();
receiver.cleanup(now);
}
Application-Level Forward Erasure Correction (AL-FEC)
The following error recovery algorithms are supported
- No-code
- Reed-Solomon GF 2^8
- Reed-Solomon GF 2^8 Under Specified
- Reed-Solomon GF 2^16
- Reed-Solomon GF 2^m
- RaptorQ
- Raptor
The Oti
module provides an implementation of the Object Transmission Information (OTI)
used to configure Forward Error Correction (FEC) encoding in the FLUTE protocol.
use flute::sender::Oti;
use flute::sender::Sender;
// Reed Solomon 2^8 with encoding blocks composed of
// 60 source symbols and 4 repair symbols of 1424 bytes per symbol
let oti = Oti::new_reed_solomon_rs28(1424, 60, 4).unwrap();
let mut sender = Sender::new(1, &oti, &Default::default());
Content Encoding (CENC)
The following schemes are supported during the transmission/reception
- Null (no compression)
- Deflate
- Zlib
- Gzip
Files multiplex / Blocks interleave
The FLUTE Sender is able to transfer multiple files in parallel by interleaving packets from each file. For example:
Pkt file1 -> Pkt file2 -> Pkt file3 -> Pkt file1 -> Pkt file2 -> Pkt file3 ...
The Sender can interleave blocks within a single file. The following example shows Encoding Symbols (ES) from different blocks (B) are interleaved. For example:
(B 1,ES 1)->(B 2,ES 1)->(B 3,ES 1)->(B 1,ES 2)->(B 2,ES 2)...
To configure the multiplexing, use the Config
struct as follows:
use flute::sender::Sender;
use flute::sender::Config;
let config = Config {
// Transfer a maximum of 3 files in parallel
multiplex_files: 3,
// Interleave a maximum of 3 blocks within each file
interleave_blocks: 3,
..Default::default()
};
let mut sender = Sender::new(1, &Default::default(), &config);
Python bindings
Installation
pip install flute-alc
Example
Flute Sender python example
from flute import sender
# Flute Sender config parameters
sender_config = sender.Config()
# Object transmission parameters (no_code => no FEC)
# encoding symbol size : 1400 bytes
# Max source block length : 64 encoding symbols
oti = sender.Oti.new_no_code(1400, 64)
# Create FLUTE Sender
flute_sender = sender.Sender(1, oti, sender_config)
# Transfer a file
flute_sender.add_file("/path/to/file", 0, "application/octet-stream", None, None)
flute_sender.publish()
while True:
alc_pkt = flute_sender.read()
if alc_pkt == None:
break
#TODO Send alc_pkt over UDP/IP
Flute Receiver python example
from flute import receiver
# Write received objects to a destination folder
receiver_writer = receiver.FluteWriter("/path/to/dest")
# FLUTE Receiver configuration parameters
receiver_config = receiver.Config()
tsi = 1
# Creation of a FLUTE receiver
udp_endpoint = receiver.UDPEndpoint("224.0.0.1", 1234)
flute_receiver = receiver.Receiver(udp_endpoint, tsi, receiver_writer, receiver_config)
while True:
# Receive LCT/ALC packet from multicast
pkt = receive_from_udp_socket()
# Push packet to the flute receiver
flute_receiver.push(bytes(pkt))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for flute_alc-1.0.1-pp39-pypy39_pp73-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3da2b52398af6ac5d961a548b52baf03e9d68f2123dfb7e9bc0e98a995a8b280 |
|
MD5 | 7b31c86454343f2d6ca299f74b5d0e31 |
|
BLAKE2b-256 | 07beda3fa7f4be62cb76ae41bb0b56d4196c1ffaf3bce1cc85e98d27a3627c75 |
Hashes for flute_alc-1.0.1-pp38-pypy38_pp73-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fdd547e6cf8ac5e3d2c6a700501bf3ad02cea1b0257a35e0313e38299618d1b1 |
|
MD5 | 7f6cad7ed57712a75e1eb381d8620f40 |
|
BLAKE2b-256 | 5a30612b3dd6243103d501cf9a1d8efd23889697d80344018d5b3f8cbf1c8092 |
Hashes for flute_alc-1.0.1-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3cd3b28609e6bd34ad091701b5341c3ad13b709ce27bbae23b883998f19b2511 |
|
MD5 | d0c4567935cd11a26495c36caccd701a |
|
BLAKE2b-256 | e8931a4e230faf39097957bba3963d2fb987151d8ef0edd34251882de9721642 |
Hashes for flute_alc-1.0.1-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 933ae9e047bce173dd3cf9b0c6f7c8f67f02f5ace3ae724c6fb08ffbdbf8b264 |
|
MD5 | cbd4f08281f168b7659aaa57e01bb34e |
|
BLAKE2b-256 | 1904856d4c02bc81a67fe6b3eed355fa25c17abfd1b3b461f79c8ccaa9dde5b6 |
Hashes for flute_alc-1.0.1-cp39-cp39-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 04cbcf2031e74075bacbc42ad094597001f5c4ddf5ad251314422e1e47986634 |
|
MD5 | 7dd41500c9eb8879b58a377478fce84a |
|
BLAKE2b-256 | bc4a5476286b95b86d35354fea1583d77e200a8a69abf32b98bb6bed250a8193 |
Hashes for flute_alc-1.0.1-cp38-cp38-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5a96d3602b433b7001fcb06fcdfdcbf23e2fda24788827ee17824ac0c13b0643 |
|
MD5 | 272c938b6a534e6e4e712c2c0017b97e |
|
BLAKE2b-256 | bcd2746d6cdc44c1520a55d3277b8d06b7f1c62d57254d86ae1726014e5ff9d7 |
Hashes for flute_alc-1.0.1-cp37-cp37m-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | cd4d7be682b2cef5107682894b1170c7b7426c2496f6fa6727b287bfe850dd68 |
|
MD5 | 08d48f824ea39ff8ead20e43d389c791 |
|
BLAKE2b-256 | 8c08ae06232d15d2d95f3d35d1528487835ca7bb7215d40657ca9a43fea2b9b2 |