File Delivery over Unidirectional Transport (FLUTE)
Project description
FLUTE - File Delivery over Unidirectional Transport
Massively scalable multicast distribution solution
The library implements a unidirectional file delivery, without the need of a return channel.
RFC
This library implements the following RFCs
RFC | Title | Link |
---|---|---|
RFC 6726 | FLUTE - File Delivery over Unidirectional Transport | https://www.rfc-editor.org/rfc/rfc6726.html |
RFC 5775 | Asynchronous Layered Coding (ALC) Protocol Instantiation | https://www.rfc-editor.org/rfc/rfc5775.html |
RFC 5661 | Layered Coding Transport (LCT) Building Block | https://www.rfc-editor.org/rfc/rfc5651 |
RFC 5052 | Forward Error Correction (FEC) Building Block | https://www.rfc-editor.org/rfc/rfc5052 |
RFC 5510 | Reed-Solomon Forward Error Correction (FEC) Schemes | https://www.rfc-editor.org/rfc/rfc5510.html |
UDP/IP Multicast files sender
Transfer files over a UDP/IP network
use flute::sender::Sender;
use flute::sender::ObjectDesc;
use flute::sender::Cenc;
use std::net::UdpSocket;
use std::time::SystemTime;
// Create UDP Socket
let udp_socket = UdpSocket::bind("0.0.0.0:0").unwrap();
udp_socket.connect("224.0.0.1:3400").expect("Connection failed");
// Create FLUTE Sender
let tsi = 1;
let oti = Default::default();
let config = Default::default();
let mut sender = Sender::new(tsi, &oti, &config);
// Add object(s) (files) to the FLUTE sender
let obj = ObjectDesc::create_from_buffer(b"hello world", "text/plain",
&url::Url::parse("file:///hello.txt").unwrap(), 1, None, None, Cenc::Null, true, None, true).unwrap();
sender.add_object(obj);
// Always call publish after adding objects
sender.publish(SystemTime::now());
// Send FLUTE packets over UDP/IP
while let Some(pkt) = sender.read(SystemTime::now()) {
udp_socket.send(&pkt).unwrap();
std::thread::sleep(std::time::Duration::from_millis(1));
}
UDP/IP Multicast files receiver
Receive files from a UDP/IP network
use flute::receiver::{writer, MultiReceiver, UDPEndpoint};
use std::net::UdpSocket;
use std::time::SystemTime;
use std::rc::Rc;
// Create UDP/IP socket to receive FLUTE pkt
let endpoint = UDPEndpoint::new(None, "224.0.0.1".to_string(), 3400);
let udp_socket = UdpSocket::bind(format!("{}:{}", endpoint.destination_group_address, endpoint.port)).expect("Fail to bind");
// Create a writer able to write received files to the filesystem
let writer = Rc::new(writer::ObjectWriterFSBuilder::new(&std::path::Path::new("./flute_dir"))
.unwrap_or_else(|_| std::process::exit(0)));
// Create a multi-receiver capable of de-multiplexing several FLUTE sessions
let mut receiver = MultiReceiver::new(writer, None, false);
// Receive pkt from UDP/IP socket and push it to the FLUTE receiver
let mut buf = [0; 2048];
loop {
let (n, _src) = udp_socket.recv_from(&mut buf).expect("Failed to receive data");
let now = SystemTime::now();
receiver.push(&endpoint, &buf[..n], now).unwrap();
receiver.cleanup(now);
}
Application-Level Forward Erasure Correction (AL-FEC)
The following error recovery algorithms are supported
- No-code
- Reed-Solomon GF 2^8
- Reed-Solomon GF 2^8 Under Specified
- Reed-Solomon GF 2^16
- Reed-Solomon GF 2^m
- RaptorQ
- Raptor
The Oti
module provides an implementation of the Object Transmission Information (OTI)
used to configure Forward Error Correction (FEC) encoding in the FLUTE protocol.
use flute::sender::Oti;
use flute::sender::Sender;
// Reed Solomon 2^8 with encoding blocks composed of
// 60 source symbols and 4 repair symbols of 1424 bytes per symbol
let oti = Oti::new_reed_solomon_rs28(1424, 60, 4).unwrap();
let mut sender = Sender::new(1, &oti, &Default::default());
Content Encoding (CENC)
The following schemes are supported during the transmission/reception
- Null (no compression)
- Deflate
- Zlib
- Gzip
Files multiplex / Blocks interleave
The FLUTE Sender is able to transfer multiple files in parallel by interleaving packets from each file. For example:
Pkt file1 -> Pkt file2 -> Pkt file3 -> Pkt file1 -> Pkt file2 -> Pkt file3 ...
The Sender can interleave blocks within a single file. The following example shows Encoding Symbols (ES) from different blocks (B) are interleaved. For example:
(B 1,ES 1)->(B 2,ES 1)->(B 3,ES 1)->(B 1,ES 2)->(B 2,ES 2)...
To configure the multiplexing, use the Config
struct as follows:
use flute::sender::Sender;
use flute::sender::Config;
let config = Config {
// Transfer a maximum of 3 files in parallel
multiplex_files: 3,
// Interleave a maximum of 3 blocks within each file
interleave_blocks: 3,
..Default::default()
};
let mut sender = Sender::new(1, &Default::default(), &config);
Python bindings
Installation
pip install flute-alc
Example
Flute Sender python example
from flute import sender
# Flute Sender config parameters
sender_config = sender.Config()
# Object transmission parameters (no_code => no FEC)
# encoding symbol size : 1400 bytes
# Max source block length : 64 encoding symbols
oti = sender.Oti.new_no_code(1400, 64)
# Create FLUTE Sender
flute_sender = sender.Sender(1, oti, sender_config)
# Transfer a file
flute_sender.add_file("/path/to/file", 0, "application/octet-stream", None, None)
flute_sender.publish()
while True:
alc_pkt = flute_sender.read()
if alc_pkt == None:
break
#TODO Send alc_pkt over UDP/IP
Flute Receiver python example
from flute import receiver
# Write received objects to a destination folder
receiver_writer = receiver.ObjectWriterBuilder("/path/to/dest")
# FLUTE Receiver configuration parameters
receiver_config = receiver.Config()
tsi = 1
# Create a FLUTE receiver with the specified endpoint, tsi, writer, and configuration
udp_endpoint = receiver.UDPEndpoint("224.0.0.1", 1234)
flute_receiver = receiver.Receiver(udp_endpoint, tsi, receiver_writer, receiver_config)
while True:
# Receive LCT/ALC packet from UDP/IP multicast (Implement your own receive_from_udp_socket() function)
# Note: FLUTE does not handle the UDP/IP layer, you need to implement the socket reception mechanism yourself
pkt = receive_from_udp_socket()
# Push the received packet to the FLUTE receiver
flute_receiver.push(bytes(pkt))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for flute_alc-1.1.0-pp39-pypy39_pp73-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e50b309732eb4b394a9f33f205e00410f4d01672df93cab8a75f956fd1cb277e |
|
MD5 | cffa9200f4a42bb319bd1446e867d6b5 |
|
BLAKE2b-256 | 4f100ccaf4c2d85d3c14bb9794fd2a8d4f893131e84bb9bfe32f3f4625c910c2 |
Hashes for flute_alc-1.1.0-pp38-pypy38_pp73-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0aa65fdc8b838f6ee432319f4537954223e822f682fff00e71052d91b2f52cfe |
|
MD5 | 8b49224b402a08fe38a2e278b65e80cd |
|
BLAKE2b-256 | 0aa7d137f5379b886e1276d1c3ff233dd71f67805a091f9a2e758f16bea828eb |
Hashes for flute_alc-1.1.0-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e23fc27559e8c81d5008d740ff38e78337c199b55db6fa5b50436c32a216b9ed |
|
MD5 | c51a667b0d729f7ba2c7ea0ea0c6a635 |
|
BLAKE2b-256 | 364bea19f4bb4bb4b1ad10b46563c2449afe9d7e721600d6625297e211d7ec95 |
Hashes for flute_alc-1.1.0-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8b730bfbc8066d9f8f10ea21c295e4c9b86ef4286bd3d54ded73ed8c4fbad3ff |
|
MD5 | cc859b07b71fa417ed82815c83eacb82 |
|
BLAKE2b-256 | fe4c6af752892ddbf797b13b04878bfc651030c3d61275a5dab89085954a5673 |
Hashes for flute_alc-1.1.0-cp39-cp39-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c26f90e38570764838f4366b97970892e87639dc4208d0d24cd25945619ce0f9 |
|
MD5 | 23cbaf48d27d93c974e4edb624b0f0e1 |
|
BLAKE2b-256 | c360d78595341169823e1f2f348b064f412d3ddcb7ef9b2d0818c7348629cecb |
Hashes for flute_alc-1.1.0-cp38-cp38-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 498191f5824623760e5d3f2cee77218659f5b8bc08f2f4477b5824b7ed70e54c |
|
MD5 | 51c259e20e1bce8b2a53ae21e433251c |
|
BLAKE2b-256 | 6e42a19e5632f3335c2483e3928fd3c7725f4941076034bb76eabef601e50cdd |
Hashes for flute_alc-1.1.0-cp37-cp37m-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d7a6f600168ed8dccbf03b46cbd53900a7f6afed4c6943444db18210ba6a497f |
|
MD5 | 1fad4393a07c89dd321b6a9d4e2f452c |
|
BLAKE2b-256 | 103587349b8e197a9d39dabca56d8fdb853da37cb29c872adfa0e950fb821feb |