File Delivery over Unidirectional Transport (FLUTE)
Project description
FLUTE - File Delivery over Unidirectional Transport
Massively scalable multicast distribution solution
The library implements a unidirectional file delivery, without the need of a return channel.
RFC
This library implements the following RFCs
RFC | Title | Link |
---|---|---|
RFC 6726 | FLUTE - File Delivery over Unidirectional Transport | https://www.rfc-editor.org/rfc/rfc6726.html |
RFC 5775 | Asynchronous Layered Coding (ALC) Protocol Instantiation | https://www.rfc-editor.org/rfc/rfc5775.html |
RFC 5661 | Layered Coding Transport (LCT) Building Block | https://www.rfc-editor.org/rfc/rfc5651 |
RFC 5052 | Forward Error Correction (FEC) Building Block | https://www.rfc-editor.org/rfc/rfc5052 |
RFC 5510 | Reed-Solomon Forward Error Correction (FEC) Schemes | https://www.rfc-editor.org/rfc/rfc5510.html |
UDP/IP Multicast files sender
Transfer files over a UDP/IP network
use flute::sender::Sender;
use flute::sender::ObjectDesc;
use flute::sender::Cenc;
use std::net::UdpSocket;
use std::time::SystemTime;
// Create UDP Socket
let udp_socket = UdpSocket::bind("0.0.0.0:0").unwrap();
udp_socket.connect("224.0.0.1:3400").expect("Connection failed");
// Create FLUTE Sender
let tsi = 1;
let oti = Default::default();
let config = Default::default();
let mut sender = Sender::new(tsi, &oti, &config);
// Add object(s) (files) to the FLUTE sender
let obj = ObjectDesc::create_from_buffer(b"hello world", "text/plain",
&url::Url::parse("file:///hello.txt").unwrap(), 1, None, None, Cenc::Null, true, None, true).unwrap();
sender.add_object(obj);
// Always call publish after adding objects
sender.publish(SystemTime::now());
// Send FLUTE packets over UDP/IP
while let Some(pkt) = sender.read(SystemTime::now()) {
udp_socket.send(&pkt).unwrap();
std::thread::sleep(std::time::Duration::from_millis(1));
}
UDP/IP Multicast files receiver
Receive files from a UDP/IP network
use flute::receiver::{writer, MultiReceiver, UDPEndpoint};
use std::net::UdpSocket;
use std::time::SystemTime;
use std::rc::Rc;
// Create UDP/IP socket to receive FLUTE pkt
let endpoint = UDPEndpoint::new(None, "224.0.0.1".to_string(), 3400);
let udp_socket = UdpSocket::bind(format!("{}:{}", endpoint.destination_group_address, endpoint.port)).expect("Fail to bind");
// Create a writer able to write received files to the filesystem
let writer = Rc::new(writer::ObjectWriterFSBuilder::new(&std::path::Path::new("./flute_dir"))
.unwrap_or_else(|_| std::process::exit(0)));
// Create a multi-receiver capable of de-multiplexing several FLUTE sessions
let mut receiver = MultiReceiver::new(writer, None, false);
// Receive pkt from UDP/IP socket and push it to the FLUTE receiver
let mut buf = [0; 2048];
loop {
let (n, _src) = udp_socket.recv_from(&mut buf).expect("Failed to receive data");
let now = SystemTime::now();
receiver.push(&endpoint, &buf[..n], now).unwrap();
receiver.cleanup(now);
}
Application-Level Forward Erasure Correction (AL-FEC)
The following error recovery algorithms are supported
- No-code
- Reed-Solomon GF 2^8
- Reed-Solomon GF 2^8 Under Specified
- Reed-Solomon GF 2^16
- Reed-Solomon GF 2^m
- RaptorQ
- Raptor
The Oti
module provides an implementation of the Object Transmission Information (OTI)
used to configure Forward Error Correction (FEC) encoding in the FLUTE protocol.
use flute::sender::Oti;
use flute::sender::Sender;
// Reed Solomon 2^8 with encoding blocks composed of
// 60 source symbols and 4 repair symbols of 1424 bytes per symbol
let oti = Oti::new_reed_solomon_rs28(1424, 60, 4).unwrap();
let mut sender = Sender::new(1, &oti, &Default::default());
Content Encoding (CENC)
The following schemes are supported during the transmission/reception
- Null (no compression)
- Deflate
- Zlib
- Gzip
Files multiplex / Blocks interleave
The FLUTE Sender is able to transfer multiple files in parallel by interleaving packets from each file. For example:
Pkt file1 -> Pkt file2 -> Pkt file3 -> Pkt file1 -> Pkt file2 -> Pkt file3 ...
The Sender can interleave blocks within a single file. The following example shows Encoding Symbols (ES) from different blocks (B) are interleaved. For example:
(B 1,ES 1)->(B 2,ES 1)->(B 3,ES 1)->(B 1,ES 2)->(B 2,ES 2)...
To configure the multiplexing, use the Config
struct as follows:
use flute::sender::Sender;
use flute::sender::Config;
let config = Config {
// Transfer a maximum of 3 files in parallel
multiplex_files: 3,
// Interleave a maximum of 3 blocks within each file
interleave_blocks: 3,
..Default::default()
};
let mut sender = Sender::new(1, &Default::default(), &config);
Python bindings
Installation
pip install flute-alc
Example
Flute Sender python example
from flute import sender
# Flute Sender config parameters
sender_config = sender.Config()
# Object transmission parameters (no_code => no FEC)
# encoding symbol size : 1400 bytes
# Max source block length : 64 encoding symbols
oti = sender.Oti.new_no_code(1400, 64)
# Create FLUTE Sender
flute_sender = sender.Sender(1, oti, sender_config)
# Transfer a file
flute_sender.add_file("/path/to/file", 0, "application/octet-stream", None, None)
flute_sender.publish()
while True:
alc_pkt = flute_sender.read()
if alc_pkt == None:
break
#TODO Send alc_pkt over UDP/IP
Flute Receiver python example
from flute import receiver
# Write received objects to a destination folder
receiver_writer = receiver.ObjectWriterBuilder("/path/to/dest")
# FLUTE Receiver configuration parameters
receiver_config = receiver.Config()
tsi = 1
# Create a FLUTE receiver with the specified endpoint, tsi, writer, and configuration
udp_endpoint = receiver.UDPEndpoint("224.0.0.1", 1234)
flute_receiver = receiver.Receiver(udp_endpoint, tsi, receiver_writer, receiver_config)
while True:
# Receive LCT/ALC packet from UDP/IP multicast (Implement your own receive_from_udp_socket() function)
# Note: FLUTE does not handle the UDP/IP layer, you need to implement the socket reception mechanism yourself
pkt = receive_from_udp_socket()
# Push the received packet to the FLUTE receiver
flute_receiver.push(bytes(pkt))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for flute_alc-1.1.1-pp39-pypy39_pp73-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 577285ba821938063f7192e3025c1ca17821d119520bc12b6dfe212b8a164eb3 |
|
MD5 | 74d6da781cd73011db84349cf24f35dc |
|
BLAKE2b-256 | f367000b162ac267d056a012a83437c1aa03087ba5f7b1a38819d13fb65e2c87 |
Hashes for flute_alc-1.1.1-pp38-pypy38_pp73-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2e2eb62e69b75a0225770b558d79cfe0a661ec4edae06ca06e6f3f94c6fa7c0c |
|
MD5 | b12354eba706295c12d4c59bfc475297 |
|
BLAKE2b-256 | ad481c9bb8ebba720f843b4efb23eb8a4112ce28d2e84155ecf3f818e2aea2db |
Hashes for flute_alc-1.1.1-cp311-cp311-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6066d2713f881e502deb3cd501d8be8091d5e6b397944b7aa342129ea6228207 |
|
MD5 | 6b509ba2fce636fd96920f97c5f825c5 |
|
BLAKE2b-256 | 13e69a026538b073bee5c8fcbe1ca67a9da8d08c87c474ab0ad17d90914cca0c |
Hashes for flute_alc-1.1.1-cp310-cp310-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ec45ae387d40c92fadc3dbae9605859d2c179ca8927acee983081fa2e801c365 |
|
MD5 | cf6ba5db77d524e7564a949bc69920b0 |
|
BLAKE2b-256 | 3c7fc41e7f98495056de8cc6d5a5df6160132b66e8999c3d129473f41313cb70 |
Hashes for flute_alc-1.1.1-cp39-cp39-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4b17bd16bd32c89dc5544128c897b2c539069b5c02c5b5136ba398b0b55bbca5 |
|
MD5 | 2925a9b0cb1b7aa04d5ead95a7cd9260 |
|
BLAKE2b-256 | 09bab1f6396e62e4190bb016adae4f3b1fb984d986b6c787bf7306e164e45b8b |
Hashes for flute_alc-1.1.1-cp38-cp38-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 740b8bb41485fdf1257651fae0001675c7f66f0d026903c16f0606c19f4a70ef |
|
MD5 | 39882ce323fa342278295c81e897806f |
|
BLAKE2b-256 | 607b7069dd144ecb1c8eb851a1910e2aea643062d3ed0018d01947a2f728680c |
Hashes for flute_alc-1.1.1-cp37-cp37m-manylinux_2_34_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a886c476ed1a720014c5111e8c358179c040225b726aa3ea2458aa387eb1b0b2 |
|
MD5 | f0230b676548076a09ed6582d17337a9 |
|
BLAKE2b-256 | 5fdcf70e11225278315bdccdc5fcfb80c6df4a383ec46f7ee0d20c21f664ab91 |