threefive 2.4.97

The Undisputed Heavyweight Champion of SCTE-35. The Belts have been Unified.

| cli | lib | examples | hls | encoding | sixfix | sidecar files | xml | iodisco.com/scte35 |

threefive is SCTE-35.

SCTE-35 Parser. SCTE-35 Decoder. SCTE-35 Encoder. MPEGTS SCTE-35 Packet Injection. HLS Manifest and Segment SCTE-35 Parser. HLS SCTE-35 Injection. SCTE-35 Xml and Xml+bin Decoder/ Encoder for DASH.

Latest version is v2.4.97

• Updated to latest version of new_reader for Zero packet loss Multicast (tested on OpenBSD and Debian).
• The cli has been Upgraded and tuned.
• Xml Node Namespaces and attribute namespaces are now configurable.
• Turned SegmentationDescriptor.segmentation_message back on.
• Fix UMID upid encoding
• Added a Super Amazing HLS SCTE-35 Parser to the threefive cli..
• Added a HLS SCTE-35 Encoder to the threefive cli..
• MPEGTS Packet Injection added to the threefive cli..
  • Sidecar files explained
• sixfix : ffmpeg turns SCTE-35 streams to bin data (0x06), sixfix switches them back to SCTE-35 (0x86).
• Xml: Experimental support for SCTE-35 xml as an input and output ( DASH SCTE-214)

Documentation

• threefive is developed and tested on OpenBSD and Debian Sid.

Latest docs are always available in the python shell

>>>> from threefive import TimeSignal
>>>> help(TimeSignal)

install

• python3

python3 -mpip install threefive

• pypy3

pypy3 -mpip install threefive

• super cool examples

• New! threefive: more OO than you know

• answers to some common problems with ffmpeg and SCTE-35

Versions and Releases

Every time I fix a bug or add a feature, I do a new release.
This makes tracking down bugs and stuff much easier.
Keep up, I do releases for reasons.

a@slow:~/threefive$ threefive version
2.4.81
a@slow:~/threefive$ 

• Release versions are odd.
• Unstable testing versions are even.

---

cli

• The New Cli

• Decode SCTE-35 HLS

• Encode SCTE-35 HLS

• MPEGTS (proy,pts,sidecar,sixfix,show,xml)

lib

• New! threefive: more OO than you know
• Parse SCTE-35 programmatically with threefive (write some code)
• How to Use threefive.Cue
• Trigger on SCTE-35 Events using threefive.Stream

online parser

• iodisco.com/scte35 the only accurate online SCTE-35 parser (powered by threefive).

examples

• SCTE-35 code Examples

decryption

• Automatic AES decryption with threefive.Segment

webvtt

• Display SCTE-35 Data as WebVTT subtitles in Video

dash

• A minimal Dash SCTE-35 mpd parser
• Experimental Dash SCTE-214 Support Now available in threefive 2.4.81 (updated 10/15/2024)

encoding

• Encoding
• Encoding | more_
• JSON to SCTE-35 Encoding

Cue Class

• src cue.py
• The threefive.Cue class decodes a SCTE35 binary, base64, or hex encoded string.

class Cue(threefive.base.SCTE35Base)
 |  Cue(data=None, packet_data=None)

 |  __init__(self, data=None, packet_data=None)
 |      data may be packet bites or encoded string
 |      packet_data is a instance passed from a Stream instance

• Cue.decode()

 |  decode(self)
 |      Cue.decode() parses for SCTE35 data

• After Calling cue.decode() the instance variables can be accessed via dot notation.

    >>>> cue.command
    {'calculated_length': 5, 'name': 'Time Signal', 'time_specified_flag': True, 'pts_time': 21695.740089}

    >>>> cue.command.pts_time
    21695.740089

    >>>> cue.info_section.table_id

    '0xfc'

• Cue.get()

 |  get(self)
 |      Cue.get returns the SCTE-35 Cue
 |      data as a dict of dicts.

Cue.get() Example

>>> from threefive import Cue
>>> cue = Cue('0XFC301100000000000000FFFFFF0000004F253396')
>>> cue.decode()
True
>>> cue
{'bites': b'\xfc0\x11\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\x00\x00\x00O%3\x96',
'info_section': {'table_id': '0xfc', 'section_syntax_indicator': False, 'private': False, 'sap_type': '0x3',
'sap_details': 'No Sap Type', 'section_length': 17, 'protocol_version': 0, 'encrypted_packet': False,
'encryption_algorithm': 0, 'pts_adjustment_ticks': 0, 'pts_adjustment': 0.0, 'cw_index': '0x0', 'tier': '0xfff',
'splice_command_length': 4095, 'splice_command_type': 0, 'descriptor_loop_length': 0, 'crc': '0x4f253396'},
'command': {'command_length': None, 'command_type': 0, 'name': 'Splice Null'},
'descriptors': [], 'packet_data': None}

• Cue.get() omits cue.bites and empty values

>>> cue.get()
{'info_section': {'table_id': '0xfc', 'section_syntax_indicator': False,'private': False, 'sap_type': '0x3',
'sap_details': 'No Sap Type', 'section_length': 17, 'protocol_version': 0, 'encrypted_packet': False,
'encryption_algorithm': 0, 'pts_adjustment_ticks': 0, 'pts_adjustment': 0.0, 'cw_index': '0x0', 'tier': '0xfff',
'splice_command_length': 4095, 'splice_command_type': 0, 'descriptor_loop_length': 0, 'crc': '0x4f253396'},
'command': {'command_type': 0, 'name': 'Splice Null'},
'descriptors': []}

• Cue.get_descriptors()

 |  get_descriptors(self)
 |      Cue.get_descriptors returns a list of
 |      SCTE 35 splice descriptors as dicts.

• Cue.get_json()

 |  get_json(self)
 |      Cue.get_json returns the Cue instance
 |      data in json.

• Cue.show()

 |  show(self)
 |      Cue.show prints the Cue as JSON

• Cue.to_stderr()

 |  to_stderr(self)
 |      Cue.to_stderr prints the Cue

Stream Class

• src stream.py

• The threefive.Stream class parses SCTE35 from Mpegts.

• Supports:

  • File and Http(s) and Udp and Multicast protocols.
  • Multiple Programs.
  • Multi-Packet PAT, PMT, and SCTE35 tables.

• threefive tries to include pid, program, anf pts of the SCTE-35 packet.

class Stream(builtins.object)
 |  Stream(tsdata, show_null=True)
 |
 |  Stream class for parsing MPEG-TS data.

|  __init__(self, tsdata, show_null=True)
|
|      tsdata is a file or http, https,
|       udp or multicast url.
|
|      set show_null=False to exclude Splice Nulls

• Stream.decode(func=show_cue)

|  decode(self, func=show_cue)
|      Stream.decode reads self.tsdata to find SCTE35 packets.
|      func can be set to a custom function that accepts
|      a threefive.Cue instance as it's only argument.

Stream.decode Example

import sys
from threefive import Stream
>>>> Stream('plp0.ts').decode()

• Pass in custom function

• func should match the interface func(cue)

Stream.decode with custom function Example

import sys
import threefive

def display(cue):
   print(f'\033[92m{cue.packet_data}\033[00m')
   print(f'{cue.command.name}')

def do():
   sp = threefive.Stream(tsdata)
   sp.decode(func = display)

if __name__ == '__main__':
    do()

---

• Stream.decode_next()

|  decode_next(self)
|      Stream.decode_next returns the next
|      SCTE35 cue as a threefive.Cue instance.

Stream.decode_next Example

"""
Stream.decode_next example.
decode_next returns the Cue every time a Cue is found.

This uses a while loop to pull the Cues from a mpegts stream.
When a Cue is found, if it's a Time Signal,
cue.command.command_type=6, print Cue.command.
You can filter on any var in the SCTE-35 Cue.
"""

import sys
import threefive
from new_reader import reader

def do():
    arg = sys.argv[1]
    with reader(arg) as tsdata:
        st = threefive.Stream(tsdata)
        while True:
            cue = st.decode_next()
            if not cue:
                return False
            if cue:
                if cue.command.command_type ==6:
                    print(cue.command)


if __name__ == "__main__":
    do()

• Stream.proxy(func = show_cue)

  • Writes all packets to sys.stdout.

  • Writes scte35 data to sys.stderr.

|  decode(self, func=show_cue_stderr)
|      Stream.decode_proxy writes all ts packets are written to stdout
|      for piping into another program like mplayer.
|      SCTE-35 cues are printed to stderr.

Stream.proxy Example

import threefive
sp = threefive.Stream('https://futzu.com/xaa.ts')
sp.decode_proxy()

• Pipe to mplayer

$ python3 proxy.py | mplayer -

---

• Stream.show()

|  show(self)
|   List programs and streams and info for MPEGTS

Stream.show() Example

>>>> from threefive import Stream
>>>> Stream('https://slo.me/plp0.ts').show()

    Service:    fancy ˹
    Provider:   fu-corp
    Pcr Pid:    1051[0x41b]
    Streams:
                Pid: 1051[0x41b]        Type: 0x1b AVC Video
                Pid: 1052[0x41c]        Type: 0x3 MP2 Audio
                Pid: 1054[0x41e]        Type: 0x6 PES Packets/Private Data
                Pid: 1055[0x41f]        Type: 0x86 SCTE35 Data

Need to verify your splice points?

• Try cue2vtt.py in the examples.

  • cue2vtt.py creates webvtt subtitles out of SCTE-35 Cue data

• use it like this

pypy3 cue2vtt.py video.ts | mplayer video.ts -sub -

---

threefive is now addressable TV compatible

           "tag": 2,
          "descriptor_length": 31,
          "name": "Segmentation Descriptor",
          "identifier": "CUEI",
          "components": [],
          "segmentation_event_id": "0x065eff",
          "segmentation_event_cancel_indicator": false,
          "segmentation_event_id_compliance_indicator": true,
          "program_segmentation_flag": true,
          "segmentation_duration_flag": false,
          "delivery_not_restricted_flag": true,
          "segmentation_message": "Call Ad Server",   < --- Boom
          "segmentation_upid_type": 12,
          "segmentation_upid_type_name": "MPU",
          "segmentation_upid_length": 16,
          "segmentation_upid": {
              "format_identifier": "ADFR",	<--- Boom
              "private_data": "0x0133f10134b04f065e060220",
              "version": 1,                            <---- Boom
              "channel_identifier": "0x33f1",                  <---- Boom
              "date": 20230223,                         <---- Boom
              "break_code": 1630,                       <---- Boom
              "duration": "0x602"                <---- Boom
          },
          "segmentation_type_id": 2,         <----  Boom
          "segment_num": 0,
          "segments_expected": 0
      },

---

Custom charsets for UPIDS aka upids.charset

Specify a charset for Upid data by setting threefive.upids.charset issue #55

• default charset is ascii
• python charsets info Here
• setting charset to None will return raw bytes.

Example Usage:

>>> from threefive import Cue,upids
>>> i="/DBKAAAAAAAAAP/wBQb+YtC8/AA0AiZDVUVJAAAD6X/CAAD3W3ACEmJibG5kcHBobkQCAsGDpQIAAAAAAAEKQ1VFSRSAIyowMljRk9c="

>>> upids.charset
'ascii'
>>> cue=Cue(i)
>>> cue.decode()
ascii
True
>>> cue.descriptors[0].segmentation_upid
'bblndpphnD\x02\x02���\x02\x00\x00'

>>> upids.charset="utf16"
>>> cue.decode()
utf16
True
>>> cue.descriptors[0].segmentation_upid
'扢湬灤桰䑮Ȃ菁ʥ\x00'

Custom Private Splice Descriptors ( new! )

threefive now supports custom private splice descriptors, right out the box.

• The first byte of the descriptor is read as an int for the Descriptor tag
• The second byte is read as an int for the desciptor length
• The next four bytes are read as ASCII for the Identifier
• remanining bytes are returned as private data

from threefive import Cue, TimeSignal
from threefive.descriptors import SpliceDescriptor

• make a Cue

c = Cue()

• add a Time Signal

c.command = TimeSignal()
c.command.time_specified_flag=True
c.command.pts_time=1234.567890

• add Splice Descriptor

sd = SpliceDescriptor()
sd.tag = 47
sd.identifier ='fufu'
sd.private_data = b'threefive kicks ass'
c.descriptors.append(sd)

• encode

c.encode()
'/DAvAAAAAAAAAP/wBQb+Bp9rxgAZLxdmdWZ1dGhyZWVmaXZlIGtpY2tzIGFzc1m+EsU='

• show

c.show()
{
    "info_section": {
        "table_id": "0xfc",
        "section_syntax_indicator": false,
        "private": false,
        "sap_type": "0x03",
        "sap_details": "No Sap Type",
        "section_length": 47,
        "protocol_version": 0,
        "encrypted_packet": false,
        "encryption_algorithm": 0,
        "pts_adjustment_ticks": 0,
        "cw_index": "0x0",
        "tier": "0xfff",
        "splice_command_length": 5,
        "splice_command_type": 6,
        "descriptor_loop_length": 25,
        "crc": "0x59be12c5"
    },
    "command": {
        "command_length": 5,
        "command_type": 6,
        "name": "Time Signal",
        "time_specified_flag": true,
        "pts_time": 1234.56789,
        "pts_time_ticks": 111111110
    },
    "descriptors": [
        {
            "tag": 47,
            "descriptor_length": 23,
            "identifier": "fufu",
            "private_data": [
                116,
                104,
                114,
                101,
                101,
                102,
                105,
                118,
                101,
                32,
                107,
                105,
                99,
                107,
                115,
                32,
                97,
                115,
                115
            ]
        }
    ]
}

• the custom Splice Descriptor

c.descriptors[0]

{'tag': 47, 'descriptor_length': 23, 'name': None, 'identifier': 'fufu', 'bites': None, 'provider_avail_id': None, 'components': None, 'private_data': b'threefive kicks ass'}

• Cool dictionaary comprehension to print the Splice Descriptor with only relevant values

{print(f'{k} = {v}') for k,v in vars(c.descriptors[0]).items() if v is not None}

tag = 47
descriptor_length = 23
identifier = fufu
private_data = b'threefive kicks ass'

Powered by threefive

⚡ x9k3: SCTE-35 HLS Segmenter and Cue Inserter.
⚡ m3ufu: SCTE-35 m3u8 Parser.
⚡ adbreak2 a cli tool that quickly and easily generates SCTE-35 Cues for HLS and stuff.
⚡ Ultra Mega Zoom Zoom ABR HLS segmenter and SCTE-35 inserter.
⚡ POIS Server is Super Cool.
⚡ bpkio-cli: A command line interface to the broadpeak.io APIs.
⚡ amt-play uses x9k3.

threefive | more

⚡ Diagram of a threefive SCTE-35 Cue.
⚡ ffmpeg and threefive and SCTE35 and Stream Type 0x6 bin data.
⚡ Issues and Bugs and Feature Requests will be considered. Please don't make me regret it.
⚡ NEW! threefive now has experimental DVB DAS Support ETSI TS 103 752-1