lsat 0.0.1

LSA-T: The first continuous LSA dataset

LSA-T is the first continuous Argentinian Sign Language (LSA) dataset. It contains 14,880 sentence level videos of LSA extracted from the CN Sordos YouTube channel with labels and keypoints annotations for each signer. Videos are in 30 FPS full HD (1920x1080).

• Download link (45GB compressed)
• Visualization notebook
• Presentation paper (TO-DO)

Format

Samples are organized in directories according to the playlists and video they belong to. For each sample i there are four files:

• i.mp4: the clip corresponding to the ith line of subtitles.
• i.json contains:
  • label: the line of subtitles corresponding to the clip.
  • start: time in seconds where the subtitle starts.
  • end: time in seconds where the subtitle ends.
  • video: title of the video which the clip belongs to.
  • playlist: title of the playlist which the clip belongs to.
• i_ap.json: the raw AlphaPose results over the clip using Halpe KeyPoints in AlphaPose default output format.
• i_signer.json contains:
  • scores: for each person in the clip, the amount of "movement" in its hands. It is used to infer who is the signer.
  • roi: the considered region of interest of the clip (bounding box of the infered signer).
  • keypoints: list of keypoints for each frame of the infered signer in same format that in i_ap.json.

Usage

This repository can be installed via pip and contains the LSA_Dataset class (in lsat.dataset.LSA_Dataset module). This class inherits from the Pytorch dataset class and implements all necessary methods for using it with a Pytorch dataloader. It also manages the downloading and extraction of the database.

Also, useful transforms for the clips and keypoints are provided in lsat.dataset.transforms

Statistics and comparison with other DBs

*Data was not available for the whole PHOENIX dataset, so the table show its train set statistics.

Evaluation splits

Citation

TO-DO