Skip to main content

Sign to speech (StoS):

Converting the American sign language into speech, and vice versa.

Project Description:

StoS is a python library, created by a team from the Egyptian information technology institute (ITI) as their graduation project.

The main purpose of the library is converting the American sign language to speech, and vice versa, this is achieved by using deep learning, NLP, and computer vision. The first release of the project is limited only to the following 50 common English word:

[age, book, call, car, day, Egypt, English, enjoy, every, excuse, football, forget, fun, good, hate, have, hello, help, 
holiday, I, am, love, meet, month, morning, my, NA, name, nice, no, not, number, okay, picture, play, read, ride, run, 
sorry, speak, sport, take, thanks, time, today, understand, what, when, where, year, yes, you, your]

StoS is divided into two main pipelines:

1. sign to speech:

In sign to speech pipeline, the primary role is converting the streamed American sign language videos into a set of sentences, or voice, using NLP pipeline, after being predicated by our DL model.

The output of that DL model is a list of words, so in order to construct a basic English sentences, these words will pass through NLP model for modification, and sentence construction and correction. As a result of the previous processes, we will have an English text, which is grammatically correct, and can be converted into audio clip up to the user preferences.

2. speech to sign:

This pipeline controls the translation process of the streamed speech into American sign language video.

Work Flow

Model Structure

Training Result

Prerequisites

  • python 3.8
  • OpenCV

Installation

pip install stos

How to use

To Convert Sign to Speech

import os
from cv2 import cv2
from stos.sign_to_speech.sign_to_speech import SignToSpeech

sts = SignToSpeech(1, 20, os.path.join('model', 'cv_model.hdf5'), os.path.join('model', 'names'),
                   display_keypoint=True, display_window=False)
for word, frame in sts.start_pipeline():
    cv2.imshow('frame', frame)
    key = cv2.waitKey(1)
    if key == ord('q'):
        exit(0)

To Convert Speech to Sign

from cv2 import cv2
from stos.speech_to_sign.speech_to_sign import SpeechToSign

sts = SpeechToSign(10)
for frame in sts.start_pipeline():
    cv2.imshow('frame', frame)
    key = cv2.waitKey(1)
    if key == ord('q'):
        exit(0)

Contributors:

  • Abdelaziz Gamal Mousa
  • Amira Mohammed Abd Elmajeed
  • Habiba Khaled Ibrahim
  • Sara Hesham Hassan
  • Amal Abdalah Saad
  • Abdallah Gaber Ahmed Elsharawy
  • Ahmed Mohamed Ibrahem Sayed
  • Mohamed Hany Ahmed Abdelaty
  • Mostafa Gaber Aboelghit Abdou Elraghy
  • Muhammad Ayman Muhammad El-sawy
  • Ahmed Tarek Mansour Al Ashry

Licence

MIT Licence

Metadata

Release files for stos 1.0.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for stos 1.0.2
File Size Uploaded Provenance Actions
stos-1.0.2.tar.gz 552.1 kB Without provenance Details for stos-1.0.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for stos 1.0.2
File Interpreter ABI Platform Actions
stos-1.0.2-py3-none-any.whl Python 3 none any Details for stos-1.0.2-py3-none-any.whl

Total release size: 566.0 kB

Release files / stos-1.0.2.tar.gz

Download URL stos-1.0.2.tar.gz
Size 552.1 kB
Tags Source
SHA-256 checksum
How to use checksums
76b5d5b83b3bad834f892cee850bafd8493dff0e31949cc53c94bd56708d79c7
BLAKE2b-256 checksum
How to use checksums
79a54d7817942eeee7ff66b25cbd6fa8cfb34525e3e5b1b411ad75e106362332
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.7.1 importlib_metadata/4.10.0 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.10

Release files / stos-1.0.2-py3-none-any.whl

Download URL stos-1.0.2-py3-none-any.whl
Size 13.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1b2bc8c1c994e710e4211e3fad1f9e8aa68880d10b53e50783865e6a38a66f10
BLAKE2b-256 checksum
How to use checksums
ab2b39cd3f9dd7f371e3c9a9500b260934b08081b444d936b02fa25e261cd473
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.7.1 importlib_metadata/4.10.0 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.10

Security for stos 1.0.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page