copytalker 0.0.2

Cross-modal data conversion driven asynchronous multi-voice translation system

CopyTalker

CopyTalker is a cross-modal data conversion driven asynchronous multi-voice translation system. It enables real-time speech-to-speech translation with support for multiple languages and voices, utilizing state-of-the-art machine learning models for speech recognition, translation, and synthesis.

Features

• Real-time Speech Translation: Instantly translate spoken language to another language with voice output
• Multi-language Support: Supports translation between 9 languages including English, Chinese, Japanese, Korean, French, German, Spanish, Russian, and Arabic
• Multiple TTS Engines: Kokoro (high-quality neural TTS), Edge TTS (cloud-based), pyttsx3 (offline)
• Cross-modal Conversion: Seamless conversion from speech to text to translated speech
• Asynchronous Processing: Efficient parallel processing with minimal latency
• Simple GUI: Easy-to-use Tkinter graphical interface
• Offline Capabilities: Download models for offline usage

Supported Languages

Code	Language
en	English
zh	Chinese (Simplified)
ja	Japanese
ko	Korean
fr	French
de	German
es	Spanish
ru	Russian
ar	Arabic

Installation

From PyPI (Recommended)

pip install copytalker

With All Features

# Full installation with all TTS engines
pip install copytalker[full]

# With CJK language support
pip install copytalker[full,cjk]

# Development installation
pip install copytalker[dev]

From Source

git clone https://github.com/cycleuser/CopyTalker.git
cd CopyTalker
pip install -e .[full]

System Dependencies

CopyTalker requires FFmpeg and PortAudio for audio processing:

Ubuntu/Debian:

sudo apt install ffmpeg portaudio19-dev python3-dev

Fedora:

sudo dnf install ffmpeg portaudio-devel python3-devel

macOS:

brew install ffmpeg portaudio

Windows: Download FFmpeg from https://ffmpeg.org/download.html and add to PATH.

Quick Start

Command Line Interface

# Start real-time translation (English to Chinese)
copytalker translate --target zh

# With auto-detection of source language
copytalker translate --source auto --target ja

# Specify TTS voice
copytalker translate --target zh --voice zf_xiaobei

# Use specific TTS engine
copytalker translate --target en --tts-engine edge-tts

# List available voices
copytalker list-voices --language zh

# List supported languages
copytalker list-languages

GUI Mode

# Launch graphical interface
copytalker --gui

# Or use dedicated command
copytalker-gui

Screenshots

Main Interface

The main window provides access to all settings, real-time transcription and translation displays, and control buttons including Start Translation, Stop, and Download Models.

Source Language Selection

Select the source language or choose Auto-detect to let Whisper identify the spoken language automatically.

Target Language Selection

Choose the target language for translation output.

Voice Selection

Pick a TTS voice for the target language. Voices change dynamically based on the selected target language and TTS engine.

TTS Engine Selection

Choose between Kokoro (high-quality neural), Edge TTS (cloud-based), pyttsx3 (offline), or auto (automatic best choice).

Translation Model Selection

Select between Helsinki-NLP (language-pair specific) or NLLB (multilingual, supports all language pairs including ja-zh).

Translation Device Selection

Assign the translation model to CPU or CUDA GPU to balance resources.

TTS Device Selection

Assign the TTS engine to CPU or CUDA GPU independently from the translation model to avoid GPU resource contention.

Python API

from copytalker import AppConfig, TranslationPipeline

# Configure
config = AppConfig()
config.stt.language = "auto"  # Auto-detect source language
config.translation.target_lang = "zh"  # Translate to Chinese
config.tts.engine = "kokoro"  # Use Kokoro TTS
config.tts.voice = "zf_xiaobei"  # Chinese female voice

# Create and start pipeline
pipeline = TranslationPipeline(config)

# Register callbacks for events
def on_transcription(event):
    print(f"Heard: {event.data.text}")

def on_translation(event):
    print(f"Translated: {event.data.translated_text}")

pipeline.register_callback("transcription", on_transcription)
pipeline.register_callback("translation", on_translation)

# Start translation
pipeline.start()

# ... (pipeline runs until stopped)

# Stop
pipeline.stop()

Using Context Manager

from copytalker import AppConfig, TranslationPipeline

config = AppConfig()
config.translation.target_lang = "ja"

with TranslationPipeline(config) as pipeline:
    # Pipeline is running
    input("Press Enter to stop...")
# Pipeline automatically stopped

Model Management

Pre-download Models

# Download Whisper model
copytalker download-models --whisper small

# Download Kokoro TTS model
copytalker download-models --kokoro

# Download all recommended models
copytalker download-models --all

Cache Management

# Show cache info
copytalker cache --info

# Clear all cached models
copytalker cache --clear

# Clear specific model type
copytalker cache --clear whisper

Configuration

CopyTalker can be configured via:

1. Command-line arguments
2. Environment variables
3. Configuration file (~/.config/copytalker/config.yaml)

Environment Variables

Variable	Description	Default
COPYTALKER_CACHE_DIR	Model cache directory	~/.cache/copytalker
COPYTALKER_DEVICE	Compute device (cpu/cuda/auto)	auto
COPYTALKER_CONFIG	Config file path	~/.config/copytalker/config.yaml

Configuration File Example

audio:
  sample_rate: 16000
  vad_aggressiveness: 3

stt:
  model_size: small
  device: auto

translation:
  target_lang: zh

tts:
  engine: kokoro
  voice: zf_xiaobei
  speed: 1.0

debug: false

Architecture

CopyTalker follows a modular pipeline architecture:

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  Audio Capture  │────▶│  Speech-to-Text │────▶│   Translation   │────▶│  Text-to-Speech │
│    (VAD)        │     │   (Whisper)     │     │ (Helsinki/NLLB) │     │    (Kokoro)     │
└─────────────────┘     └─────────────────┘     └─────────────────┘     └─────────────────┘

1. Audio Capture: Records audio with Voice Activity Detection (WebRTC VAD)
2. Speech Recognition: Transcribes using Faster-Whisper
3. Translation: Translates using Helsinki-NLP or NLLB models
4. Text-to-Speech: Synthesizes using Kokoro, Edge TTS, or pyttsx3

Development

Setup Development Environment

git clone https://github.com/cycleuser/CopyTalker.git
cd CopyTalker
pip install -e .[dev]

Running Tests

# Run all tests
pytest

# Run with coverage
pytest --cov=copytalker

# Run only unit tests
pytest tests/unit/

# Run fast tests only (skip slow)
pytest -m "not slow"

Code Quality

# Format code
black src/copytalker tests
isort src/copytalker tests

# Lint
ruff check src/copytalker

# Type checking
mypy src/copytalker

Requirements

• Python 3.9 or higher
• FFmpeg
• PortAudio (for PyAudio)
• Audio input/output capabilities
• PyTorch 2.0+ (on macOS: CPU or MPS; on Linux/Windows: CPU or CUDA)

See pyproject.toml for detailed Python package dependencies.

macOS Installation Notes

CopyTalker works on macOS (both Intel and Apple Silicon). On macOS, CUDA is not available, so PyTorch uses CPU or MPS (Apple Silicon) for inference.

If you encounter torch/numpy conflicts on macOS, install PyTorch first:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
pip install copytalker

If PyAudio fails to install on macOS, set the compiler flags:

LDFLAGS="-L$(brew --prefix portaudio)/lib" CFLAGS="-I$(brew --prefix portaudio)/include" pip install pyaudio

Linux Installation Notes

On Linux, PyAudio is compiled from source and requires the PortAudio development headers and a C compiler. Install them before running pip install:

# Ubuntu/Debian
sudo apt install ffmpeg portaudio19-dev python3-dev build-essential

# Fedora
sudo dnf install ffmpeg portaudio-devel python3-devel gcc

Agent Integration (OpenAI Function Calling)

CopyTalker exposes OpenAI-compatible tools for LLM agents:

from copytalker.tools import TOOLS, dispatch

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=TOOLS,
)

result = dispatch(
    tool_call.function.name,
    tool_call.function.arguments,
)

CLI Help

License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Acknowledgments

• faster-whisper for speech recognition
• Helsinki-NLP for translation models
• Facebook NLLB for multilingual translation
• Kokoro TTS for high-quality neural TTS
• Various TTS libraries for voice synthesis