talk2stat

A Python package for bidirectional communication with statistical software

talk2stat enables programs (such as LaTeX through runcode, Node.js, or any Python application) to communicate seamlessly with statistical packages. It uses sockets for client-server communication and bi-directional pipes for program I/O.

Supported languages: R, Julia, Matlab, Python

Key features:

• Bidirectional communication with multiple statistical backends
• Socket-based client-server architecture (localhost only)
• Configurable timeouts and debug logging
• Comprehensive test suite (35+ tests)
• Modern Python packaging (pyproject.toml + setup.py)

Requirements

• Python: 3.8 or higher
• Dependencies: pexpect
• Platform: Unix-like systems (Linux, macOS); Windows requires WSL

Installation

From PyPI

pip install talk2stat

Or in a virtual environment (recommended):

python -m venv myenv
source myenv/bin/activate
pip install talk2stat

From source

git clone <repository>
cd talk2stat
pip install -e .

With test dependencies

pip install -e ".[test]"

Quick Start

Server Setup

Create a configuration file (e.g., julia.config):

[SERVERCONFIG]
PORT = 54345
DEBUGFILE = debug.log
PIPETIMEOUT = 60
IDLE_TIMEOUT = 60
CONNECT_TIMEOUT = 10

Start the server:

from talk2stat import server
server('/path/to/config/dir', 'julia')

Client Usage

from talk2stat import client
client('/path/to/config/dir', 'julia', '```1 + 1```')  # Inline code
client('/path/to/config/dir', 'julia', '/path/to/script.jl')  # File
client('/path/to/config/dir', 'julia', 'QUIT')  # Terminate server

Configuration

Each language requires a config file (<language>.config) in your project directory.

Required settings:

• PORT — Socket port number (e.g., 54345)
• DEBUGFILE — Log file name (e.g., debug.log)

Optional settings (v0.1.7+):

• PIPETIMEOUT — Code execution timeout in seconds (default: 60)
• IDLE_TIMEOUT — Client idle timeout in seconds (default: 60)
• CONNECT_TIMEOUT — Connection timeout in seconds (default: 10)

Example Configuration

[SERVERCONFIG]
PORT = 54345
DEBUGFILE = julia_debug.log
PIPETIMEOUT = 60
IDLE_TIMEOUT = 120
CONNECT_TIMEOUT = 15

Testing

Run all tests

pytest

Run with verbose output

pytest -v

Run specific test class

pytest tests/test_talk2stat.py::TestTimeoutDefaults -v

Generate coverage report

pip install -e ".[dev]"
pytest --cov=talk2stat --cov-report=html

Test suite covers:

• Argument validation for all supported languages
• Configuration file reading and defaults
• Timeout configuration and fallback behavior
• Error handling (connection refused, timeouts, missing files)
• Edge cases and parametrized scenarios

See tests/README.md for detailed testing documentation.

Architecture

Communication Flow

User Program
    ↓
Client (socket) ←→ Server (socket)
                       ↓
                  Statistical Package
                  (via pexpect pipe)

Key Functions

• server(dirname, language, doFork=True, messages=False) — Start server for statistical package
• client(dirname, language, inputfile) — Send code/commands to server
• check_server_args() — Validate server arguments
• check_client_args() — Validate client arguments
• read_config() — Load configuration file

Timeout Management (added in v0.1.7)

Timeout Management

• IDLE_TIMEOUT — Server drops idle clients after N seconds (prevents zombie connections)
• CONNECT_TIMEOUT — Client connection timeout (default 10s)
• PIPETIMEOUT — Code execution timeout (already existed, now configurable)

All timeouts are user-configurable via config files and have sensible defaults.

Robustness Improvements

• Socket timeouts prevent indefinite blocking
• Configurable per-language, per-project
• Graceful handling of connection errors
• Detailed logging for debugging

Usage Examples

LaTeX Integration (runcode)

\begin{code}{julia}
x = randn(100, 100)
mean(x)
\end{code}

runcode uses talk2stat to execute Julia code and embed results.

Python Integration

import talk2stat

# Server runs in background (forked)
talk2stat.server('/projects/myproject', 'julia')

# Client sends code
talk2stat.client('/projects/myproject', 'julia', 'example.jl')

# Stop server
talk2stat.client('/projects/myproject', 'julia', 'QUIT')

Inline Code Execution

# Code between ``` ``` is executed directly
talk2stat.client('/config', 'R', '```plot(1:10)```')

Security Considerations

• Localhost only: Sockets bind to 127.0.0.1 (no network exposure)
• Single-user: Designed for single-user projects (no authentication)
• No token authentication: Not needed for local use, but would require modification for network deployment

Troubleshooting

Connection refused:

• Ensure server is running: talk2stat.server()
• Check port number in config file
• Verify config file exists and is readable

Timeout errors:

• Increase PIPETIMEOUT for slow computations
• Increase IDLE_TIMEOUT if clients are slow to respond
• Check debug log file for details

Debug logging:

• Check DEBUGFILE (specified in config) for server logs
• Run with messages=True for verbose console output:

  talk2stat.server('/config', 'julia', messages=True)
  

Development

Adding a New Statistical Backend

1. Add language to check_server_args() and check_client_args()
2. Define prompt character and executable command
3. Specify how to run files (e.g., source() for R, include() for Julia)
4. Add test cases

Contributing

1. Write tests first (see tests/README.md)
2. Run test suite: pytest -v
3. Ensure coverage: pytest --cov=talk2stat
4. Follow existing code style

License

MIT License (see LICENSE file)

Citation

If you use talk2stat in research, please cite:

Bar, H. "How to Talk to Your Programs: Interprocess Communication for Data Scientists", The American Statistician (Teacher's Corner)

Documentation

• README.md — This file; user guide and quick start
• CONFIGURATION.md — Detailed configuration reference
• tests/README.md — Testing guide and examples
• llms.txt — AI-targeted technical documentation (for code analysis and generation)