codibox 1.3.0

A reusable package for executing Python code in Docker containers and extracting charts/images

Codibox - Safe Code Execution Package

A reusable Python package for executing Python code in Docker containers or on the host machine, with automatic chart/image extraction and result processing.

📸 Screenshots

Streamlit Chart Generator Example

The Streamlit Chart Generator example demonstrates an interactive dashboard for generating charts from data using codibox.

🎯 Features

• Dual Backend Support: Choose between Docker (secure, isolated) or Host (fast, direct execution)
• Automatic Image Extraction: Automatically finds and extracts charts/images from code execution
• Markdown Processing: Converts notebook output to markdown with embedded images
• CSV File Extraction: Automatically extracts generated CSV/Excel files
• Dependency Management: Auto-installs required packages for host mode
• Error Handling: Comprehensive error handling with fallbacks
• Backward Compatible: DockerCodeExecutor alias maintained for existing code

📦 Installation

From PyPI (Recommended)

pip install codibox

From Source

# Clone the repository
git clone https://github.com/otmane-elbourki/codibox.git
cd codibox

# Install in editable mode
pip install -e .

Development Installation

pip install -e ".[dev]"

🚀 Quick Start

Docker Backend (Default - Secure)

from codibox import CodeExecutor

# Initialize executor with Docker backend
executor = CodeExecutor(backend="docker", container_name="sandbox")

# Execute code
result = executor.execute(
    code="""
    import matplotlib.pyplot as plt
    import numpy as np
    
    x = np.linspace(0, 10, 100)
    y = np.sin(x)
    
    plt.figure(figsize=(10, 6))
    plt.plot(x, y, label='sin(x)')
    plt.title('Sine Wave')
    plt.savefig('temp_code_files/chart.png')
    plt.close()
    """
)

# Check results
if result.success:
    print(f"✅ Success! Generated {len(result.images)} images")
    for img in result.images:
        print(f"  📊 {img}")
    print(f"📄 Markdown: {result.markdown[:100]}...")
else:
    print(f"❌ Failed: {result.stderr}")

Host Backend (Fast - Direct Execution)

from codibox import CodeExecutor

# Initialize executor with Host backend (fast mode)
executor = CodeExecutor(backend="host", auto_install_deps=True)

# Execute code (same API as Docker backend)
result = executor.execute(
    code="""
    import matplotlib.pyplot as plt
    import numpy as np
    
    x = np.linspace(0, 10, 100)
    y = np.cos(x)
    
    plt.figure(figsize=(10, 6))
    plt.plot(x, y)
    plt.title('Cosine Wave')
    plt.savefig('temp_code_files/cosine.png')
    plt.close()
    """
)

# Results work the same way
print(f"Images: {result.images}")
print(f"Success: {result.success}")

Backward Compatibility

# Old code still works!
from codibox import DockerCodeExecutor

executor = DockerCodeExecutor(container_name="sandbox")
result = executor.execute(code="...")

📋 API Reference

CodeExecutor

Main class for executing Python code with dual backend support.

Initialization

CodeExecutor(
    backend: str = "docker",              # "docker" or "host"
    container_name: str = "sandbox",      # Docker container name (docker only)
    host_temp_dir: str = "/tmp",          # Temporary directory on host
    image_output_dir: str = "temp_code_files",  # Directory for images
    timeout: Optional[int] = 300,        # Execution timeout (seconds)
    cleanup_on_error: bool = True,       # Cleanup on error
    auto_setup: bool = False,             # Auto-setup Docker container (docker only)
    auto_install_deps: bool = True,       # Auto-install dependencies (host only)
)

Methods

execute(code, input_files=None, working_dir=None) -> ExecutionResult

Execute Python code and return results.

Parameters:

• code (str): Python code to execute
• input_files (List[str], optional): Input files to copy to container/host
• working_dir (str, optional): Working directory (default: /home/sandboxuser for Docker, temp dir for Host)

Returns: ExecutionResult object

check_container() -> bool

Check if Docker container is running (Docker backend only).

get_container_status() -> Dict[str, Any]

Get detailed status of Docker container (Docker backend only).

Returns: Dictionary with exists, running, status, image

setup_container(image_name="python_sandbox:latest", force_rebuild=False) -> bool

Set up Docker container by building image and starting container (Docker backend only).

ExecutionResult

Result object returned by execute() method.

@dataclass
class ExecutionResult:
    # Raw outputs
    markdown: str                    # Raw markdown from notebook execution
    success: bool                    # Whether execution succeeded
    images: List[str]                # List of image file paths on host
    csv_files: List[str]             # List of generated CSV/Excel files
    stdout: Optional[str]            # Standard output
    stderr: Optional[str]            # Standard error
    execution_time: Optional[float]  # Execution time in seconds
    
    # Processed outputs
    markdown_processed: Optional[str]      # Markdown with resolved image paths
    markdown_with_images: Optional[str]    # Markdown with base64-embedded images
    image_map: Optional[Dict[str, str]]    # Mapping: markdown_ref -> actual_path

ImageProcessor

Utility class for processing images from execution results.

from codibox import ImageProcessor

processor = ImageProcessor(image_base_dir="/tmp/temp_code_files")

# Find all images
images = processor.find_images()

# Process markdown to embed images
processed_markdown = processor.process_markdown(result.markdown)

# Get HTML for all images
html = processor.get_all_images_html()

🔧 Configuration Examples

Docker Backend Setup

from codibox import CodeExecutor

# Basic setup
executor = CodeExecutor(backend="docker", container_name="sandbox")

# Auto-setup container if not running
executor = CodeExecutor(
    backend="docker",
    container_name="sandbox",
    auto_setup=True  # Automatically builds/starts container
)

# Manual container setup
executor = CodeExecutor(backend="docker")
if not executor.check_container():
    executor.setup_container()

# Custom timeout
executor = CodeExecutor(backend="docker", timeout=600)  # 10 minutes

Host Backend Setup

from codibox import CodeExecutor

# Basic setup (auto-installs dependencies)
executor = CodeExecutor(backend="host")

# Disable auto-installation
executor = CodeExecutor(
    backend="host",
    auto_install_deps=False  # You must install dependencies manually
)

# Custom temp directory
executor = CodeExecutor(
    backend="host",
    host_temp_dir="/custom/tmp"
)

📊 Usage Examples

With Input Files

executor = CodeExecutor(backend="docker")

result = executor.execute(
    code="""
    import pandas as pd
    import matplotlib.pyplot as plt
    
    df = pd.read_csv('data.csv')
    plt.hist(df['age'], bins=20, edgecolor='black')
    plt.title('Age Distribution')
    plt.savefig('temp_code_files/age_dist.png')
    plt.close()
    """,
    input_files=["datasets/customer_data.csv"]
)

print(f"Generated {len(result.images)} charts")
print(f"CSV files: {result.csv_files}")

Processing Images

from codibox import CodeExecutor, ImageProcessor

executor = CodeExecutor(backend="host")
result = executor.execute(code="...")

# Process images
processor = ImageProcessor()
markdown_with_images = processor.process_markdown(result.markdown)

# Or get all images as HTML
images_html = processor.get_all_images_html()
print(images_html)

Accessing Processed Results

result = executor.execute(code="...")

# Raw markdown (with image references)
print(result.markdown)

# Processed markdown (with resolved paths)
if result.markdown_processed:
    print(result.markdown_processed)

# Markdown with embedded base64 images
if result.markdown_with_images:
    print(result.markdown_with_images)

# Image mapping
if result.image_map:
    for ref, path in result.image_map.items():
        print(f"{ref} -> {path}")

🐳 Docker Backend Details

Container Setup

The Docker backend requires a running container with required packages installed. You have three options:

Option 1: Manual Docker Setup (Recommended)

Step 1: Create and start the container:

docker run -d --name sandbox --network none --cap-drop all \
  --pids-limit 124 --tmpfs /tmp:rw,size=124M \
  python:3.10-slim sleep infinity

Step 2: Install required packages:

docker exec sandbox pip install pandas matplotlib seaborn jupyter nbconvert ipynb-py-convert

Step 3: Verify container is running:

docker ps -a | grep sandbox
# If container exists but is stopped, start it:
docker start sandbox

Step 4: Use in Python:

from codibox import CodeExecutor

executor = CodeExecutor(backend="docker", container_name="sandbox")
result = executor.execute(code="...")

Option 2: Auto-setup (Development)

The package can automatically set up the container using the included Dockerfile:

executor = CodeExecutor(backend="docker", auto_setup=True)
# Container will be built and started automatically if not running

Option 3: Use Existing Container

If you already have a container with the required packages:

executor = CodeExecutor(backend="docker", container_name="my_container")

Container Requirements

The container must have:

• Python 3.10+
• Required packages: pandas, numpy, matplotlib, jupyter, nbconvert, ipynb-py-convert
• Working directory: /tmp (used by default)

Note: The package includes a Dockerfile and requirements.txt in the docker/ directory for building a compatible container, but manual setup with python:3.10-slim is faster and recommended.

💻 Host Backend Details

Automatic Dependency Management

The Host backend automatically checks and installs required packages:

• matplotlib
• pandas
• numpy
• jupyter
• nbconvert
• ipynb-py-convert

Python 3.13 Compatibility

The Host backend automatically handles Python 3.13 SQLite issues by falling back to direct Python execution when Jupyter fails.

Security Considerations

⚠️ Important: The Host backend runs code with user permissions and has no isolation. Use Docker backend for untrusted code.

🔄 Backend Comparison

Feature	Docker Backend	Host Backend
Security	✅ Isolated, secure	⚠️ Runs with user permissions
Speed	~2-5 seconds	~0.5-2 seconds
Setup	Requires Docker	No setup needed
Dependencies	Pre-installed in container	Auto-installed on first use
Isolation	✅ Full isolation	❌ No isolation
Network	❌ No network access	✅ Full network access
Use Case	Production, untrusted code	Development, trusted code

📝 Examples

See the examples/ directory for complete usage examples:

• basic_usage.py - Simple code execution examples
• ai_analyst/ - Complete Streamlit dashboard application
• README.md - Detailed examples documentation

Running Examples

# Setup Docker container (if using Docker backend)
cd examples
./setup_example.sh

# Run basic examples
python basic_usage.py

# Run Streamlit app
cd ai_analyst
streamlit run streamlit_chart_generator.py

🧪 Testing

Run the test suite to verify both backends:

python3 test/test_both_backends.py

This will test:

• Docker backend execution
• Host backend execution
• Image generation and extraction
• Markdown processing
• Error handling

📚 Advanced Usage

Custom Working Directory

# Docker backend
result = executor.execute(
    code="...",
    working_dir="/custom/path"
)

# Host backend
executor = CodeExecutor(
    backend="host",
    host_temp_dir="/custom/tmp"
)

Error Handling

result = executor.execute(code="...")

if not result.success:
    print(f"Execution failed: {result.stderr}")
    print(f"Execution time: {result.execution_time}s")
else:
    print(f"Success! Generated {len(result.images)} images")

Container Management

executor = CodeExecutor(backend="docker")

# Check container status
status = executor.get_container_status()
print(f"Container exists: {status['exists']}")
print(f"Container running: {status['running']}")

# Setup container
if not executor.check_container():
    executor.setup_container(force_rebuild=True)

🔍 Troubleshooting

Docker Backend Issues

Container not found:

# Auto-setup container
executor = CodeExecutor(backend="docker", auto_setup=True)

# Or manually setup
executor.setup_container()

Images not extracted:

• Ensure code saves images to temp_code_files/ directory
• Check container working directory matches executor configuration

Host Backend Issues

Dependencies not installed:

# Enable auto-installation
executor = CodeExecutor(backend="host", auto_install_deps=True)

Python 3.13 SQLite errors:

• Automatically handled with fallback to direct Python execution
• No action needed

Images not found:

• Ensure temp_code_files/ directory exists (created automatically)
• Check image paths in code match image_output_dir setting

📦 Package Structure

.
├── __init__.py              # Package initialization
├── executor.py              # CodeExecutor class
├── image_processor.py       # Image processing utilities
├── type_definitions.py      # Type definitions
├── setup.py                 # Package setup configuration
├── pyproject.toml           # Modern Python packaging config
├── Makefile                 # Development commands
├── docker/
│   ├── Dockerfile          # Docker image definition
│   └── requirements.txt    # Docker dependencies
├── scripts/
│   └── bump_version.py     # Version management script
├── test/
│   └── test_both_backends.py  # Test script
├── examples/               # Usage examples
│   ├── basic_usage.py
│   └── ai_analyst/         # Streamlit dashboard example
└── .github/
    └── workflows/          # CI/CD workflows
        ├── ci.yml          # Continuous Integration
        └── publish.yml     # PyPI publishing

🗺️ Future Roadmap

Performance & Scalability

• Container Pool Executor - Pre-warmed container pool for parallel execution (target: 1-3s per execution, 10x throughput)
• FastAPI Microservice - RESTful API with async execution, job queue, and WebSocket support
• Optimizations - Shared volumes, batch operations, result caching, parallel extraction

Production Features

• Monitoring - Prometheus metrics, Grafana dashboards, health checks
• Security - API authentication, rate limiting, enhanced container hardening
• Reliability - Auto-recovery, circuit breakers, retry logic, dead letter queues
• Deployment - Docker Compose, Kubernetes manifests, auto-scaling

Current Performance

• Docker: ~2-5s per execution | Host: ~0.5-2s per execution
• Target: <1s with pool, <500ms with warm containers
• Throughput: 1 req/s → 50-100 req/s (with horizontal scaling)

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

Development Setup

# Install development dependencies (includes pre-commit hooks)
make install-dev

# Run all checks (format, lint, type-check, test)
make ci

# Individual commands
make test        # Run tests
make format      # Format code with ruff
make lint        # Lint code with ruff
make type-check  # Type checking with mypy
make build       # Build package

CI/CD

The project uses GitHub Actions for automated testing and publishing:

• Continuous Integration: Runs on every push/PR

  • Tests on Python 3.10, 3.11, 3.12
  • Code formatting and linting checks
  • Build verification

• PyPI Publishing: Automatically publishes when you create a GitHub release

  • No manual uploads needed
  • Uses trusted publishing (no API tokens required)

Pre-commit Hooks

Pre-commit hooks automatically check code quality before each commit:

• Code formatting (ruff format)
• Linting (ruff check)
• Type checking (mypy)
• Basic file validation

Install with: make install-dev or pre-commit install

Releasing a New Version

1. Bump version:

   make bump-patch   # 1.1.0 -> 1.1.1
   make bump-minor   # 1.1.0 -> 1.2.0
   make bump-major   # 1.1.0 -> 2.0.0
   

2. Commit and tag:

   git commit -am "Bump version to X.Y.Z"
   git tag -a vX.Y.Z -m "Release vX.Y.Z"
   git push && git push --tags
   

3. Create GitHub Release:

   • Go to GitHub → Releases → Draft new release
   • Select the tag (vX.Y.Z)
   • Add release notes
   • Publish (triggers automatic PyPI publish)

Version Numbering: Follow Semantic Versioning

• MAJOR (X.0.0): Breaking changes
• MINOR (0.X.0): New features, backward compatible
• PATCH (0.0.X): Bug fixes, backward compatible

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

👤 Author

Otmane El Bourki

• Email: [e-mail]
• GitHub: @otmane-elbourki

🔗 Links

• PyPI: https://pypi.org/project/codibox/
• GitHub: https://github.com/otmane-elbourki/codibox
• Issues: https://github.com/otmane-elbourki/codibox/issues