synthesize-ai 0.1.1

Natural language testing platform powered by AI — transform plain English into executable tests across any framework

Synthesize

Natural language testing platform powered by AI

Stop writing test code. Start describing what you want tested.

English in → Tests out → Results back

Website • Documentation • Plugin Registry • Discord

---

What is Synthesize?

Synthesize transforms plain English descriptions into executable tests across any testing framework. No Gherkin, no keywords, no boilerplate—just describe what you want tested.

Write this:

suite: User Authentication
tests:
  - User can login with valid credentials
  - Bad password shows error message
  - Account locks after 5 failed attempts

Get working tests in:

• Robot Framework
• Playwright
• Cypress
• Selenium
• Cucumber
• pytest

With built-in:

• ✅ Self-healing when tests break
• ✅ Compliance traceability (SOC2, HIPAA, SOX)
• ✅ Cost optimization (70%+ reduction in LLM costs)
• ✅ Historical analytics and flakiness detection
• ✅ Team collaboration and knowledge sharing

---

Why Synthesize?

For QA Engineers

Stop maintaining brittle tests

• Self-healing automatically fixes broken selectors
• Multi-framework support—use what you know
• 70% less time on test maintenance

For Engineering Teams

Ship faster with confidence

• Write tests in English, not code
• Parallel execution and smart test selection
• Real-time collaboration and reviews

For Enterprises

Meet compliance requirements effortlessly

• Built-in SOC2, HIPAA, SOX mapping
• Audit trails and traceability reports
• Self-hosted deployment option

---

Quick Start

Installation

# Install Synthesize
pip install synthesize-ai

# Set your API key
export ANTHROPIC_API_KEY=sk-ant-...

# Initialize a project
synthesize init

Your First Test

Create a test file (tests/login.synth):

suite: User Login
context: Web app, login page at /auth/login

tests:
  - User can login with valid credentials
  - User sees error with invalid password
  - Session expires after 30 minutes

Generate and run:

# Generate Robot Framework tests
synthesize generate tests/login.synth

# Or generate Playwright
synthesize generate tests/login.synth --format playwright

# Execute with self-healing
synthesize run tests/login.synth --heal auto --max-retries 3

---

Features

🤖 AI-Powered Generation

• Natural Language Processing: Claude interprets your intent
• Context-Aware: Understands your application's domain
• Multi-Framework: Same intent → 6 different frameworks

🔧 Self-Healing Tests

• Automatic Repair: Fixes broken locators and assertions
• Smart Retry: Learns from failures
• Confidence Scoring: Reports healing reliability

📊 Analytics & Insights

• Flakiness Detection: Identifies unstable tests automatically
• Trend Analysis: Track test duration and failure patterns
• Predictive: ML-powered failure prediction

🔒 Security & Compliance

• OWASP ZAP Integration: Dynamic security scanning
• Secrets Detection: Prevent credential leaks
• Compliance Mapping: SOC2, HIPAA, SOX traceability

💰 Cost Optimization

• LLM Caching: 60-70% cache hit rate
• Local Models: Ollama support for simple tasks
• Smart Routing: Haiku for simple, Opus for complex
• Result: 70%+ cost reduction

👥 Team Collaboration

• Team Project Sharing: Team members see and edit shared projects
• Team Secrets Vault: Shared encrypted secrets injected at runtime
• Admin User Management: Create/manage users with support for email, Google, GitHub, SAML, LDAP auth
• Email Invites: Team invitations via Amazon SES (DKIM-verified domain)
• Role-Based Access: Admin, tester, and user roles with plan-based feature gating

---

Supported Frameworks

Framework	Status	Plugin
Robot Framework	✅ Built-in	robot
Playwright	✅ Built-in	playwright
Cypress	✅ Built-in	cypress
Selenium	✅ Built-in	selenium
Cucumber/Behave	✅ Built-in	cucumber
pytest	✅ Built-in	pytest-agents
TestCafe	🔌 Plugin Available	testcafe
Puppeteer	🔌 Plugin Available	puppeteer
Appium	🔌 Community	appium
Custom	🛠️ Build Your Own	Guide

---

Examples

Web Testing

suite: E-commerce Checkout
context: Online store, products at /products

tests:
  - User can add item to cart
  - User can apply discount code
  - User can complete checkout with credit card
  - User receives order confirmation email

API Testing

suite: User API
context: REST API at https://api.example.com
environment:
  type: api
  base_url: ${API_URL}

tests:
  - GET /users returns 200 with user list
  - POST /users creates new user
  - PUT /users/:id updates existing user
  - DELETE /users/:id returns 404 for non-existent user

Database Testing

suite: Data Migration Validation
context: MySQL database, compliance_records table
compliance: [HIPAA-164.312, SOX-302]

tests:
  - Record count matches source
  - No orphaned junction records
  - PII fields are encrypted at rest

Compliance Testing

suite: HIPAA Compliance Validation
context: Patient portal, PHI data handling
compliance: [HIPAA-164.312(a)(1), HIPAA-164.312(e)(1)]

tests:
  - PHI is encrypted in transit
  - Access logs record all PHI access
  - Session timeout enforces 15-minute limit
  - Audit trail captures all data modifications

---

CLI Reference

Test Generation & Execution

synthesize generate <test.synth>              # Generate .robot files
synthesize generate <test.synth> --format <format>  # Choose framework
synthesize run <test.synth>                   # Generate + execute
synthesize run <test.synth> --heal auto       # With self-healing
synthesize run <test.synth> --smart           # Smart test selection

Plugin Management

synthesize plugin list                       # Show installed plugins
synthesize plugin install <name>             # Install plugin
synthesize plugin create <name>              # Create new plugin
synthesize plugin formats                    # List available formats

Cost & Analytics

synthesize cost report                       # LLM cost analysis
synthesize cost stats                        # Cache statistics
synthesize analytics trend <test>            # Duration trends
synthesize analytics flaky-tests             # Identify flaky tests

Team Collaboration

synthesize sync                              # Sync shared knowledge
synthesize review submit <test.synth>         # Submit for review
synthesize review list --assigned-to-me      # My reviews

Security

synthesize security scan <test.synth>         # Security scan
synthesize security scan-secrets ./          # Detect leaked secrets

Full CLI Reference

---

Tutorials

The Developer Guide contains 12 step-by-step tutorials:

1. Your First Test — Create and run an AI-generated test in 2 minutes
2. API Testing from OpenAPI — Auto-generate tests from Swagger/Postman/DDL specs
3. Self-Healing Tests — Auto-fix broken locators and assertions
4. Compliance Auditing — SOC2/HIPAA/SOX mapping and audit reports
5. Watch Mode for TDD — Live-reload pipeline on .synth file changes
6. Performance Testing — HTTP load tests with baseline regression detection
7. CI/CD Integration — GitHub Actions, GitLab CI, Jenkins configs
8. Import Existing Tests — Migrate .robot and .feature files to .synth
9. Interactive Chat Mode — Conversational test authoring
10. Multi-Environment Promotion — Gate-based dev → staging → production pipeline
11. Source Code Analysis — Scan React/Vue/Django/Flask apps, generate BDD tests
12. LLM Cost Optimization — Caching, local models, smart routing

---

Synthesize Studio

Web-based multi-framework testing platform

• 🌐 Live at: https://studio.synthesize-ai.dev
• 🎨 Monaco Editor: VS Code-like editing experience
• ⚡ Real-time Execution: WebSocket-powered live run progress
• 🔄 Self-Healing UI: Visual healing suggestions with Heal All for batch fixes
• 📊 Dashboard: Recharts-powered stats, trends, and recent runs with drill-down
• 💰 Cost Monitor: Per-user LLM cost tracking with daily/provider/task breakdowns
• 💬 Framework-Aware Chat: AI assistant adapts to your selected framework
• 📄 Report Viewer: Inline report.html and log.html with embedded screenshots
• 🔍 Page Analyzer: Extract DOM elements and drag-and-drop into .synth editor
• 📈 Analytics: Trend analysis, flaky test detection, regression detection, ML failure predictions
• 🔒 Security Center: Secrets scanning, risk scoring, OWASP ZAP integration
• 👥 Team Sharing: Create teams, invite members via email (SES), share projects
• 📤 GitHub Export: Push projects directly to GitHub repos (paid plans)
• 📚 Tutorials: 12 step-by-step in-app tutorials from beginner to advanced
• 🔌 21 Lifecycle Hooks: Extensible hook system for framework plugins
• 🔐 Auth: Local email/password, GitHub OAuth, Google OAuth with plan-based feature gating
• 🔑 Secrets Vault: Personal and team-scoped encrypted secrets, injected into test generation and execution
• 🛡️ Admin Panel: User CRUD, role/plan management, activity monitoring, admin-provisioned accounts (email, SAML, LDAP)

Technology Stack:

• Frontend: Next.js 14 + TypeScript + Tailwind + Zustand + Recharts
• Backend: FastAPI + PostgreSQL (SQLAlchemy async) + Anthropic Claude
• Email: Amazon SES API (HTTPS, DKIM-verified domain)
• Billing: Stripe (free/pro/team plans)
• Deployment: DigitalOcean DOKS (Kubernetes) with nginx ingress

Try Studio →

---

VS Code Extension

Rich IDE integration for .synth files

• 🎨 Syntax highlighting
• ✅ Real-time validation
• 💡 IntelliSense for keywords and compliance tags
• 🔍 Hover documentation
• ⚡ CodeLens actions (Generate, Run, Preview)
• ⌨️ Keyboard shortcuts

Install:

code --install-extension synthesize.vscode-synthesize

Extension Documentation →

---

Plugin System

Synthesize has a powerful plugin architecture with 21 lifecycle hooks:

Create a Plugin

# Scaffold a new plugin
synthesize plugin create my-framework

# Implement generator
cd ~/.synth/plugins/my-framework/
# Edit generator.py

Minimal Plugin:

from synthesize.plugins.base import BaseGenerator

class MyFrameworkGenerator(BaseGenerator):
    def generate(self, intent):
        # Transform intent to test code
        return generated_code
    
    def get_file_extension(self):
        return '.test.js'

Plugin Manifest (plugin.yaml):

name: my-framework
version: 1.0.0
provides:
  generators: [my-framework]
entry_points:
  generator: generator.MyFrameworkGenerator

Available Hooks

Plugins can hook into 21 lifecycle events:

• pre_generate, post_generate - Code generation
• pre_execute, post_execute - Test execution
• pre_heal, post_heal - Self-healing
• pre_validate, post_validate - Test validation
• pre_parse, post_parse - Input parsing
• pre_report, post_report - Report generation
• pre_analyze, post_analyze - Analytics processing
• pre_security_scan, post_security_scan - Security scanning
• pre_save, post_save - Project saving
• on_error, on_retry - Error handling
• on_test_pass, on_test_fail - Test result callbacks

Plugin Development Guide →

---

Architecture

┌─── Natural Language (.synth files) ──────────────┐
│  suite: Login Tests                             │
│  tests:                                          │
│    - User can login with valid credentials      │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── AI Interpretation Engine ────────────────────┐
│  • Intent parsing (Claude)                      │
│  • Context understanding                        │
│  • Action decomposition                         │
│  • Data requirement extraction                  │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── Code Generation (Multi-Framework) ───────────┐
│  • Plugin-based generators                      │
│  • Template rendering                           │
│  • Locator strategies                           │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── Quality Gate ─────────────────────────────────┐
│  • Structural validation                        │
│  • Security checks                              │
│  • Compliance mapping                           │
└────────────────┬────────────────────────────────┘
                 ▼
┌─── Execution Engine ─────────────────────────────┐
│  • Smart orchestration                          │
│  • Parallel execution                           │
│  • Real-time monitoring                         │
└────────────────┬────────────────────────────────┘
                 ▼
         ┌───────┴───────┐
         │               │
       Pass            Fail
         │               │
         ▼               ▼
    ┌────────┐    ┌─────────────┐
    │Results │    │Self-Healing │
    │   DB   │    │   Engine    │
    └────┬───┘    └──────┬──────┘
         │               │
         └───────┬───────┘
                 ▼
┌─── Analytics & Insights ─────────────────────────┐
│  • Flakiness detection                          │
│  • Trend analysis                               │
│  • Cost tracking                                │
│  • Compliance reporting                         │
└──────────────────────────────────────────────────┘

---

Configuration

Global config (.synth.yaml):

# LLM Settings
llm:
  provider: anthropic
  model: claude-sonnet-4-5-20250929
  max_tokens: 4000
  
  # Cost Optimization
  cache:
    enabled: true
    ttl_seconds: 86400
  
  routing:
    simple: ollama/llama3.2    # Local model for simple tasks
    complex: anthropic/opus    # Claude Opus for complex

# Framework Plugins
plugins:
  enabled:
    - robot
    - playwright
    - cypress
  
  settings:
    robot:
      browser: chrome
      headless: false

# Self-Healing
healing:
  enabled: true
  max_retries: 3
  strategies:
    - locator_alternatives
    - ai_suggestion
    - dom_analysis

# Team Collaboration
collaboration:
  server_url: https://kb.synthesize-ai.dev
  team_id: ${TEAM_ID}
  auto_sync: true

# Security
security:
  zap:
    enabled: true
    active_scan: false
  secrets:
    enabled: true
    fail_on_high: true

# Analytics
analytics:
  enabled: true
  flake_detection: true
  trend_analysis: true

Full Configuration Guide →

---

Pricing

Open Source (Free Forever)

• ✅ Core CLI with all commands
• ✅ 6 built-in framework plugins
• ✅ Self-healing engine
• ✅ Local execution
• ✅ Community support

Pro ($49/user/month)

• ✅ Everything in Open Source, plus:
• ✅ Synthesize Studio web application
• ✅ Cost optimization (saves $70+/month on LLM costs)
• ✅ Historical analytics and insights
• ✅ Priority support
• ✅ Cloud execution option

Enterprise (Custom Pricing)

• ✅ Everything in Pro, plus:
• ✅ Team collaboration (shared projects, team secrets vault)
• ✅ Admin user provisioning (email, Google, GitHub, SAML, LDAP)
• ✅ Compliance reporting (SOC2, HIPAA, SOX)
• ✅ Security scanning and secrets detection
• ✅ Multi-environment promotion pipelines
• ✅ Self-hosted / private cloud deployment
• ✅ Satellite agents for firewall-restricted networks
• ✅ Custom plugins and integrations
• ✅ SLA and dedicated support

View Pricing Details →

---

Comparison

Feature	Synthesize	Selenium Grid	Cypress Cloud	TestRail
Natural Language Tests	✅	❌	❌	❌
AI-Powered Generation	✅	❌	❌	❌
Self-Healing	✅	❌	❌	❌
Multi-Framework	✅ (6+)	✅	❌	✅
Cost Optimization	✅	❌	❌	❌
Compliance Built-in	✅	❌	❌	✅
Team Collaboration	✅	❌	✅	✅
Plugin Ecosystem	✅ (21 hooks)	❌	Limited	❌
Self-Hosted Option	✅	✅	❌	✅

---

Use Cases

🏥 Healthcare - HIPAA Compliance Testing

• Built-in HIPAA requirement mapping
• Audit trail generation
• PII detection and validation

💰 FinTech - SOX Compliance Auditing

• Financial control testing
• Change management validation
• Segregation of duties verification

🛒 E-commerce - End-to-End Testing

• Payment gateway validation
• Multi-currency testing
• Localization verification

🏢 Enterprise - Migration Testing

• Data integrity validation
• Schema comparison
• Performance benchmarking

---

Example Workflows

Representative scenarios showing how Synthesize fits real testing problems. These are illustrative, not customer results.

Compliance-Heavy Release (Healthcare)

Problem: Manual HIPAA compliance testing is slow and hard to evidence every release.

With Synthesize:

• Express compliance tests in plain English, mapped to HIPAA requirements
• Generate audit-ready traceability reports automatically
• Self-healing keeps the suite green as the application changes

Reducing Test Maintenance (FinTech)

Problem: Brittle, framework-locked tests consume a large share of QA time.

With Synthesize:

• Regenerate the same intent across Selenium, Playwright, or Cypress — no rewrites
• Self-healing repairs broken locators instead of failing the run
• Cost controls (caching, local models, smart routing) reduce LLM spend

Stabilizing CI/CD (E-commerce)

Problem: Flaky tests intermittently block the pipeline.

With Synthesize:

• Flakiness detection identifies and quarantines unstable tests
• Smart orchestration runs the right tests in parallel
• Trend analytics surface regressions before they reach main

---

Community

• 💬 Discord: Join our community
• 🐦 Twitter: @synthesize_dev
• 📝 Blog: blog.synthesize-ai.dev
• 📺 YouTube: Tutorials and demos
• 📖 Docs: docs.synthesize-ai.dev

---

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Areas we need help with:

• 🔌 New framework plugins
• 📚 Documentation improvements
• 🐛 Bug reports and fixes
• ✨ Feature requests
• 🌍 Internationalization

---

Roadmap

Q1 2026

• ✅ Plugin system v2 with 21 lifecycle hooks
• ✅ Cost optimization (70% reduction)
• ✅ Multi-framework support (6 frameworks)
• ✅ Synthesize Studio web app
• ✅ Studio Dashboard with Recharts analytics
• ✅ Per-user LLM cost tracking (PostgreSQL-backed)
• ✅ Framework-aware chat (6 frameworks)
• ✅ Run history with RF report/log persistence
• ✅ Page analyzer with DOM element extraction
• ✅ 12 in-app tutorials (beginner to advanced)
• ✅ Enhanced regression detection (moving averages, confidence intervals)
• ✅ ML failure predictions (weighted prediction model)
• ✅ Live report preview tab in Studio
• ✅ OWASP ZAP security scanner integration
• ✅ WebSocket real-time run progress
• ✅ Team sharing with email invites (Amazon SES)
• ✅ GitHub export (push projects to GitHub repos)
• ✅ Stripe billing (free/pro/team plans)

Q2 2026

• 🚧 Mobile testing (Appium plugin)
• 🚧 GitHub App for PR integration
• 🚧 Visual regression testing
• 🚧 Satellite agent for firewall-restricted environments

Q3 2026

• 📋 AI-powered test maintenance
• 📋 Auto-documentation generation
• 📋 Multi-language support (beyond English)
• 📋 Performance testing suite

Q4 2026

• 📋 Enterprise SSO backends (SAML/LDAP identity provider integration)
• 📋 Advanced compliance frameworks
• 📋 On-premise deployment automation
• 📋 Global plugin registry

View Full Roadmap →

---

FAQ

Q: Do I need to know Python/JavaScript? A: No. Write tests in plain English. Synthesize generates the code.

Q: What frameworks are supported? A: 6 built-in (Robot, Playwright, Cypress, Selenium, Cucumber, pytest), plus community plugins.

Q: How accurate is the AI generation? A: 90%+ accuracy with proper context. Self-healing handles edge cases.

Q: Can I use my own test framework? A: Yes! Build a plugin in ~100 lines of Python.

Q: Is my test data secure? A: Yes. All data processing is local. LLM calls are encrypted. Self-hosted option available.

Q: What's the pricing? A: Open source is free. Pro is $49/user/month. Enterprise is custom pricing.

Q: How does self-healing work? A: When a test fails, Synthesize analyzes the failure, attempts fixes (new locators, retry logic), and reports confidence.

Q: Can I migrate existing tests? A: Yes! Import from Robot Framework, Selenium, Cucumber, or plain English.

More FAQs →

---

License

MIT © 2026 Synthesize. Built by Lanier Developments.

---

Credits

Built with:

• Claude - AI interpretation
• Robot Framework - Test execution
• Playwright - Browser automation
• Next.js - Web application
• FastAPI - Backend API

---

Get Started • Documentation • Plugins • Discord