docusync 2.2.0

CLI tool for syncing documentation from multiple repositories for Docusaurus

📚 DocuSync

Effortlessly sync documentation from multiple repositories into your Docusaurus site

Features • Installation • Quick Start • Usage • Configuration

---

🎯 Overview

DocuSync is a powerful CLI tool designed specifically for Docusaurus projects that need to aggregate documentation from multiple GitHub repositories. It automatically clones, organizes, and generates the proper category structure for your multi-repository documentation setup.

Perfect for:

• 🏢 Microservices architectures - Centralize docs from multiple services
• 📦 Monorepo projects - Sync docs from different packages
• 🔧 SDK ecosystems - Aggregate documentation from multiple SDKs
• 🌐 Multi-team projects - Combine docs from different teams

✨ Features

• 🚀 Fast & Efficient - Shallow cloning with configurable depth
• 🎨 Docusaurus Integration - Auto-generates _category_.json files
• 📋 Multi-Repository Support - Sync from unlimited GitHub repositories
• 🔧 Flexible Configuration - JSON-based configuration with validation
• 🔐 Multiple Auth Methods - Support for SSH keys and HTTPS with Personal Access Tokens
• 📊 Beautiful Output - Rich console interface with progress indicators
• 🧹 Clean & Safe - Automatic cleanup of temporary files
• ✅ Type-Safe - Built with Pydantic for robust configuration
• 🎯 Selective Sync - Sync all repos or just one
• 🔨 MD/MDX Fixer - Automatically fix common Markdown/MDX issues that cause Docusaurus build failures

📦 Installation

Using uv (Recommended)

uv add docusync

Using pip

pip install docusync

For Development

git clone https://github.com/Roman505050/docusync.git
cd docusync
uv sync --all-groups

🚀 Quick Start

1. Initialize configuration

docusync init

2. Edit docusync.json with your repositories

3. Sync your documentation

docusync sync

4. Run Docusaurus

npm run start

That's it! Your documentation is now synced and ready to go! 🎉

📖 Usage

Sync All Repositories

docusync sync

With verbose output:

docusync sync -v

Keep temporary files for debugging:

docusync sync --no-cleanup

Automatically fix MD/MDX issues after sync:

docusync sync --fix-md

Sync Single Repository

docusync sync-one <repository-name>

Example:

docusync sync-one api-gateway

With automatic MD/MDX fixing:

docusync sync-one api-gateway --fix-md

Fix Markdown/MDX Files

Fix common MDX/Markdown issues that cause Docusaurus build failures:

# Fix all .md files in a directory
docusync fix docs/

# Fix a single file
docusync fix docs/my-file.md

# Preview changes without applying them
docusync fix docs/ --dry-run

# Fix only files in the target directory (non-recursive)
docusync fix docs/ --no-recursive

What the fixer fixes:

• ❌ Invalid JSX tag names (e.g., <1something> → <1something>)
• ❌ HTML comments in MDX context (e.g., <!-- comment --> → {/* comment */})
• ❌ Unclosed void elements (e.g., <br> → <br />)
• ❌ Invalid HTML attributes (e.g., class= → className=, for= → htmlFor=)
• ❌ Self-closing tag spacing (e.g., <tag/> → <tag />)
• ❌ Malformed numeric entities (e.g., &#123 → {)

List Configured Repositories

docusync list

Initialize Configuration

docusync init

With custom config path:

docusync init -c custom-config.json

⚙️ Configuration

Basic Configuration

Create a docusync.json file in your Docusaurus project root:

{
  "repositories": [
    {
      "github_path": "acme-corp/api-gateway",
      "docs_path": "docs",
      "display_name": "API Gateway",
      "position": 1,
      "description": "Central API gateway documentation"
    },
    {
      "github_path": "acme-corp/user-service",
      "docs_path": "documentation",
      "display_name": "User Service",
      "position": 2,
      "description": "User management and authentication service"
    },
    {
      "github_path": "acme-corp/payment-processor",
      "docs_path": "docs",
      "display_name": "Payment Processor",
      "position": 3,
      "description": "Payment processing and billing documentation"
    }
  ],
  "paths": {
    "temp_dir": ".temp-repos",
    "docs_dir": "docs"
  },
  "git": {
    "clone_depth": 1,
    "default_branches": ["main", "master"]
  }
}

Configuration Options

repositories (required)

Array of repositories to sync:

Field	Type	Description
github_path	string	GitHub repository path (owner/repo)
docs_path	string	Path to documentation within the repository
display_name	string	Display name for the category
position	integer	Sidebar position (must be unique)
description	string	Category description for Docusaurus
protocol	string	(Optional) Clone protocol: "ssh" or "https"
pat_token_env	string	(Optional) Environment variable with PAT token for this repo
ssh_key_path	string	(Optional) Path to SSH private key for this repo

paths (required)

Field	Type	Description
temp_dir	string	Directory for temporary clones (auto-deleted)
docs_dir	string	Target directory for documentation

git (required)

Field	Type	Description
clone_depth	integer	Git clone depth (1 for shallow clone)
default_branches	array	Default branches to try cloning
default_protocol	string	Clone protocol: "ssh" or "https" (default: "ssh")
default_ssh_key_path	string	Default SSH private key path (optional, e.g., ~/.ssh/id_ed25519)
default_pat_token_env	string	Default environment variable name containing GitHub Personal Access Token (optional)

Authentication & Protocols

SSH Authentication (default):

• Uses [e-mail]:owner/repo.git format
• Requires SSH key setup with GitHub
• Best for local development
• Supports custom SSH keys per repository

HTTPS with PAT Token:

• Uses https://github.com/owner/repo.git format
• Requires GitHub Personal Access Token
• Best for CI/CD pipelines
• Token is read from environment variable
• Supports different tokens per repository

Example with HTTPS:

{
  "git": {
    "clone_depth": 1,
    "default_branches": ["main", "master"],
    "default_protocol": "https",
    "default_pat_token_env": "GITHUB_PAT_TOKEN"
  }
}

Then set your token:

export GITHUB_PAT_TOKEN="ghp_your_token_here"

Example with custom SSH keys:

{
  "repositories": [
    {
      "github_path": "acme-corp/api-docs",
      "protocol": "ssh",
      "ssh_key_path": "~/.ssh/acme_corp_key",
      ...
    },
    {
      "github_path": "partner-org/service-docs",
      "protocol": "ssh",
      "ssh_key_path": "~/.ssh/partner_org_key",
      ...
    }
  ],
  "git": {
    "default_protocol": "ssh",
    "default_ssh_key_path": "~/.ssh/id_ed25519"
  }
}

Per-repository protocol override:

{
  "repositories": [
    {
      "github_path": "acme-corp/payment-processor",
      "docs_path": "docs",
      "display_name": "Payment Processor",
      "position": 3,
      "description": "Payment processing documentation",
      "protocol": "https"
    }
  ]
}

Multiple organizations with different authentication:

{
  "repositories": [
    {
      "github_path": "acme-corp/api-docs",
      "display_name": "ACME API",
      "protocol": "ssh",
      "ssh_key_path": "~/.ssh/acme_corp_key",
      ...
    },
    {
      "github_path": "partner-org/service-docs",
      "display_name": "Partner Service",
      "protocol": "https",
      "pat_token_env": "PARTNER_ORG_PAT_TOKEN",
      ...
    },
    {
      "github_path": "contractor/integration-docs",
      "display_name": "Integration",
      "protocol": "https",
      "pat_token_env": "CONTRACTOR_PAT_TOKEN",
      ...
    }
  ],
  "git": {
    "default_protocol": "ssh",
    "default_ssh_key_path": "~/.ssh/id_ed25519",
    "default_pat_token_env": "GITHUB_PAT_TOKEN"
  }
}

Then set individual credentials:

# For HTTPS repositories
export PARTNER_ORG_PAT_TOKEN="ghp_partner_token"
export CONTRACTOR_PAT_TOKEN="ghp_contractor_token"
export GITHUB_PAT_TOKEN="ghp_default_token"

Priority for authentication:

• SSH: Repository ssh_key_path → Global default_ssh_key_path → System default
• HTTPS: Repository pat_token_env → Global default_pat_token_env → No token

🎨 Docusaurus Integration

Automatic _category_.json Generation

DocuSync automatically creates _category_.json files in each synced documentation directory, following the Docusaurus category format:

{
  "label": "API Gateway",
  "position": 1,
  "link": {
    "type": "generated-index",
    "description": "Central API gateway documentation"
  }
}

This enables Docusaurus to:

• ✅ Automatically generate index pages for each category
• ✅ Properly order documentation in the sidebar
• ✅ Display category descriptions on index pages
• ✅ Create a beautiful, organized documentation structure

Project Structure After Sync

your-docusaurus-project/
├── docusaurus.config.js
├── docusync.json
├── docs/
│   ├── api-gateway/
│   │   ├── _category_.json      # Auto-generated ✨
│   │   ├── intro.md
│   │   ├── getting-started.md
│   │   └── api-reference.md
│   ├── user-service/
│   │   ├── _category_.json      # Auto-generated ✨
│   │   ├── overview.md
│   │   └── configuration.md
│   └── payment-processor/
│       ├── _category_.json      # Auto-generated ✨
│       ├── setup.md
│       └── webhooks.md
└── .temp-repos/                 # Cleaned up after sync

🔍 Real-World Example

Here's a complete workflow for a microservices documentation setup:

# 1. Initialize your Docusaurus project
npx create-docusaurus@latest my-docs classic

# 2. Navigate to your project
cd my-docs

# 3. Initialize DocuSync
docusync init

# 4. Configure your repositories in docusync.json
cat > docusync.json << 'EOF'
{
  "repositories": [
    {
      "github_path": "your-org/auth-service",
      "docs_path": "docs",
      "display_name": "Authentication Service",
      "position": 1,
      "description": "User authentication and authorization"
    },
    {
      "github_path": "your-org/billing-service",
      "docs_path": "docs",
      "display_name": "Billing Service",
      "position": 2,
      "description": "Payment processing and billing management"
    }
  ],
  "paths": {
    "temp_dir": ".temp-repos",
    "docs_dir": "docs"
  },
  "git": {
    "clone_depth": 1,
    "default_branches": ["main", "master"]
  }
}
EOF

# 5. Sync documentation
docusync sync -v

# 6. Start Docusaurus
npm run start

🛠️ Development

Setup Development Environment

# Clone the repository
git clone https://github.com/Roman505050/docusync.git
cd docusync

# Install dependencies
uv sync --all-groups

Code Quality

# Format code
uv run black src/
uv run isort src/

# Lint
uv run flake8 src/

# Type check
uv run mypy src/

Running Tests

# Run all tests
uv run pytest

# With coverage
uv run pytest --cov=docusync --cov-report=html

📋 Requirements

• Python 3.12 or higher
• Git installed on your system
• GitHub Access: Either SSH keys configured OR Personal Access Token for HTTPS
• Docusaurus 2.x or higher (for the target project)

🤝 Contributing

Contributions are welcome! Here's how you can help:

1. 🍴 Fork the repository
2. 🔧 Create a feature branch (git checkout -b feature/amazing-feature)
3. ✅ Commit your changes (git commit -m 'Add amazing feature')
4. 📤 Push to the branch (git push origin feature/amazing-feature)
5. 🎉 Open a Pull Request

Please make sure to:

• Add tests for new features
• Update documentation as needed
• Follow the existing code style
• Run linters before submitting

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Click - Beautiful CLI framework
• Rich - Rich text and formatting in the terminal
• Pydantic - Data validation using Python type hints
• Docusaurus - The documentation framework this tool was built for

📧 Support

• 🐛 Bug Reports: Open an issue
• 💡 Feature Requests: Open an issue
• 📖 Documentation: Read the docs

🌟 Show Your Support

If you find DocuSync helpful, please consider giving it a ⭐️ on GitHub!

---

Made with ❤️ for the Docusaurus community

Report Bug • Request Feature