🧠 Your Company Brain (v4.6.0)

Your Company Brain is an automated, offline-first knowledge extraction pipeline and multi-provider AI Q&A system. It connects to 38 connectors across your company's communication, engineering, HR, productivity, customer support, and analytics stack, pulls all scattered information, runs it through local or cloud LLMs, and outputs structured "Skills" — machine-readable procedure cards that AI agents can directly execute, or lets you query your company data directly in your terminal using natural-language search (ycb --ask).

Instead of manually writing instruction manuals for your AI tools or searching across 10 different dashboards, Your Company Brain generates them automatically and answers questions across any model you choose.

🚀 What's New in v4.6.0 — Task-Oriented Quick-Start Guide, First-Login Tour & Clean UI:

• Interactive Quick-Start Guide (ycb --help / ycb help): Restructured help output around common developer workflows (connecting data sources, running first sync, querying company data, model switching, status checking) with clear step-by-step instructions before the flat command reference. Supports --plain mode for clean terminal piping.
• First-Login Quick-Start Overview: Upon first successful login, automatically displays a concise 7-line overview of core commands immediately after the welcome banner, guiding new users on what to do next.
• Solid Filled-Block ASCII Art Branding: Rebuilt CLI startup branding with custom 6-row thick filled rectangular glyphs (█), 3-layer diagonal drop shadow, and a glowing magenta-to-pink gradient sweep.
• Pristine Terminal UI & Error Consolidation: Eliminated noisy third-party log output during sync; errors are cleanly collected and summarized in a structured report panel at sync completion.

🚀 What's New in v4.5.0 — Fixed Google OAuth Port (eliminates redirect_uri_mismatch):

• Fixed OAuth Callback Port 8765: All four Google connectors (google_docs, gmail, google_drive, google_sheets) now use a hardcoded fixed port 8765 for the OAuth callback server, so the redirect URI is always http://localhost:8765/. Previously, a random ephemeral port was chosen every run, meaning Google Cloud Console could never be configured to match it.
• One-time Cloud Console step required: You must add http://localhost:8765/ as an Authorized redirect URI in Google Cloud Console for your OAuth Client ID. See Google Connector Setup below.
• Port-in-use detection: If port 8765 is already occupied when the OAuth flow starts, ycb now prints a clear error with the exact netstat/lsof command to find what's using it — no raw socket traceback.
• redirect_uri_mismatch detection: If Google Cloud Console hasn't been updated yet, the error now includes step-by-step instructions to fix it rather than a cryptic Error 400.

🚀 What's New in v4.4.0 — Google OAuth Reliability & Credential Validation:

• Fixes invalid_grant / stale-token failures on ycb sync --google-docs: A shared google_auth_helper.py module now handles all Google OAuth flows (Docs, Gmail, Drive, Sheets). When a cached refresh token is revoked or expired (invalid_grant), the stale token is automatically deleted and a fresh browser OAuth flow is triggered — no more silent "0 items ingested" failures.
• Global OAuth Token Storage: The Google OAuth token is now stored in ~/.ycb/google_token.pickle (not a CWD-relative token.pickle). This means ycb sync --google-docs works correctly regardless of which directory you run it from.
• Strict GOOGLE_CREDENTIALS_PATH Validation: ycb connect google_docs now validates the credentials file (existence → JSON parse → correct installed/web structure) and re-prompts on failure rather than silently storing a broken path. Garbage input is rejected with a clear [ERROR] message.
• Complete ycb disconnect Cleanup: ycb disconnect google_docs (and gmail, google_drive, google_sheets) now also deletes the global ~/.ycb/google_token.pickle and any legacy CWD token.pickle, so that re-connecting always starts from a clean OAuth state.

🚀 What's New in v4.3.0 — Interactive Sync Default, Auto-Export & Log Routing:

• Interactive Batched Sync by Default (ycb sync): ycb sync without flags now runs in interactive batched mode with visible checkpoints and progress bars — no more staring at a blank screen wondering if sync has stalled.
• Full Continuous Sync Flag (ycb sync --full): To run an uninterrupted, continuous pass through all sources without stopping, use ycb sync --full. (The legacy --once flag is retained as an alias).
• Automatic Skills Export: Upon sync completion, skills are automatically written to output/skills_file.json whenever new skills are synthesized — manual ycb export is no longer required!
• Clean Console & Persistent Log Routing: Terminal output is kept pristine during sync; full debug logs are silently routed to %USERPROFILE%\.ycb\logs\ycb.log (Windows) or ~/.ycb/logs/ycb.log (macOS/Linux). Pass --verbose to stream debug logs to the terminal.
• One-Time Notice: Existing users running ycb sync for the first time on v4.3.0 will see a clear one-time notice explaining the new default behavior and pointing to --full.

🚀 What's New in v4.2.0 — Centralized Global Configuration (~/.ycb/config.json):

• Centralized Global Config: Settings are no longer scattered in per-folder .env files. All provider selections, API keys, active license sessions, and connector credentials are now persisted centrally in %USERPROFILE%\.ycb\config.json (Windows) or ~/.ycb/config.json (macOS/Linux).
• Cross-Folder Consistency: Run ycb switch model, ycb login, or ycb connect from any directory — your active models and credentials immediately persist across your entire system.
• Automatic Zero-Loss Migration: If you run ycb in an existing folder containing a legacy .env, settings are automatically detected and copied into your global config without modifying or deleting your original .env.
• Resilient Diagnostics: Malformed or corrupted JSON in config.json is detected with clean line and column warnings on stderr rather than crashing, safely falling back to safe defaults.

🚀 What's New in v4.1.0 — NVIDIA NIM Direct Provider:

• NVIDIA NIM Direct Provider (nvidia_nim): Bypasses OpenRouter's shared free-tier quota entirely. Talks directly to https://integrate.api.nvidia.com/v1 using your own personal NVIDIA API key — dedicated throughput, no quota shared with anyone.
• Streaming Reasoning Separation: NVIDIA's reasoning_content field (the model's internal thinking trace) is silently stripped from all responses; only the final content answer is surfaced to the caller.
• Clean Error Messaging: check_available() fails fast with a clear, actionable message if NVIDIA_NIM_API_KEY is missing — no silent fallbacks.
• Model Switch Support: Available as option #9 in ycb switch model under a dedicated NVIDIA NIM category.

🚀 What's New in v4.0.0 — Hybrid General/Company Q&A & Dynamic Context Auto-Shifting:

• Hybrid General & Company Q&A (ycb --ask): Operates as a full-featured AI assistant with access to your knowledge base. When company data is relevant, it cites it; when data is absent or thin, it falls back to general knowledge rather than failing or blocking the LLM call.
• Dynamic Context-Capacity Auto-Shifting: If a query's context exceeds your default model's context budget (e.g. openrouter/free at 90k chars), ycb automatically routes that query to a high-capacity model (such as nvidia/nemotron-3-ultra-550b-a55b:free with a 1,000,000-token / 3.15M-character context window).
• Non-Sticky, Per-Request Routing: Auto-shifting is strictly per-query. Follow-up queries immediately return to your fast default model without mutating .env or overriding your preferences.
• Transparent Model Attribution: The Answer Panel subtitle and --verbose audit clearly display the model that answered and indicate whether an auto-shift occurred.
• Runtime Context Window & Availability Recovery: Intercepts context_length_exceeded errors and 403 harness-gated models, seamlessly rotating through the free fallback chain.

---

✨ What Does It Actually Do?

1. Reads Your Data: Securely connects to 38 connectors (Slack, Google Docs, GitHub, Notion, Discord, Linear, Outlook, GitLab, Dropbox, Mixpanel, Amplitude, Algolia, Exa, Perplexity, Facebook, Todoist, Web Crawler, Jira, Asana, Calendly, ClickUp, Airtable, Datadog, Segment, Front, Zoom, Twitter, HubSpot, Salesforce, Monday, Basecamp, Ashby, BambooHR, Deel, Rippling, Gmail, Google Sheets, Google Drive) ingesting raw text through a unified connector registry.
2. Understands Context: It breaks the text down and uses a local AI model (gemma4:e4b) to figure out what the text is actually about (e.g., "Is this a refund policy?" or "Is this a server deployment guide?").
3. Synthesizes "Skills": It groups related information together and writes step-by-step procedures, including decision points (if/then rules) and edge cases.
4. Exports for AI Agents: It outputs everything into a clean skills_file.json that you can plug directly into tools like LangChain, AutoGen, or your own custom AI bots so they know exactly how your company operates.

---

🛠️ Technical Architecture & Algorithms

For technical deep-dives, here is exactly how the pipeline operates under the hood:

1. Ingestion & Connectors (Data Layer)

• Abstract Base Connector: All 38 connectors implement BaseConnector, a shared abstract class that enforces a standard ingest(days_back) interface, provides retry_with_backoff() for exponential backoff on API rate limits (429/5xx errors), and checks CONNECTORS_ENABLED for dynamic toggling without code changes.
• Connector Registry: registry.py maintains a central list of all 38 connector classes. get_enabled_connectors() instantiates each one and filters out unconfigured connectors (e.g., if GITHUB_TOKEN is missing, GitHubConnector silently skips). This means SyncScheduler never imports individual connectors — it only loops over whatever is enabled.
• Error Isolation: Each connector runs in a try/except block inside the sync loop. A bad token or an API outage on one connector logs an error and moves on — it will not crash the entire sync run.
• Idempotency: Before processing, the system calls get_processed_source_item_ids() which returns all item IDs already stored in processed_chunks. Any item already in that set is skipped. Running sync 10 times on the same data produces the same result as running it once.

2. Document Chunking Algorithm

• Semantic Sentence Splitting: Instead of naive character-count splitting (which breaks code blocks and sentences in half), the DocumentChunker uses regex-based sentence boundary detection.
• Sliding Window Overlap: Text is chunked into 5-sentence blocks with a 2-sentence overlap. This guarantees that context isn't lost across chunk boundaries, which is critical for accurate LLM extraction.

3. Knowledge Extraction (LLM Pipeline)

• Combined Single LLM Call: The KnowledgeExtractor sends a single optimised prompt that simultaneously classifies the chunk type (procedure, policy, decision, incident, general) AND extracts key concepts as a JSON array. This halves API latency compared to two sequential calls.
• Structured JSON Fallbacks: Because open-source LLMs can hallucinate formatting, the prompt enforces strict JSON output, which is then parsed using Python's built-in json library with regex fallbacks to strip out markdown code fences.

4. Skill Synthesis (Clustering & Generation)

• Context Window Assembly: The SkillsSynthesizer filters the database for high-confidence chunks (confidence_score >= 0.2) classified as actionable knowledge.
• Schema Enforcement: It asks the LLM to act as a technical writer, reading the raw concepts and synthesizing them into a strict domain model (Skill Pydantic class). This generates the final procedure steps, prerequisites, and edge cases.

5. UI Architecture (Event-Driven Dashboard)

• Decoupled State: The rich terminal dashboard runs independently of the backend pipeline.
• Log Interception: Instead of polluting SyncScheduler with UI logic, a custom Python logging.Handler intercepts backend logs (e.g., "Phase 1: Ingesting data"), updates the UI's internal state machine, and computes progress/ETA — the SyncScheduler never knows a UI exists.

---

⚙️ The Pipeline Workflow

graph TD;
    A[38 Data Sources via REST/GraphQL APIs] -->|BaseConnector.ingest| B(ConnectorRegistry)
    B -->|RawDataItem objects| C(SQLite: raw_data_items)
    C -->|Sliding Window Chunker| D{Local LLM: gemma4:e4b}
    D -->|Single combined prompt| E(SQLite: processed_chunks)
    E -->|Confidence filtering + concept clustering| F[LLM Synthesizer]
    F -->|Pydantic Skill schema| G((skills_file.json))

---

🚀 Setup Guide

Prerequisites

• Python 3.11+
• Ollama — Download from ollama.ai
• API keys for the connectors you want to enable (see .env.example)

---

Option A — Install via pip (Recommended for users)

# 1. Create and activate a virtual environment
python -m venv venv
# On Windows:
.\venv\Scripts\Activate.ps1
# On macOS/Linux:
source venv/bin/activate

# 2. Install the package
pip install ycb

# 3. Pull the required AI model and start Ollama
ollama pull gemma4:e4b
ollama serve            # Keep running in background

# 4. Initialize — creates global config ~/.ycb/config.json, verifies Ollama & license key
ycb                     # Or reference directly without activating: .\venv\Scripts\ycb.exe init
ycb init

# 5. Connect your platforms
#    Connect any of the 38 supported platforms on demand:
ycb connect slack
ycb connect google_docs

Note: If ycb is not recognized, make sure your virtual environment is activated, reference the full path to the executable (e.g. .\venv\Scripts\ycb.exe), or ensure your Python Scripts folder is on PATH. Tip: ycb init creates your global configuration at ~/.ycb/config.json (or %USERPROFILE%\.ycb\config.json on Windows). If an existing .env is present in your current folder, ycb will automatically migrate all settings to the global config on first run without modifying your .env.

---

Option B — Developer / Source Setup

# Clone and enter the repo
git clone https://github.com/AmrutiyaKirtan/Company_Brain.git
cd Company_Brain

# Create virtual environment
python -m venv venv
.\venv\Scripts\activate         # Windows
# source venv/bin/activate      # macOS/Linux

# Install in editable mode
pip install -e .

# Pull the required AI model
ollama pull gemma4:e4b
ollama serve

# Initialize global configuration
ycb init

---

Connector Credentials & Global Configuration

All credentials and selections are stored centrally in ~/.ycb/config.json. You can easily connect and disconnect platforms using the CLI — unused connectors are automatically skipped if their credentials are absent:

ycb connect <connector>      # Interactively enter credentials
ycb disconnect <connector>   # Remove credentials

Connector	Credential Key	Where to get it
Slack	SLACK_USER_TOKEN (or OAuth)	api.slack.com/apps
Google Docs/Drive/Sheets	OAuth via credentials.json	console.cloud.google.com
GitHub	GITHUB_TOKEN	github.com/settings/tokens
Notion	NOTION_TOKEN	notion.so/my-integrations
Linear	LINEAR_API_KEY	Linear Settings
Discord	DISCORD_BOT_TOKEN	discord.com/developers
Jira / Confluence	JIRA_API_TOKEN, JIRA_DOMAIN	id.atlassian.com
Asana	ASANA_TOKEN	app.asana.com/-/developer_console
ClickUp	CLICKUP_API_TOKEN	app.clickup.com/settings/apps
HubSpot	HUBSPOT_ACCESS_TOKEN	app.hubspot.com
Salesforce	SALESFORCE_CLIENT_ID, SALESFORCE_DOMAIN	login.salesforce.com

Run ycb status to see all active, available, and unlocked connectors for your tier.

---

Google Connector Setup (Required: Redirect URI)

All four Google connectors (google_docs, gmail, google_drive, google_sheets) use a fixed OAuth callback port 8765. The redirect URI sent to Google on every run is always:

http://localhost:8765/

You must register this URI once in Google Cloud Console before your first ycb connect google_docs (or gmail/drive/sheets):

1. Go to https://console.cloud.google.com/apis/credentials
2. Click the OAuth 2.0 Client ID listed in your credentials.json
3. Under "Authorized redirect URIs", click "+ Add URI"
4. Enter exactly: http://localhost:8765/
5. Click Save and wait ~30 seconds before running ycb connect

Why this step? Google Cloud Console only allows OAuth callbacks to URIs that are explicitly pre-registered. ycb previously used a random port on every run (port=0), which made it impossible to pre-register. As of v4.5.0, the port is fixed at 8765 so this one-time setup is all that's needed.

Port already in use? If something is running on port 8765, ycb connect will print:

[Google OAuth] Port 8765 is already in use on this machine.
Find what's using it:
  Windows:  netstat -ano | findstr :8765
  macOS/Linux: lsof -i :8765

Close that process and re-run.

---

💻 Usage & Commands

If installed via pip, use the ycb command from anywhere (or python main.py in developer mode):

Authentication & Workspace Initialization

# Initialize workspace — configures database, Ollama runtime, and prompts for license
ycb init

# Sign in with an existing license key and registered email
ycb login
ycb login --license-key CB-XXXX-XXXX-XXXX --email you@company.com

# Sign out and clear local session state and cached credentials
ycb logout

🎟️ Plan Tiers & Access

Tier	Connectors Unlocked	Description
Solo	google_docs, slack (2)	Free tier for individual exploration and core communication knowledge.
Max	All 38 Connectors	Monthly subscription unlocking your entire SaaS & engineering stack with automated renewal checks and background self-healing.
Enterprise	All 38 Connectors	Organization-wide multi-seat licensing with priority support.

---

Connect a Data Source

Interactively prompt for and configure credentials for a single connector on demand.

ycb connect slack

Disconnect a Data Source

Remove stored credentials for a specific connector.

ycb disconnect slack

Run an Interactive Sync (Default)

Processes sources in small batches with real-time progress and pauses after each batch for review. Newly generated skills are automatically exported to output/skills_file.json.

# Interactive mode is now the default!
ycb sync

# The explicit flag is also supported for backward compatibility:
ycb sync --interactive

Run a Full Continuous Sync

Processes all enabled connectors in one continuous pass without stopping or prompting.

ycb sync --full

# (Legacy alias --once is also supported)
ycb sync --once

Run a Single-Connector Sync

Sync only a specific configured connector.

# Interactive batch mode for a single connector:
ycb sync --connector slack

# Continuous full mode for a single connector:
ycb sync --connector slack --full

Stream Verbose Debug Logs

By default, console logs stay clean while detailed execution logs are written to %USERPROFILE%\.ycb\logs\ycb.log (Windows) or ~/.ycb/logs/ycb.log (macOS/Linux). Pass --verbose to view full debug logs in your terminal.

ycb sync --verbose

Check Current Status

View license tier, active connectors, tier-locked connectors, and skills stats.

ycb status

Export Your Skills

Export structured data for use by AI agents.

# Export as JSON (Best for AI Agents)
ycb export --output output/skills.json

# Export as Markdown (Best for Humans)
ycb export --output output/skills.md --format markdown

Ask Questions About Your Company Knowledge & Beyond (--ask)

Run natural-language Q&A directly in your terminal. The Q&A engine operates as a hybrid assistant: it uses company knowledge when relevant (clearly citing it), and seamlessly draws on general AI knowledge for questions outside or partially outside your synced data:

# Ask a company-data question (cites company data when found)
ycb --ask "how do we handle customer refund requests"

# Ask a general knowledge question (answers normally using LLM intelligence)
ycb --ask "what is machine learning and how does gradient descent work"

# Scope question to a specific connector
ycb --ask "what PRs were merged this week" --connector github

# Run with verbose retrieval debugging audit and model attribution
ycb --ask "tell me about our onboarding workflow" --verbose

⚡ Dynamic Context Auto-Shifting

When a broad question retrieves a large volume of company data (>90k characters), ycb automatically shifts that single query to a high-capacity model (e.g. nvidia/nemotron-3-ultra-550b-a55b:free with a 1,000,000-token / 3.15M-character context window) so no information is lost:

• Non-Sticky Routing: Auto-shifting applies only to the specific query that requires it. Follow-up queries immediately return to your fast default model.
• Full Attribution: The Answer Panel subtitle and --verbose audit show the exact model that answered and whether an auto-shift occurred:

  Model: nvidia/nemotron-3-ultra-550b-a55b:free (auto-shifted from 'openrouter/free')
  

Switch AI Models & Providers (switch model)

Select which model answers your questions and synthesizes skills:

# Open the interactive categorized model selection menu
ycb switch model

# List all supported categorized models without prompting
ycb switch model --list

# Switch directly by model name or slug
ycb switch model gemma4:e4b                                       # Local (Ollama)
ycb switch model gpt-5.6-terra                                    # OpenAI
ycb switch model claude-sonnet-5                                  # Claude
ycb switch model nvidia/nemotron-3-ultra-550b-a55b                # NVIDIA NIM Direct (your quota)
ycb switch model openrouter/free                                  # OpenRouter Auto-Router
ycb switch model nvidia/nemotron-3-ultra-550b-a55b:free           # OpenRouter 1M-token free model
ycb switch model mistralai/mistral-large-2407                     # Custom OpenRouter slug

NVIDIA NIM Direct vs OpenRouter free: Both use the same Nemotron 3 Ultra 550B model, but nvidia_nim hits NVIDIA's inference endpoint with your personal NVIDIA_NIM_API_KEY (no shared quota), while the OpenRouter :free variant shares a free-tier pool with all OpenRouter users.

---

Company Brain — Complete Technical Reference

This document covers every aspect of the Company Brain project: what it does, how it works under the hood, every API used, every technology used, the full data pipeline, CLI commands, and the engineering decisions made. Written for YC interviews and technical deep-dives.

---

Table of Contents

1. What is Company Brain?
2. The Core Problem It Solves
3. High-Level Architecture
4. Technology Stack
5. Connectors — All 38 Data Sources
6. Full Data Pipeline — Step by Step
7. Database Design
8. The LLM Integration
9. CLI Commands Reference
10. Terminal Dashboard (UI)
11. Key Engineering Decisions
12. Directory Structure

---

1. What is Company Brain?

Company Brain is an offline-first knowledge extraction pipeline. It connects to 38 of your company's existing communication, engineering, HR, customer support, and analytics tools, pulls all the scattered information, runs it through a local AI model, and outputs structured "Skills" — machine-readable procedure cards that AI agents can directly execute.

In plain English: Your team's knowledge lives in thousands of Slack messages, GitHub issues, Notion pages, and Jira tickets. Right now, no AI agent can act on that knowledge because it's buried in unstructured text across 38 different platforms. Company Brain reads all of it and converts it into clean, structured instructions.

---

2. The Core Problem It Solves

Without Company Brain	With Company Brain
AI agents have no idea how your company operates	AI agents get a skills_file.json with exact procedures
Building custom SOPs takes weeks of manual work	Company Brain auto-generates them by reading your existing docs
Institutional knowledge lives in people's heads / old Slack threads	It's extracted, structured, and searchable
Onboarding new hires is slow because no one knows where to find information	Skills are tagged by category, have prerequisites, and success criteria
Your knowledge is siloed across 38+ tools	A single unified sync pulls everything into one knowledge base

---

3. High-Level Architecture

┌───────────────────────────────────────────────────────────────────────────┐
│                              Data Sources (38)                             │
│  Slack  GitHub  Notion  Discord  Linear  Google Docs  Google Sheets        │
│  Outlook/Teams  GitLab  Dropbox  Mixpanel  Amplitude  Algolia  Exa        │
│  Perplexity  Facebook  Todoist  Web Crawler  Jira  Asana  Calendly         │
│  ClickUp  Airtable  Datadog  Segment  Front  Zoom  Twitter  HubSpot        │
│  Salesforce  Monday  Basecamp  Ashby  BambooHR  Deel  Rippling  Gmail      │
│  Google Drive                                                             │
└────────────────────────────────┬──────────────────────────────────────────┘
                                 │  REST / GraphQL APIs
                                 ▼
┌───────────────────────────────────────────────────────────────────────────┐
│                     Connector Layer (BaseConnector)                        │
│                                                                            │
│  • Every connector inherits BaseConnector (get_source_name, ingest)        │
│  • registry.py: get_enabled_connectors() loops all 38, skips unconfigured  │
│  • retry_with_backoff() handles 429/5xx with exponential backoff           │
│  • Error isolation: one failing connector never blocks the others          │
│  • CONNECTORS_ENABLED env var toggles connectors without code changes      │
└────────────────────────────────┬──────────────────────────────────────────┘                                                                  │
│  • Every connector inherits BaseConnector (get_source_name, ingest)        │
│  • registry.py: get_enabled_connectors() loops all 16, skips unconfigured  │
│  • retry_with_backoff() handles 429/5xx with exponential backoff           │
│  • Error isolation: one failing connector never blocks the others          │
│  • CONNECTORS_ENABLED env var toggles connectors without code changes      │
└────────────────────────────────┬──────────────────────────────────────────┘
                                 │  List[RawDataItem] (Pydantic model)
                                 ▼
┌───────────────────────────────────────────────────────────────────────────┐
│                          SQLite Database                                   │
│                      (via SQLAlchemy ORM)                                  │
│   Table: raw_data_items     Table: processed_chunks     Table: skills      │
└────────────────────────────────┬──────────────────────────────────────────┘
                                 │
                                 ▼
┌───────────────────────────────────────────────────────────────────────────┐
│                         Processing Pipeline                                │
│                                                                            │
│  1. DocumentChunker  → sentence-based sliding window (5 sentences,         │
│     2-sentence overlap), URL/email masking, noise filtering                │
│                                                                            │
│  2. KnowledgeExtractor → single optimised prompt to gemma4:e4b via Ollama  │
│     - Classifies chunk type (procedure/policy/decision/incident/general)   │
│     - Extracts key concepts as JSON array                                  │
│     - Computes heuristic confidence score (0.0 – 1.0)                     │
│                                                                            │
│  3. SkillsSynthesizer → filters high-confidence chunks, clusters by        │
│     primary concept, asks Gemma to write a complete Skill card             │
│     (steps, decisions, prerequisites, edge cases)                          │
└────────────────────────────────┬──────────────────────────────────────────┘
                                 │
                                 ▼
┌───────────────────────────────────────────────────────────────────────────┐
│                            Export Layer                                    │
│              skills_file.json  /  skills_file.md                          │
│       (ready to plug into LangChain, AutoGen, or custom bots)             │
└───────────────────────────────────────────────────────────────────────────┘

---

4. Technology Stack

Category	Library / Tool	Why We Used It
Language	Python 3.11	Mature ecosystem for AI/NLP pipelines
Data Validation	pydantic v2	Enforces strict schema for every data model
Database ORM	sqlalchemy v2	Maps Python classes to SQLite tables cleanly
Database	SQLite	Zero-config local persistence, no server needed
HTTP Client	requests	Used by all new connectors for REST/GraphQL API calls
Slack Integration	slack-sdk v3	Official Slack client; handles pagination & auth
Google Integration	google-api-python-client, google-auth-oauthlib	Shared OAuth 2.0 flow for both Google Docs and Google Sheets
Local AI	ollama (Python client)	Runs Gemma 4 locally; zero data sent externally
CLI Framework	click	Clean, composable command-line interface
Terminal UI	rich	Beautiful dashboard with live animations, progress bars
Background Scheduler	apscheduler	Runs sync jobs on a 30-minute interval
Env Management	python-dotenv	Loads .env credentials without hardcoding secrets

---

5. Connectors — All 38 Data Sources

Company Brain uses a pluggable connector architecture. All 38 connectors inherit from BaseConnector and are registered in registry.py. The sync loop dynamically loads only the ones with valid credentials configured in .env.

Tier 1 — Core Team Communication & Engineering Knowledge

Connector	Auth	What It Ingests	Dedup Key
Slack	Bot Token	Messages, threads, replies from joined channels	slack:{channel_id}:{ts}
Google Docs	OAuth 2.0	Document text, paragraph headers, tables	gdoc:{doc_id}
Google Sheets	OAuth 2.0	Spreadsheet tabs as key-value row text	gsheet:{spreadsheet_id}:{sheet_name}
Google Drive	OAuth 2.0	Folder files metadata and text contents	gdrive:{file_id}
Gmail	OAuth 2.0	Email threads, messages, attachments text	gmail:{message_id}
GitHub	Personal Access Token	Issues, PR descriptions, labels, review comments	github:{repo}:{issue_number}
Notion	Integration Token	Page block hierarchies, databases, nested pages	notion:{page_id}
Discord	Bot Token	Server text channels and message history	discord:{channel_id}:{message_id}
Linear	API Key	Issues, descriptions, team context, comments (GraphQL)	linear:{issue_id}
Outlook / Teams	Azure OAuth 2.0 (MS Graph)	Outlook email threads + Teams channel messages	outlook:{message_id} / teams:{channel_id}:{message_id}

Tier 2 — Extended Dev, File & Search Sources

Connector	Auth	What It Ingests	Dedup Key
GitLab	Personal Access Token	Projects, issues, merge requests, notes	gitlab:{project}:{type}:{iid}
Dropbox	OAuth Access Token	Text files (.md, .txt, .json, .csv, .doc)	dropbox:{file_id}
Mixpanel	API Secret	Tracked event schema catalog	mixpanel:event_schema:{project_id}
Amplitude	API Key + Secret	Event taxonomy definitions	amplitude:taxonomy:{key}
Algolia	App ID + API Key	Indexed search records from named index	algolia:{index}:{objectID}
Exa	API Key	Web research results for search queries	exa:{hash(query)}
Perplexity	API Key	AI search completions for configured prompts	perplexity:{hash(prompt)}
Facebook	Page Access Token	Page posts and customer comments	facebook:{post_id}
Todoist	API Token	Project tasks, descriptions, comments	todoist:{task_id}
Web Crawler	None / Config URL	Scraped web pages, documentation site text	webcrawler:{url_hash}

Tier 3 — Project Management, Issue Tracking & Analytics

Connector	Auth	What It Ingests	Dedup Key
Jira	API Token + Domain	Issues, sprint boards, work logs, comments	jira:{issue_key}
Asana	Personal Access Token	Tasks, project boards, subtasks, stories	asana:{task_gid}
ClickUp	API Key	Tasks, docs, comments, custom fields	clickup:{task_id}
Airtable	Personal Access Token	Base tables, records, field mappings	airtable:{base_id}:{record_id}
Monday.com	API Token	Boards, items, column updates	monday:{item_id}
Basecamp	OAuth 2.0	Message boards, to-dos, campfires	basecamp:{recording_id}
Datadog	API Key + APP Key	Incident logs, monitors, alert post-mortems	datadog:{incident_id}
Segment	Public API Key	Tracking plans, event schema catalog	segment:{event_id}

Tier 4 — CRM, Support, HR & Enterprise

Connector	Auth	What It Ingests	Dedup Key
Front	API Token	Shared inbox conversations, messages	front:{conversation_id}
Zoom	Account Credentials	Cloud recording transcripts, meeting chat logs	zoom:{meeting_id}
Twitter / X	Bearer Token	User tweets, replies, thread context	twitter:{tweet_id}
HubSpot	Private App Token	CRM contacts, deal notes, ticket history	hubspot:{object_type}:{id}
Salesforce	OAuth 2.0	Accounts, Opportunities, Case resolution notes	salesforce:{object}:{id}
Calendly	Personal Access Token	Scheduled event types, meeting notes	calendly:{event_uuid}
Ashby	API Key	Candidates, interview feedback, job postings	ashby:{candidate_id}
BambooHR	API Key + Subdomain	HR policies, company announcements	bamboohr:{announcement_id}
Deel	API Token	Compliance docs, contractor agreements	deel:{contract_id}
Rippling	API Key	HR docs, policy handbooks, team structures	rippling:{document_id}

How to Enable/Disable Connectors

Edit the CONNECTORS_ENABLED variable in your .env:

# Enable only what you use — unconfigured connectors are silently skipped
CONNECTORS_ENABLED=slack,google_docs,github,notion,linear,jira,hubspot

Leave it empty to attempt all 38 connectors (only those with valid credentials will run).

---

6. Full Data Pipeline — Step by Step

Step 1: Ingestion

The SyncScheduler calls get_enabled_connectors() from registry.py. This returns a list of instantiated connectors whose credentials are present. For each connector, ingest(days_back=30) is called and the returned RawDataItem objects are collected.

# Every piece of data becomes this shape regardless of source
class RawDataItem(BaseModel):
    id: str              # e.g., "github:owner/repo:42" or "notion:abc123"
    source: DataSourceType   # enum: "slack", "github", "notion", etc.
    source_id: str       # platform-native ID (channel ID, doc ID, issue ID)
    title: str           # human-readable display name
    content: str         # full normalized text
    author: str          # username or display name
    created_at: datetime
    updated_at: datetime
    raw_metadata: dict   # source-specific extras (labels, state, channel, etc.)

Idempotency check: Before processing, the system calls get_processed_source_item_ids() on the database. This returns the set of all item IDs already processed into chunks. The pipeline filters out any item whose ID is already in this set. This means:

• You can stop the sync halfway through and resume exactly where you left off.
• Running sync again never re-processes already-seen items.

---

Step 2: Chunking — DocumentChunker

Long documents are split into smaller, manageable pieces before being sent to the LLM.

Text Cleaning (always runs first):

- Collapses multiple whitespace characters into single spaces
- Strips control characters (ASCII 0-31, 127-159)
- Replaces URLs with the token [URL]
- Replaces email addresses with the token [EMAIL]

Sliding-Window Sentence Chunking:

• Default chunk size: 5 sentences
• Default overlap: 2 sentences (retains tail context from the previous chunk)
• Uses regex (?<=[.!?])\s+ to split on sentence boundaries (not mid-sentence)
• Chunks shorter than 20 characters are discarded

Why overlap? Without overlap, if a procedure starts at the end of one chunk and continues at the start of the next, the LLM would only see half the context. The overlap ensures key context is preserved across boundaries.

---

Step 3: Knowledge Extraction — KnowledgeExtractor

For every chunk, the extractor makes a single combined LLM call (optimised from two separate calls):

Prompt: "Classify this text as ONE of: procedure, decision, incident, policy, or general.
         Then extract 3-5 key concepts. Return as JSON: { type: ..., concepts: [...] }"

Output: { "type": "procedure", "concepts": ["Handle Refunds", "Verify order status", "Payment processor"] }

Confidence Scoring (no LLM call, pure heuristic):

+0.3  if chunk is longer than 100 words
+0.2  if chunk is 50–100 words
+0.3  if classified as "procedure", "policy", or "decision"
+0.2  if 4+ key concepts were extracted
+0.1  if 2–3 key concepts were extracted
Max score: 1.0

Every processed chunk becomes a ProcessedChunk object and is saved to the database.

---

Step 4: Skill Synthesis — SkillsSynthesizer

This is where the real magic happens.

Phase A — Filtering: Only chunks with confidence_score >= 0.2 move forward.

Phase B — Clustering: Chunks are grouped by their first (primary) key concept. For example, all chunks where the first concept is "Handle Refunds" end up in the same cluster. Clusters with fewer than 2 chunks get merged into a miscellaneous cluster.

Phase C — Skill Generation: For each cluster, the LLM is given all the chunk texts combined and asked to synthesize a complete Skill card:

{
  "name": "clear skill name",
  "description": "what this skill does",
  "category": "refunds / pricing / incidents / ...",
  "procedure_steps": ["Step 1", "Step 2", "..."],
  "decision_points": {"if customer > 30 days": "deny refund"},
  "prerequisites": ["who can perform this", "required access"],
  "success_criteria": ["how to verify completion"],
  "exceptions": ["edge cases to watch for"]
}

The final Skill object is upserted (insert or update) into the SQLite skills table.

---

Step 5: Export

Running python main.py export serializes all skills from the database into a single SkillsFile object and writes it to disk.

{
  "version": "1.0.0",
  "generated_at": "2026-07-25T00:00:00",
  "company_name": "Your Company",
  "skills": [
    {
      "id": "skill-uuid-...",
      "name": "Handle Customer Refunds",
      "description": "...",
      "category": "refunds",
      "procedure_steps": ["Check eligibility", "Verify order", "Issue refund"],
      "decision_points": {"if_disputed": "escalate to manager"},
      "examples": [],
      "prerequisites": ["Customer support access"],
      "success_criteria": ["Refund confirmation sent"],
      "exceptions_and_edge_cases": ["Active subscriptions need billing cancel"],
      "source_items": ["github:myorg/repo:42", "slack-C01-123...", "notion:abc123"],
      "confidence_score": 0.75
    }
  ],
  "metadata": {
    "total_skills": 1,
    "by_category": {"refunds": 1}
  }
}

---

7. Database Design

The system uses SQLite with SQLAlchemy ORM. The database lives at data/company_brain.db (path configurable via DATABASE_URL in .env).

3 Tables:

raw_data_items
├── id (PK)          — unique ID e.g. "github:myorg/repo:42", "notion:abc123"
├── source           — enum: "slack", "github", "notion", "linear", etc.
├── source_id        — platform-native ID (channel ID, doc ID, issue ID)
├── title            — display name
├── content          — full raw text
├── author           — username or display name
├── created_at
├── updated_at
└── raw_metadata     — JSON blob with source-specific fields (labels, state, etc.)

processed_chunks
├── id (PK)          — "chunk-{uuid}"
├── source_item_id   — FK to raw_data_items.id
├── chunk_text       — the actual text block
├── chunk_index      — position within source document
├── key_concepts     — JSON list e.g. ["Refunds", "Policy"]
├── chunk_type       — "procedure", "policy", "decision", "incident", "general"
├── confidence_score — float 0.0 to 1.0
└── created_at

skills
├── id (PK)          — "skill-{uuid}"
├── name             — e.g. "Handle Customer Refunds"
├── description
├── category         — e.g. "refunds"
├── procedure_steps  — JSON list
├── decision_points  — JSON dict
├── examples         — JSON list
├── prerequisites    — JSON list
├── success_criteria — JSON list
├── exceptions_and_edge_cases — JSON list
├── source_items     — JSON list of contributing raw_data_items IDs
├── last_updated
└── confidence_score

---

8. The LLM Integration (Multi-Provider Engine)

Your Company Brain features a pluggable, multi-provider LLM abstraction layer (src/qa/providers/) that decouples knowledge extraction, skill synthesis, and Q&A retrieval from any single model back-end.

Supported Providers & Models

Provider	Supported Models	Description / Use Case
Local (Ollama)	gemma4:e4b	100% offline, privacy-first default running on localhost:11434.
OpenAI (ChatGPT)	gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna	Frontier cloud models with 200K token context window budget.
Claude (Anthropic)	claude-fable-5, claude-opus-5, claude-sonnet-5, claude-haiku-4-5	High-reasoning Claude models with optional extended thinking mode.
NVIDIA NIM (Direct)	nvidia/nemotron-3-ultra-550b-a55b	Your own NVIDIA API quota at integrate.api.nvidia.com — no shared free-tier pool. Requires NVIDIA_NIM_API_KEY. Streaming; reasoning_content stripped automatically.
OpenRouter	openrouter/free, nvidia/nemotron-3-ultra-550b-a55b:free, nvidia/nemotron-3-super-120b-a12b:free, custom slugs	Auto-routing free tier or any custom open-source model slug. Supports 10-model free fallback chain with dynamic context auto-shifting.

Architecture & Provider Interface (LLMProvider)

All providers implement the LLMProvider abstract base class (src/qa/providers/base.py):

class LLMProvider(ABC):
    display_name: str
    max_context_chars: int

    @abstractmethod
    def check_available(self) -> bool:
        """Health check and API authentication verification."""
        ...

    @abstractmethod
    def generate(self, question: str, context: str, history: List[Dict[str, str]]) -> str:
        """Multi-turn Q&A answer generation."""
        ...

    @abstractmethod
    def generate_text(self, prompt: str, max_tokens: int = 512, temperature: float = 0.0) -> str:
        """Raw prompt completion for concept extraction and skill synthesis."""
        ...

JSON Parsing Robustness

Because LLMs sometimes wrap structured JSON in markdown code fences, we extract payloads with regex fallbacks:

json_match = re.search(r'\[.*?\]', result_text, re.DOTALL)  # for arrays
json_match = re.search(r'\{.*\}', result_text, re.DOTALL)   # for objects
if json_match:
    data = json.loads(json_match.group())

---

9. CLI Commands Reference

All commands are run from inside the project root directory using the venv Python interpreter.

# Workspace setup & authentication
ycb init
ycb login --license-key CB-XXXX-XXXX-XXXX --email you@company.com
ycb logout

# Model and provider switching
ycb switch model                    # Interactive categorized menu
ycb switch model --list             # Print all models
ycb switch model gemma4:e4b         # Switch to local Ollama
ycb switch model gpt-5.6-terra      # Switch to OpenAI
ycb switch model claude-sonnet-5    # Switch to Anthropic
ycb switch model openrouter/free    # Switch to OpenRouter

# Connector credential configuration
ycb connect slack
ycb disconnect slack

# Start interactive batch sync (DEFAULT)
# Processes 2 items at a time, shows checkpoints, and auto-exports to output/skills_file.json
ycb sync

# Run continuous full sync (processes everything without pausing)
ycb sync --full

# Stream full debug logs to console (logs are always stored in ~/.ycb/logs/ycb.log)
ycb sync --verbose

# Sync a single connector only
ycb sync --connector slack

# Check current license tier, unlocked connectors, and extracted skills stats
ycb status

# Ask natural-language questions about your company data
ycb --ask "what processes or procedures does our company have documented"

# Scope question to a specific connector only
ycb --ask "what updates or commits were made recently" --connector github

# Export skills as JSON (for AI agents)
ycb export --output output/skills.json

# Export skills as readable Markdown (for humans)
ycb export --output output/skills.md --format markdown

# Disable the Rich dashboard UI (useful for piping output to logs)
ycb sync --full --plain

Interactive Batch Menu (appears after each batch of 2 documents):

[Batch 1 complete. 5 total chunks processed so far.]
Do you want to: (1) Process next batch (2) Synthesize skills now and STOP (3) Quit immediately?

• 1 → Continue to next 2 items
• 2 → Stop ingesting new data, run synthesis on what you have right now, and export
• 3 → Exit immediately without synthesizing

---

10. Terminal Dashboard (UI)

The terminal UI is built with the rich library and runs inside a rich.live.Live context. This means the dashboard panel stays fixed at the top of the terminal while log messages scroll underneath it.

Key components:

• SyncState — A plain Python class that holds the current progress percentage, file count, skill count, task statuses, and elapsed time.
• SyncDashboard — A custom __rich_console__ renderable that reads from SyncState and draws the panel using rich.panel.Panel, rich.console.Group, and rich.progress.Progress.
• ProceduralNetwork — The animated neural network animation. It's NOT pre-built frames. It uses math.sin(time.time() * 3 + offset) to procedurally compute the brightness/state of each node and edge in real time at ~10 FPS.
• DashboardLogHandler — A custom Python logging.Handler that intercepts log messages from the backend (like "Phase 1: Ingesting data") and maps them to UI state updates (progress bar %, which task is active). This is how the backend and UI stay decoupled — the SyncScheduler never knows a UI exists.

---

11. Key Engineering Decisions

Why an Abstract Base Connector + Registry pattern?

With 16 connectors, hardcoding each one into SyncScheduler would create a monolithic, hard-to-maintain sync loop. Instead, BaseConnector enforces a uniform ingest() interface, and registry.py acts as a plugin system. Adding a new connector is three steps: write the class, register it in registry.py, add credentials to .env.example. The scheduler code never changes.

Why SQLite instead of a cloud database?

Privacy-first design. Company data stays on-premise. SQLite also requires zero configuration, which makes setup trivial. The path is configurable via DATABASE_URL if you want to swap in PostgreSQL for production.

Why local LLM (Ollama) instead of OpenAI?

Companies have confidential Slack data, GitHub issues, emails. Sending it to a third-party API is a major legal and trust risk. Ollama + Gemma 4 gives equivalent results while keeping everything local. This is a core value proposition: "your data, your machine, your AI."

Why a single combined LLM call instead of two?

The original design made two LLM calls per chunk: one for classification, one for concept extraction. This was optimised to a single JSON-returning prompt that does both simultaneously, cutting per-chunk latency roughly in half without sacrificing output quality.

Why a sliding window chunker instead of splitting by headers?

We also implemented chunk_by_structure() (splits by # headers). But most Slack messages and informal docs don't have formal headers. The sentence-based sliding window handles messy, unstructured text far better across all 16 source types.

Why concept-overlap clustering instead of embeddings/vector search?

For an MVP, cosine similarity over embeddings requires storing large float arrays and running nearest-neighbor search. Instead, we cluster purely by the primary concept extracted by the LLM. It's O(N) instead of O(N²) and produces very interpretable clusters. Embedding-based clustering is a listed future improvement (embedding field already exists on ProcessedChunk, it's just null for now).

Why the interactive batch system?

Running a full sync across 16 connectors through a local 4B model can take a long time. Users need to be able to stop, inspect partial results, and decide whether to continue. The batch system also lets users validate quality early without committing to the full pipeline run.

Idempotency

Every raw item has a globally unique ID (e.g., github:myorg/repo:42, notion:abc123, slack-C05-1234). Before any processing, we query processed IDs from the DB and filter them out. Running sync 10 times on the same data produces the same result as running it once.

---

12. Directory Structure

company_brain_mvp/
│
├── main.py                              ← CLI entry point (self-relative sys.path setup)
├── requirements.txt                     ← All Python dependencies
├── .env                                 ← Your API keys (gitignored)
├── .env.example                         ← Template for .env with all 38 connectors
├── credentials.json                     ← Google OAuth credentials (gitignored)
├── token.pickle                         ← Saved Google OAuth token (gitignored)
│
├── src/
│   ├── connectors/
│   │   ├── base_connector.py            ← Abstract BaseConnector: ingest(), retry_with_backoff()
│   │   ├── registry.py                  ← Central connector registry: get_enabled_connectors()
│   │   │
│   │   │   ── Tier 1 Connectors ──
│   │   ├── slack_connector.py           ← Slack: messages, thread replies
│   │   ├── google_docs_connector.py     ← Google Docs: paragraph text, tables
│   │   ├── google_sheets_connector.py   ← Google Sheets: tabs as key-value row text
│   │   ├── google_drive_connector.py    ← Google Drive: file metadata & content text
│   │   ├── gmail_connector.py           ← Gmail: emails, threads, message text
│   │   ├── github_connector.py          ← GitHub: issues, PRs, comments (REST API v3)
│   │   ├── notion_connector.py          ← Notion: page blocks, recursive children
│   │   ├── discord_connector.py         ← Discord: server channels, messages
│   │   ├── linear_connector.py          ← Linear: issues, comments (GraphQL API)
│   │   ├── outlook_teams_connector.py   ← Outlook emails + Teams messages (MS Graph)
│   │   │
│   │   │   ── Tier 2 Connectors ──
│   │   ├── gitlab_connector.py          ← GitLab: issues, merge requests, notes
│   │   ├── dropbox_connector.py         ← Dropbox: text file downloads
│   │   ├── analytics_connector.py       ← Mixpanel + Amplitude: event schemas
│   │   ├── search_connector.py          ← Algolia + Exa + Perplexity: search results
│   │   ├── facebook_connector.py        ← Facebook: page posts, comments
│   │   ├── todoist_connector.py         ← Todoist: tasks, project details, comments
│   │   ├── web_crawler_connector.py     ← Web Crawler: scraped web page text
│   │   │
│   │   │   ── Tier 3 Connectors ──
│   │   ├── jira_connector.py            ← Jira: issues, comments, work logs
│   │   ├── asana_connector.py           ← Asana: tasks, projects, subtasks
│   │   ├── clickup_connector.py         ← ClickUp: tasks, docs, custom fields
│   │   ├── airtable_connector.py        ← Airtable: base records & fields
│   │   ├── monday_connector.py          ← Monday.com: boards, updates, items
│   │   ├── basecamp_connector.py        ← Basecamp: message boards, to-dos
│   │   ├── datadog_connector.py         ← Datadog: monitors, alert logs
│   │   ├── segment_connector.py         ← Segment: tracking plans & event schema
│   │   │
│   │   │   ── Tier 4 Connectors ──
│   │   ├── front_connector.py           ← Front: shared inbox conversations
│   │   ├── zoom_connector.py            ← Zoom: meeting transcripts & logs
│   │   ├── twitter_connector.py         ← Twitter / X: tweets, replies, threads
│   │   ├── hubspot_connector.py         ← HubSpot: contacts, deals, ticket notes
│   │   ├── salesforce_connector.py      ← Salesforce: objects, opportunities, cases
│   │   ├── calendly_connector.py        ← Calendly: scheduled events & notes
│   │   ├── ashby_connector.py           ← Ashby: candidate notes, jobs
│   │   ├── bamboohr_connector.py        ← BambooHR: HR policies & announcements
│   │   ├── deel_connector.py            ← Deel: contracts & compliance docs
│   │   └── rippling_connector.py        ← Rippling: policy handbooks & HR docs
│   │
│   ├── processors/
│   │   ├── chunker.py                   ← Sentence-based sliding window chunker
│   │   ├── knowledge_extractor.py       ← gemma4:e4b: single combined classify+extract call
│   │   └── skills_synthesizer.py        ← Cluster chunks → generate Skill cards
│   │
│   ├── models/
│   │   └── domain.py                    ← Pydantic models: RawDataItem, ProcessedChunk, Skill
│   │
│   ├── storage/
│   │   ├── database.py                  ← SQLAlchemy table definitions (3 tables)
│   │   └── storage_manager.py           ← CRUD: save_raw_items, save_chunks, get_processed_ids
│   │
│   ├── licensing/
│   │   ├── validator.py                 ← Online REST verification, machine ID binding & offline grace checks
│   │   └── tier_mapping.py              ← Plan tiers (solo, max, enterprise), self-healing cache & connector gating
│   │
│   ├── qa/
│   │   ├── ask_engine.py                ← Multi-turn Q&A retrieval, history management
│   │   └── providers/                   ← Pluggable LLM engines
│   │       ├── base.py                  ← LLMProvider ABC + ProviderError + create_openai_client
│   │       ├── ollama_provider.py       ← Local Ollama (gemma4:e4b, fully offline)
│   │       ├── openai_provider.py       ← OpenAI GPT models
│   │       ├── anthropic_provider.py    ← Anthropic Claude models
│   │       ├── openrouter_provider.py   ← OpenRouter: free fallback chain + auto-shifting
│   │       ├── nvidia_nim_provider.py   ← NVIDIA NIM Direct: integrate.api.nvidia.com (personal quota)
│   │       └── __init__.py              ← Provider registry, MODEL_CONTEXT_CHARS, load_active_provider
│   │
│   ├── sync/
│   │   └── sync_scheduler.py            ← Orchestrates full & interactive sync via registry
│   │
│   └── cli/
│       ├── main.py                      ← Click commands: init, login, logout, sync, status, export, switch, connect, disconnect
│       └── ui/
│           ├── animation.py             ← ASCII procedural neural network animation
│           ├── branding.py              ← Solid filled-block ASCII branding & first-login overview
│           ├── dashboard.py             ← Rich Live ASCII dashboard (SyncDashboard, SyncState)
│           ├── help_guide.py            ← Task-oriented step-by-step help guide & command reference
│           ├── logger.py                ← Custom logging.Handler → UI state bridge
│           └── components.py            ← Rich tables/panels for status & export
│
├── tests/                               ← Unit test suite for all 38 connectors
│
├── data/
│   └── company_brain.db                 ← SQLite database (auto-created on first run)
│
└── output/
    ├── skills_file.json                 ← Final structured output for AI agents
    └── skills_file.md                   ← Human-readable version