WSO2 Docs MCP Server

A production-ready Model Context Protocol (MCP) server that provides AI assistants (Claude Desktop, Claude Code, Cursor, VS Code) with semantic search over WSO2 documentation via Retrieval-Augmented Generation (RAG).

Under the hood, it uses a blazing-fast dual-ingestion engine:

• GitHub Native: Fetches raw Markdown directly from WSO2's public GitHub repositories via the Git Trees API (avoids web-scraping noise and rate limits)
• Web Crawl Fallback: For products without dedicated GitHub docs repos (like the WSO2 Library)

Architecture

flowchart TD
    %% Styling
    classDef github fill:#1f2328,color:#fff,stroke:#e1e4e8
    classDef default fill:#0969da,color:#fff,stroke:#0969da
    classDef db fill:#218bff,color:#fff,stroke:#218bff
    classDef web fill:#0969da,color:#fff,stroke:#0969da

    subgraph Sources ["Information Sources"]
        GH["GitHub Repos\n(wso2/docs-apim, etc.)"]:::github
        Web["WSO2 Websites\n(ballerina.io, lib)"]:::web
    end

    subgraph Ingestion ["Dual-Ingestion Pipeline"]
        A["GitHubDocFetcher\n(Git Trees API)"] 
        B["DocCrawler\n(HTML scraping)"]
        
        C["MarkdownParser\n(Front-matter & Heads)"]
        D["DocParser\n(Cheerio HTML parsing)"]
        
        E["DocChunker\n(Semantic Splitting)"]
    end

    subgraph Embedding ["Vectorization & Storage"]
        F["EmbedderFactory\n(Ollama / HuggingFace)"]
        G[("pgvector\n(PostgreSQL)")]:::db
    end

    %% Flow
    GH --> A
    Web --> B
    
    A -->|"Raw .md"| C
    B -->|"Clean HTML"| D
    
    C -->|"ParsedSection[]"| E
    D -->|"ParsedSection[]"| E
    
    E -->|"Tokens/Chunks"| F
    F -->|"768-dim Vectors"| G

Documentation Sources

Prerequisites

• Node.js ≥ 20
• Docker (for pgvector)
• Embeddings — no API key required by default:
  • Ollama (recommended) — runs locally, model auto-downloaded on first run
  • If Ollama is not running, the server automatically falls back to HuggingFace ONNX (in-process, also downloads automatically)
  • Cloud providers are also supported: OpenAI, Google Gemini, Voyage AI

Quick Start

Choose the setup path that fits your use case:

• Install from npm — simplest, no cloning required
• Clone and build — for development or contributions

Install from npm

Install the package globally to get the wso2-docs-mcp-server, wso2-docs-crawl, and wso2-docs-migrate commands available system-wide:

npm install -g wso2-docs-mcp-server

Prefer no global install? You can use npx wso2-docs-mcp-server, npx wso2-docs-crawl, and npx wso2-docs-migrate in every step below — just replace the bare command with its npx equivalent.

1. Start pgvector

Download the docker-compose.yml and start the database:

curl -O https://raw.githubusercontent.com/iamvirul/wso2-docs-mcp-server/main/docker-compose.yml
docker compose up -d

2. Start Ollama (optional but recommended)

Install Ollama and pull the default embedding model:

ollama pull nomic-embed-text
ollama serve

No Ollama? Skip this step. The server automatically falls back to HuggingFace ONNX — model downloads on first use with no extra setup.

3. Run database migration

DATABASE_URL="postgresql://wso2mcp:wso2mcp@localhost:5432/wso2docs" \
  wso2-docs-migrate

... View full README on GitHub