@portkey-ai/mcp-tool-filter

Ultra-fast semantic tool filtering for MCP (Model Context Protocol) servers using embedding similarity. Reduce your tool context from 1000+ tools down to the most relevant 10-20 tools in under 10ms.

Features

• ⚡ Lightning Fast: <10ms filtering latency for 1000+ tools with built-in optimizations
• 🚀 Performance Optimized: 6-8x faster dot product, smart top-K selection, true LRU cache
• 🎯 Semantic Understanding: Uses embeddings for intelligent tool matching
• 📦 Zero Dependencies on Runtime: Only requires an embedding provider API
• 🔄 Flexible Input: Accepts chat completion messages or raw strings
• 💾 Smart Caching: Caches embeddings and context for optimal performance
• 🎛️ Configurable: Tune scoring thresholds, top-k, and always-include tools
• 📊 Performance Metrics: Built-in timing for optimization

Installation

npm install @portkey-ai/mcp-tool-filter

Quick Start

import { MCPToolFilter } from '@portkey-ai/mcp-tool-filter';

// 1. Initialize the filter (choose embedding provider)

// Option A: Local Embeddings (RECOMMENDED for low latency < 5ms)
const filter = new MCPToolFilter({
  embedding: {
    provider: 'local',
  }
});

// Option B: API Embeddings (for highest accuracy)
const filter = new MCPToolFilter({
  embedding: {
    provider: 'openai',
    apiKey: process.env.OPENAI_API_KEY,
  }
});

// 2. Load your MCP servers (one-time setup)
await filter.initialize(mcpServers);

// 3. Filter tools based on context
const result = await filter.filter(
  "Search my emails for the Q4 budget discussion"
);

// 4. Use the filtered tools in your LLM request
console.log(result.tools); // Top 20 most relevant tools
console.log(result.metrics.totalTime); // e.g., "2ms" for local, "500ms" for API

Embedding Provider Options

Local Embeddings (Recommended)

Pros:

• ⚡ Ultra-fast: 1-5ms latency
• 🔒 Private: No data sent to external APIs
• 💰 Free: No API costs
• 🌐 Offline: Works without internet

Cons:

• Slightly lower accuracy than API models
• First initialization downloads model (~25MB)

const filter = new MCPToolFilter({
  embedding: {
    provider: 'local',
    model: 'Xenova/all-MiniLM-L6-v2', // Optional: default model
    quantized: true, // Optional: use quantized model for speed (default: true)
  }
});

Available Models:

• Xenova/all-MiniLM-L6-v2 (default) - 384 dimensions, very fast
• Xenova/all-MiniLM-L12-v2 - 384 dimensions, more accurate
• Xenova/bge-small-en-v1.5 - 384 dimensions, good balance
• Xenova/bge-base-en-v1.5 - 768 dimensions, higher quality

Performance:

• Initialization: 100ms-4s (one-time, downloads model)
• Filter request: 1-5ms
• Cached request: <1ms

API Embeddings

For highest accuracy, use OpenAI or other API providers:

const filter = new MCPToolFilter({
  embedding: {
    provider: 'openai',
    apiKey: process.env.OPENAI_API_KEY,
    model: 'text-embedding-3-small', // Optional
    dimensions: 384, // Optional: match local model for fair comparison
  }
});

Pros:

• 🎯 Highest accuracy: 5-15% better than local
• 🔄 Easy to switch models
• 🌐 No local resources needed

Cons:

• 🐌 Slow: 400-800ms per request
• 💰 Costs money: ~$0.02 per 1M tokens
• 🔒 Data sent to external API
• 📶 Requires internet connection

Performance:

• Initialization: 200ms-60s (depends on tool count)
• Filter request: 400-800ms
• Cached request: 1-3ms

Quick Comparison

| Aspect | Local | API | Winner | |--------|-------|-----|--------| | Speed | 1-5ms | 400-800ms | 🏆 Local (200x faster) | | Accuracy | Good (85-90%) | Best (100%) | 🏆 API | | Cost | Free | ~$0.02/1M tokens | 🏆 Local | | Privacy | Fully local | Data sent to API | 🏆 Local | | Offline | ✅ Works offline | ❌ Needs internet | 🏆 Local | | Setup | Zero config | Needs API key | 🏆 Local |

**📊 See TRADEOFFS.md for detailed a

... View full README on GitHub