SpecialAgent

Connect AI to Unreal Engine 5

Full Python API access • 71+ level design tools • Visual feedback loop

---

What is SpecialAgent?

SpecialAgent bridges AI assistants and Unreal Engine 5 through the Model Context Protocol (MCP). Connect Claude, GPT, or any MCP-compatible LLM directly to your editor and control it through natural language.

At its core, SpecialAgent provides unrestricted Python execution with full access to UE5's Python API—meaning your AI assistant can do anything the editor can do. On top of that foundation, 71+ purpose-built tools handle common level design tasks without writing a single line of code.

Native HTTP/SSE transport. No external bridges or dependencies.

---

Features

Two Layers of Power

Full Python Access

Execute arbitrary Python with complete unreal module access. Your AI assistant can:

• Import and process assets
• Create and modify Blueprints
• Generate materials and textures
• Automate project configuration
• Build custom editor utilities
• Run validation and QA checks
• Anything the UE5 Python API supports

This is the unlimited foundation. If you can script it, AI can do it.

Level Design Toolkit

71+ specialized tools for world-building workflows:

Category	Capabilities
Actors	Spawn, transform, duplicate, delete, batch operations
Patterns	Grid, circular, spline, and scatter placement
Landscape	Sculpt height, flatten, smooth, paint material layers
Foliage	Paint vegetation with density control
Lighting	Spawn and configure lights, build lightmaps
Streaming	Manage sub-levels for open worlds
Navigation	Rebuild NavMesh, test pathfinding
Performance	Analyze statistics, detect overlaps
Organization	Folders, tags, labels, selection management

Visual Feedback Loop

Capture viewport screenshots and return them to vision-enabled LLMs. Your AI assistant can see what it built, evaluate the results, and refine its approach.

Describe intent → Execute → Screenshot → AI analyzes → Iterate

---

Installation

Requirements

• Unreal Engine 5.6 or later
• Windows, Mac, or Linux
• MCP-compatible client (Cursor, Claude Desktop, etc.)

Setup

1. Clone or download this repository into your project's Plugins folder:

   YourProject/
   └── Plugins/
       └── SpecialAgent/
   

2. Regenerate project files (right-click .uproject → Generate Visual Studio/Xcode project files)

3. Build and launch your project

4. Enable the plugin in Edit → Plugins → Search "SpecialAgent"

5. Restart the editor

---

Quick Start

1. Verify the Server

Once the editor launches, check the Output Log for:

LogSpecialAgent: MCP Server started on port 8767

Or test with curl:

curl http://localhost:8767/health

2. Configure Your MCP Client

Add SpecialAgent to your MCP client configuration:

{
  "mcpServers": {
    "SpecialAgent": {
      "url": "http://localhost:8767/sse",
      "transport": "sse"
    }
  }
}

3. Connect and Build

Your AI assistant now has access to:

• Python execution with full UE5 API
• 71+ level design tools
• Viewport screenshot capture
• Editor utilities (save, undo, redo)

---

Service Categories

Service	Methods	Description
Python	3	Execute scripts, run files, list modules
Screenshot	2	Capture viewport for AI vision
World	30+	Actor manipulation and spatial queries
Assets	4	Content Browser search and inspection
Landscape	5	Terrain sculpting and layer painting
Foliage	3	Vegetation painting and removal
Lighting	4	Light spawning and configuration
Streaming	4	Sub-level loading and visibility
Performance	3	Statistics and overlap analysis
Navigation	2	NavMesh building and path

... View full README on GitHub