KeyboardExtAX

Fast, read-only accessibility snapshots of active third-party iOS keyboard extensions in Simulator.

Why this project exists

A custom iOS keyboard runs in a separate app-extension process. When a client app presents that keyboard, ordinary UI automation usually sees the client application's accessibility tree but not the keyboard's useful descendants. The entire keyboard may appear as one opaque element.

That creates a practical automation gap:

• screenshots can show the keyboard, but extracting reliable tap coordinates from pixels is slow and brittle;
• the client application's accessibility snapshot does not necessarily expose the extension's keys and controls;
• starting a new XCTest session for every query is expensive and can disturb keyboard focus;
• XCTest element handles become stale whenever the keyboard changes layout;
• automation agents should not need to manage test plans, ports, build products, or runner processes.

KeyboardExtAX fills that gap. It attaches XCTest directly to the already-running keyboard extension by bundle identifier, parses its accessibility hierarchy, and returns plain JSON with absolute logical coordinates. A persistent runner keeps warm snapshots fast, while a small controller owns building, caching, ports, and per-simulator lifecycle.

KeyboardExtAX observes only. Use a separate automation tool, such as XcodeBuildMCP, to launch apps, focus text fields, and perform gestures with the returned coordinates.

Quick start: MCP

Configure keyboard-ext-ax-mcp as a stdio MCP server. Most MCP clients use a configuration shaped like this:

{
  "mcpServers": {
    "KeyboardExtAX": {
      "command": "keyboard-ext-ax-mcp",
      "env": {
        "DEVELOPER_DIR": "/Applications/Xcode.app/Contents/Developer"
      }
    }
  }
}

If the MCP client does not inherit your shell PATH, replace the command with the absolute path printed by command -v keyboard-ext-ax-mcp.

The server exposes one tool:

keyboard_snapshot(
  simulator_id: "<SIMULATOR_UDID>",
  extension_bundle_id: "com.example.keyboard.extension",
  raw: false
)

Example input:

{
  "simulator_id": "2D78332B-A3C3-4CEE-962B-99EF531FD201",
  "extension_bundle_id": "com.example.keyboard.extension"
}

The result is returned as structured MCP content. The MCP layer delegates directly to the same controller used by the CLI, so build caching, sessions, errors, and output are identical.

Quick start: CLI

With the client text field focused and the requested keyboard visible:

keyboard-ext-ax snapshot \
  --simulator <SIMULATOR_UDID> \
  --extension com.example.keyboard.extension

Add --output /tmp/keyboard.json to write the result to a file. Add --raw for the nested tree and XCTest diagnostics.

Scope and limitations

KeyboardExtAX currently supports:

• macOS hosts;
• iOS Simulator;
• third-party keyboard extensions;
• Xcode 26.5;
• read-only accessibility snapshots.

It does not:

• tap, swipe, or type;
• launch or foreground the client app;
• select or enable a keyboard for the user;
• access physical iOS devices;
• guarantee compatibility with widgets, Live Activities, or other extension types;
• guarantee parser compatibility with untested Xcode versions.

The parser reads XCUIApplication.debugDescription. Its text format is not a documented compatibility contract, so each Xcode version must be validated before support is claimed.

Requirements

• macOS
• Xcode 26.5 with the desired iOS Simulator runtime installed
• Python 3.10 or newer
• A keyboard app and extension already installed in the target simulator
• The keyboard enabled in Settings → General → Keyboard → Keyboards
• A foreground client app with a focused text field

If multiple Xcode installa

... View full README on GitHub