Generative AI Starter

Table of Contents

1. Purpose & Features
2. Architecture & Repository Structure
3. Getting Started
4. Usage
5. Testing
6. Optional: Langfuse Support
7. GenAI Teacher Chat Mode
8. Troubleshooting & Codespaces Notes
9. Further Reading

Purpose & Features

This repository can be used as a starting point for building custom LLM applications using Open Source tooling and models. It incorporates Ollama, Open WebUI, Langchain, Streamlit, Chroma, and PGVector using Docker to containerize the application and Docker Compose to run the various service dependencies.

Key Features:

• LLM integration (Ollama, LangChain)
• RAG (Retrieval Augmented Generation) examples
• Agentic AI patterns (ReAct, multi-tool agents)
• Streamlit web interface
• Vector database support (Chroma, PGVector)
• Integrated teaching assistant (GenAI Teacher)

Architecture & Repository Structure

Main Components:

• src - Application code and examples
• etc - Scripts and configuration
• tests - Unit and integration tests
• Dockerfile, compose.yml - Containerization and orchestration

Getting Started

Prerequisites

• Docker Engine installed
  • Option 1: Docker Engine configured with >= 12 GB memory
  • Option 2: Docker Engine configured with >= 8 GB memory and Ollama Executable installed

    brew install ollama
    

Optional Prerequisites (for "Web" example)

• Register for a free Brave Search API key here
  • 2000 calls/month free, affordable scaling
  • Brave Search API info
• Add your key to a .env file in the project root:

  echo "BRAVE_SEARCH_API_KEY=your_key_here" >> .env
  

Usage

Running the Backing Services and LLM

Option 1 (CPU-only, easiest):

docker compose --profile=cpu up -d

Option 2 (GPU, fastest):

docker compose up -d
./etc/ollama_entrypoint.sh

Option 3 (GitHub Codespaces):

1. Open in Codespaces: Click "Code" → "Codespaces" → "Create codespace"
2. Setup environment:

   docker compose --profile=cpu up -d
   

3. Access the application:
   • Streamlit App: Auto-opens or check "Ports" tab
   • Open WebUI: Available on port 3000

Common Docker Compose Commands

docker compose --profile=cpu up -d      # Start all services
docker compose down                     # Stop all services
docker compose logs -f                  # View service logs
docker compose ps                       # Check service status
docker compose run --rm app pytest tests/ -v   # Run tests

Accessing the Demo App

• Streamlit: http://localhost:8501/
• Open WebUI: http://localhost:3000/

MCP Servers & Standardized Tool Loading

This project includes support for Model Context Protocol (MCP) servers, enabling standardized tool loading from OpenAPI specifications.

PokéAPI MCP Example

The project includes a complete MCP server generated from the PokéAPI OpenAPI specification, demonstrating how to:

• Convert REST APIs to standardized MCP tools
• Deploy tool servers independently via Docker
• Load tools dynamically in LangChain agents
• Test tools through standardized MCP protocols

What's Included:

• mcp-pokemon/ - Generated MCP server for PokéAPI

... View full README on GitHub