rag-pipeline-utils

RAG for Node.js that tells you when your LLM is making things up.

Your chatbot just told someone they get thirty vacation days. The handbook says twenty. This library catches that: it checks every sentence of an answer against the sources it was given, numbers, "not"s and conditions included, with no extra model calls. It also builds the pipeline for you, if you want one.

npm install @devilsdev/rag-pipeline-utils

Node.js 22+. CommonJS, ESM and TypeScript types out of the box.

New in 3.9

• Dropped conditions are caught. An answer that leaves out a condition its source sets, like "with manager approval", now fails the check.
• Much faster on long answers. A 1,000-sentence answer is checked in under half a second.
• Cleaner text from PDFs and web pages. It's normalised before it's compared.
• /edge without nodejs_compat (3.9.1). Cached pipelines and cost estimates on Cloudflare Workers no longer throw "process.emitWarning is not a function"; the warning goes to console.warn.

Release notes · Changelog

Upgrading from 3.8.1 or earlier? Upgrade to 3.8.2 or later for its security fixes; 3.9.0 is recommended. If you store documents with top-level access restrictions in Qdrant, Pinecone, Weaviate or pgvector, re-ingest them: see 3.8.2.

Catch the made-up bits

Bring an answer from anywhere: this library, LangChain, Mastra, the AI SDK or LlamaIndex. Pass it in with the sources it was given:

import { checkAnswer } from "@devilsdev/rag-pipeline-utils";

const check = await checkAnswer({
  answer:
    "Full-time employees receive thirty days of paid vacation per year. " +
    "Remote work is allowed two days per week.",
  sources: [
    {
      pageContent:
        "Full-time employees receive twenty days of paid vacation per year.",
    },
    {
      pageContent:
        "Remote work is allowed two days per week with manager approval.",
    },
  ],
});

console.log(check.grounded); // false
console.log(check.groundedness); // 0
console.log(check.unsupported); // ["Full-time employees receive thirty days of paid vacation per year.", "Remote work is allowed two days per week."]

Both sentences are caught: the

... View full README on GitHub