RAG in .NET

Overview

Retrieval augmented generation (RAG) improves response quality by injecting trusted, relevant context into prompts. Instead of relying only on model memory, the system retrieves grounded data from a knowledge base.

RAG Flow

sequenceDiagram
    participant U as User
    participant API as .NET API
    participant EMB as Embedding Service
    participant VS as Vector Store
    participant LLM as LLM

    U->>API: Ask question
    API->>EMB: Create embedding for query
    EMB-->>API: Query vector
    API->>VS: Top-k similarity search
    VS-->>API: Relevant chunks
    API->>LLM: Prompt with chunks
    LLM-->>API: Grounded answer
    API-->>U: Response + citations

Practical Example - Retrieval Interface

public sealed record RetrievedChunk(string Id, string Text, double Score);

public interface IVectorSearch
{
    Task<IReadOnlyList<RetrievedChunk>> SearchAsync(string query, int topK, CancellationToken ct);
}

public sealed class RagService
{
    private readonly IVectorSearch _vectorSearch;
    private readonly ILlmClient _llmClient;

    public RagService(IVectorSearch vectorSearch, ILlmClient llmClient)
    {
        _vectorSearch = vectorSearch;
        _llmClient = llmClient;
    }

    public async Task<string> AskAsync(string question, CancellationToken ct)
    {
        var chunks = await _vectorSearch.SearchAsync(question, topK: 5, ct);
        var context = string.Join("\n\n", chunks.Select(c => c.Text));
        var prompt = $"Answer with citations. Context:\n{context}\nQuestion: {question}";
        return await _llmClient.CompleteAsync(prompt, ct);
    }
}

Production Example - Data Pipeline

A production RAG pipeline usually includes document ingestion, chunking strategy, metadata tagging, embedding generation, and index refresh jobs.

Things to Watch Out For

Key Takeaways