Infrastructure Decisions for Production RAG Systems : Quantifying the Build versus Buy Trade-off in Financial Services

Sammanfattning

Retrieval-Augmented Generation (RAG) enables large language models to ground their responses in external documents, yet organizations deploying RAG in production face a key infrastructure decision: build a custom pipeline from individual cloud components, or buy an integrated managed platform. This thesis compares the Build versus Buy approach on Amazon Web Services using a collection of SEC 10-K financial filings. A custom-orchestrated stack (Amazon EKS, Aurora PostgreSQL with pgvector, and Reciprocal Rank Fusion) is benchmarked against a fully managed stack (Amazon Bedrock Knowledge Bases with OpenSearch Serverless) on retrieval accuracy, query latency, generation quality, and total cost of ownership. Results show that the custom stack achieves higher retrieval accuracy (Recall@1 of 0.50 versus 0.26) and approximately half the query latency. When a state-of-the-art generation model is used, both stacks produce comparable answer quality; however, a less capable model reveals statistically significant degradation in the managed stack, indicating that retrieval quality becomes critical under model-budget constraints. The custom stack is cheaper in fixed infrastructure cost, saving $1,200 per month, but its higher implementation effort limits the allowable maintenance budget to 10-13 engineering hours per month before the cost advantage is erased. These findings provide empirical guidance for organizations weighing control and cost efficiency against implementation effort when deploying RAG systems on cloud infrastructure.