I Tried AWS Bedrock Knowledge Bases for the First Time. Here’s What Happened.
My honest experience building an intelligent app using Bedrock and its built-in retrieval system
I’ve been following the rise of AWS Bedrock with curiosity — especially when they announced the Knowledge Bases feature. A managed RAG (retrieval-augmented generation) service that connects Bedrock models to your own data? Yes, please.
So I gave it a shot. Here’s what worked, what was weird, and what I wish I knew going in.
🚀 What I Wanted to Build
A simple Q&A bot for my study notes. The idea: upload a bunch of Word or PDF files, let AWS handle the embeddings and retrieval, and let the LLM (Claude, in my case) answer user questions contextually.
No fine-tuning, no building vector stores, no Reinvent-level infrastructure.
🔧 Getting Started Was… Surprisingly Smooth
I used the AWS Console to create a Knowledge Base linked to an S3 bucket. Bedrock took care of:
- Creating a vector store under the hood
- Chunking my documents
- Generating embeddings using Titan Embeddings
- Hooking it all up to Claude for Q&A
I had it running in about 20 minutes — no infrastructure setup, no training pipeline, just upload and go. For a managed service, that’s impressive.
😕 But There Were Some Surprises
1. Limited file support: I had to pre-process my Word file into plaintext. PDFs were hit-or-miss — especially when they contained figures or images, which the parser struggled to handle cleanly.
2. No visibility into what’s stored: You can’t see the actual chunks or embeddings in the vector store. It’s truly managed — and that’s a double-edged sword.
3. No custom chunking or logic: If you want to tweak how your documents are broken up or how context is selected, you’re out of luck (for now).
4. Model choice is limited: Only certain models support Knowledge Base Q&A. Claude worked well, but if you wanted a Titan model or more fine control, it’s not there yet.
5. Unexpected OpenSearch Costs: Even though everything ran in serverless mode, I was surprised to see OpenSearch usage from the underlying vector store pushing up my AWS bill more than expected. 💸💸💸
💡It’s a good reminder that “serverless” doesn’t always mean “cheap” — especially when managed services run opaque workloads behind the scenes.
🌐 What Worked Surprisingly Well
- Claude’s answers were surprisingly strong — especially when the input was well-organized and free of clutter.
- The latency wasn’t bad, even for multi-hop queries.
- Zero infra to maintain. That’s underrated.
- It scales out of the box — I didn’t touch a single CPU config.
✅ Verdict: A Solid V1, If You Keep It Simple
Bedrock Knowledge Bases feel like a great starting point for internal tools, MVPs, or fast prototyping — especially if you’re already in AWS.
But if you want more control over chunking, retrieval logic, or multi-source blending, you’ll hit the limits quickly.
The unexpected OpenSearch cost was a bit of a bummer — something to watch for if you’re spinning this up for more than just testing.
Still, for “LLM + your docs, fast” use cases? It’s damn good.
🙋️ Over to You
Tried Bedrock Knowledge Bases yourself? Curious how it stacks up to LangChain or custom pipelines? Let me know your thoughts, wins, or frustrations — I’m all ears.
If this helped you rethink how you’re using LLMs, hit the 👏 or drop a comment — I’d love to hear how you’re stacking your prompts, retrieval, and tuning.