Building a production RAG assistant in 12 weeks — what worked, what didn't

The starting point

I've built Laravel apps for years. MySQL, Blade templates, Docker, standard PHP shop stuff. Zero production AI experience.

The trigger was a real business problem at Depot RTB — my warehouse management platform for veterinary clinics in Thailand. Clinic staff kept asking product questions ("What's the loading dose for Metacam on a 15kg dog?"), policy questions ("What temperature range for zone B cold storage?"), procedure questions ("What documents do I need to order controlled medications?"). Answers existed — scattered across product spec sheets, SOP PDFs, business process docs.

Traditional lookup: 3-8 minutes per query. Multiply by 30 queries per day. Multiply by 20 clinics. That's a lot of wasted time, and nobody was going to memorize 200 pages of SOPs.

I gave myself 12 weeks to ship production RAG. Not learn RAG — ship it, with monitoring, safety, and cost tracking. This is what I actually built and what I got wrong.

Weeks 1-4: Foundations (skip if you've done this)

Boring but necessary. Three things stood out:

• Abstract the SDK. I built a ClaudeService wrapper around Anthropic's PHP SDK from day one. When cache tracking landed in Week 11, I updated one class instead of grep-and-replacing 20 files.
• SSE streaming is not optional. Users staring at a spinner for 20 seconds = users closing the tab. Streaming first token in 2 seconds = users perceive the app as fast. Same total generation time, completely different UX.
• Build something useful in Week 4. I built a document summarizer as my "hello world" for LLM integration. Not part of Depot Assistant — just a working thing that taught me the streaming controller pattern I'd reuse everywhere.

Weeks 5-8: RAG deep dive — the surprising parts

This is where I expected to spend most of the interesting time. Instead I spent it rejecting things.

Anti-patterns I rejected empirically

• Reranking. Added a Voyage reranker to top-20 retrieved docs. Latency went up 500-800ms. Quality on my 15-question golden set: unchanged. Removed.
• Hybrid search (BM25 + vector). Marginal quality gain, doubled the query complexity, added Postgres pg_trgm dependency. For Thai text specifically, BM25 struggled (tokenization issues). Removed.
• HyDE (Hypothetical Document Embeddings). Ask LLM to generate a fake answer, embed that, retrieve on the fake embedding. Idea: fake answer might match real docs better than the raw question. Reality on Thai queries: worse retrieval. The LLM's fake answers drifted from source vocabulary. Removed.

What actually worked

• Chunk size 400 + overlap 50. I tested 4 chunking strategies (fixed, recursive, semantic, structural) with the same golden dataset. Fixed 400/50 hit 100% at retrieval — same as the more complex strategies. Simpler wins.
• Voyage voyage-3-large for Thai. OpenAI ada embeddings on Thai text = poor semantic clustering (I tested). Voyage's newer models are trained on more multilingual corpus and handle Thai + English mixed queries cleanly.
• Postgres + pgvector. I already had Postgres. pgvector added vector search without a separate service. No Pinecone, no Weaviate, no ops overhead. If your corpus is under 1M chunks, this is the right answer.

Lesson: measure before adopting. Every "clever" RAG technique had a cost. Most weren't worth it on my corpus.

Weeks 9-12: The stuff nobody talks about

Foundations and RAG are the fun parts. What follows is the production reality that separates a demo from a shippable product.

Week 9: Eval framework (with a bug that hid 40% of true quality)

I built an LLM-as-judge to score answers on 3 dimensions (accuracy, groundedness, citation quality). Ran the full 30-question golden set. Pass rate: 50%.

That felt low. I dug in. The judge was scoring answers as "hallucinated" when the source content clearly contained the claim. Why?

The judge received only document titles in its context, not full content. So when an answer said "Metacam has a 6-month post-opening shelf life" and the judge only saw "Metacam Oral Suspension 32ml" as the title, it couldn't verify. Score: hallucinated.

Fixed the judge to receive full document text. Rerun: pass rate 90%. Real quality was there all along — I just wasn't measuring correctly.

Lesson: your eval framework can lie to you. Test the judge before trusting the scores.

Week 10: Safety layers ($4.20/month overhead)

Two production requirements: don't leak PII, don't get jailbroken.

• PII redaction: regex patterns for Thai national ID (13-digit format), phone numbers (Thai formats), email, credit card. Log queries with PII detected, redact before storage. Zero LLM cost.
• Injection detection: 2-layer defense. Layer A = regex patterns for obvious attacks ("ignore instructions", "you are now"). Layer B = Haiku classifier for semantic detection when regex misses. Cost: ~$4.20/month at 1000 queries/day.

These add real overhead but they're non-negotiable for a client-facing product. The classifier catches cases regex misses (rephrased injections, Thai-language attacks, social engineering).

Week 11: Cost optimization (85% cache + 66% Haiku routing)

Blended cost per query started at $0.045. Two techniques got it to $0.038.

• Prompt caching. Anthropic's cache_control marker on the sources block. 5-minute TTL. First call = write (25% premium). Repeated calls = 85% savings on the cached portion. Real-world hit rate on bursty clinic workload: 41%.
• Smart model routing. Rule-based classifier: no relevant docs → Haiku (refusal is trivial); reasoning keywords → Sonnet (safe default); multi-hop retrieval → Sonnet. Haiku pass rate on A/B test: 9/10 (equal to Sonnet) at 33% of the cost.

Combined savings: 29% off baseline. Not the 80% you see in optimistic blog posts, but measured production reality.

Week 12: Production deployment (nginx buffering killed my streaming)

I'd been running php artisan serve in dev. Streaming worked. Moved to production: nginx + php-fpm. Streaming broke — full response arrived at the 20-second mark instead of word-by-word.

Root cause: nginx buffers by default. Also: php-fpm buffers. Also: my frontend was rendering markdown only on final chunk arrival.

Fix required 3 layers:

• nginx: proxy_buffering off on the SSE endpoint (not globally)
• php-fpm: fastcgi_buffering off + longer read timeout
• Frontend: render markdown incrementally per chunk, not at stream end

All three together = streaming that feels instant.

What broke (and how I caught it)

Every project has bugs. What matters is the discipline of catching them and being honest about them. Three that would have shipped to production if I hadn't tested rigorously:

Bug 1: LLM judge scoring on titles only

Covered above. The scariest kind of bug — a working system that lies about its own quality.

Bug 2: SDK positional args scrambled

I called Anthropic's PHP SDK using createStream(...$params). Trace showed createStream(1024, Array, 'sonnet', NULL, NULL, ...). The SDK expected named arguments; PHP spread operator sent positional. Args ended up in the wrong slots.

Fix: explicit named args in both complete() and streamComplete() methods. Documented in the code comments so future-me doesn't repeat this.

Bug 3: ContextBuilder regression from refactor

Week 11 refactor left placeholder variables never populated. The system prompt showed source documents labeled [doc_1] through [doc_5], but the actual content was mismatched — [doc_1] label with [doc_5] content. The LLM couldn't cite correctly and refused answers it should have given.

Caught by running eval-retrieval after every ContextBuilder change. Would have shipped otherwise.

Lesson: bugs from silent refactors are the hardest to catch. End-to-end tests after every change to critical paths are worth the tedium.

Real production metrics

Measured on the actual Depot RTB deployment, not projections:

Nothing extraordinary. Everything measured. That's the whole point.

What I'd do differently

• Build eval framework Week 3, not Week 9. If I'd caught the judge bug earlier, I'd have trusted my measurements sooner and stopped second-guessing quality. Eval infrastructure is force multiplier for every subsequent decision.
• Skip Reranking/HyDE experiments earlier. I spent nearly a week testing these. Reading the blogs, I should have known — those techniques help specific corpus types, not general Thai + English mixed content. Ship the simple thing first, add complexity when measurement demands it.
• Ship v0.1 to a real user Week 6, not Week 8. Real usage exposes bugs and priorities that eval datasets don't. Two weeks earlier = two weeks more real feedback.
• Write the runbook in Week 1. I wrote deployment docs in Week 12. If I'd started with them, I would have made cleaner architectural decisions (fewer magic files, more explicit config).

What ships as Depot Assistant

Depot RTB was the first deployment. Everything that shipped there — the code, the safety layers, the monitoring, the docs — ships identically to every future deployment.

Depot Assistant is now a productized RAG platform for Laravel apps. If your team is looking at "we need to add AI features" and doesn't want to become an AI shop yourself, that's the audience.

Pricing is flat $2,500 for install + configuration in your infrastructure. You own the code + docs. Case study (with all the metrics above) lives at /case-study.html. Landing at depot-ai.potissolution.com.

Or just email me: depot-ai@potissolution.com. No pitch — I'll ask about your app and tell you honestly if this fits or not.

Questions or feedback?

I write about production Laravel + AI on Twitter and LinkedIn. If this was useful, share it with someone building on Laravel. If I got something wrong, please tell me — I want to know.