Systematically Improving RAG Applications

Stop building RAG systems that shine in demos but fail in production. In six weeks, you will master a systematic, data-driven approach - from synthetic evaluation to intelligent request routing - and transform a prototype into a mission-critical solution.

What You Will Receive

• Improvement Flywheel: creation of synthetic datasets to accurately identify weak spots even before the system launch.
• Fine‑tuning Framework: training custom embeddings with minimal data (starting from 6,000 examples) to achieve an accuracy increase of 20–40%.
• Feedback Acceleration: design of interfaces that gather five times more quality feedback without irritating users.
• Segmentation System: analysis of queries to prioritize improvements and significantly enhance relevance.
• Multimodal Architecture: configuration of individual indexes for documents, images, and tables.
• Query Routing: a unified system that automatically selects the optimal search mechanism for each query.

Course Program

1. Evaluation Systems - synthetic datasets and metrics for objective evaluation.
2. Fine‑tune Embeddings - customization of embeddings for business context.
3. Feedback Systems - collection and analysis of signals from users.
4. Query Segmentation - identifying query segments with the greatest potential for improvement.
5. Specialized Search - building specialized indexes for different types of content.
6. Query Routing - automatic routing of queries between search mechanisms.

Course Instructor

Jason Liu - an expert in developing AI systems across various domains: from computer vision at the University of Waterloo to content policy at Facebook and recommendation systems at Stitch Fix, where his solutions drove a revenue increase of $50 million. His experience in large-scale data organization, designing multimodal retrieval models, and processing hundreds of millions of recommendations per week forms the foundation of his consulting projects and this program for implementing RAG systems.