Introduction to RAG

Retrieval-Augmented Generation (RAG) is an innovative framework that combines information retrieval with language model capabilities to generate contextually accurate responses. Unlike traditional approaches that require fine-tuning language models, RAG relies on embedding documents into a vector database, which is then used to craft precise responses.

RAG operates in two main phases:

1.Indexing Phase: Documents are split into chunks, embedded using an embedding model, and stored in a vector database.

2.Retrieval Phase: Chains powered by frameworks such as Lang Chain associate this database with a language model, retrieving relevant chunks to craft the final response.

In this blog, we explore one advanced strategies in both the indexing and retrieval phases,

The strategies are semantic chunking in indexing phase and multi-query retrieval in retrieval phase:

Advanced Indexing Strategy: Semantic Chunking

Basic RAG implementations rely on fixed-size or paragraph-based chunking, which may not align with the semantic structure of the document. Advanced semantic chunking identifies natural topic boundaries within text, ensuring coherent chunks that improve retrieval accuracy.

Semantic Chunking Techniques

Semantic chunking employs methods like topic modeling and content analysis to determine optimal chunk boundaries. Tools like Lang Chain’s Semantic hunker enable this advanced approach. Below are four key semantic chunking strategies or we can say thresholds:

1.Percentile Chunking:

Description: Calculates embedding similarity scores between adjacent chunks and splits at scores below a defined percentile.

Method:

•  Determines a threshold using the X percentile (e.g., 90th).
•  Splits chunks with scores below this threshold.

Best For: General use cases balancing chunking and semantic coherence.

2.Standard Deviation Chunking

Description: Measures mean similarity scores and identifies significant deviations.

Method:

• Computes mean and standard deviation of scores.
• Sets a threshold based on deviations from the mean.
• Splits chunks with significant score differences.

Best For: Identifies chunks that significantly deviate from norm.

3.Interquartile Range (IQR) Chunking

Description: Evaluates similarity scores as outliers

Method:

• Calculates the IQR using the 25th (Q1) and 75th (Q3) percentiles.
• Splits chunks with scores outside this range

Best For: Handling outliers and focusing on central similarity scores.

4.Gradient-Based Chunking

Description: Analyzes the rate of change in similarity scores, splitting chunks at significant shifts.

Method:

• Computes the gradient (change in similarity scores).
• Splits chunks at significant changes indicated by a set threshold.

Best For: Detecting and chunking based on substantial content shifts.

By employing these advanced chunking techniques, indexing becomes more refined, creating a robust foundation for retrieval.

Advanced Retrieval Strategy: Multi-Query Retrieval

Multi-query retrieval represents a leap in retrieval-phase sophistication by automating prompt tuning and enhancing document retrieval.

How Multi-Query Retrieval Works

When a user inputs a query, the following steps unfold:

1.Query Expansion:

• The user’s input is passed to a language model prompt that generates multiple queries from diverse perspectives.

2.Parallel Search:

• These queries are executed independently, retrieving a set of relevant documents for each query.

3.Ranking and Selection:

• Retrieved documents are ranked using a scoring algorithm. The highest-scoring documents are selected as context for the final language model response.

Benefits of Multi-Query Retrieval

• Broader Context: By examining the query from multiple perspectives, this method ensures all relevant aspects of the user’s intent are captured.

• Enhanced Recall: It retrieves diverse documents, increasing the likelihood of surfacing the most pertinent information.

• Automation: Reduces the manual effort of prompt engineering by leveraging the LLM to dynamically generate optimal queries.

Conclusion

Advanced RAG strategies, such as semantic chunking and multi-query retrieval, significantly enhance the performance and accuracy of retrieval-augmented generation systems. Semantic chunking optimizes document segmentation, while multi-query retrieval ensures comprehensive context by leveraging diverse perspectives.

By incorporating these techniques, businesses and developers can build robust RAG-based systems that meet complex user needs with precision and efficiency.