Partha Sarathi Roy

Partha Sarathi Roy

Decoding the generation component: How RAG creates coherent text

Decoding the generation component: How RAG creates coherent text

Introduction In part one of the "Understanding RAG series," we outlined the three main components of RAG: indexing, retrieval, and generation. In the previous parts, we discussed techniques for improving indexing and retrieval. This blog will focus on methods to enhance the generation component of Retrieval-Augmented Generation (RAG)

Improving RAG accuracy with reranking techniques

Improving RAG accuracy with reranking techniques

Introduction Creating a robust Retrieval Augmented Generation (RAG) application presents numerous challenges. As the complexity of the documents increases, we often encounter a significant decrease in the accuracy of the generated answers. This issue can stem from various factors, such as chunk length, metadata quality, clarity of the document, or

The role of retrieval in improving RAG performance

The role of retrieval in improving RAG performance

Introduction In part one of the "Understanding RAG" series, we covered the basics and advanced concepts of Retrieval-Augmented Generation (RAG). This part delves deeper into the retrieval component and explores enhancement strategies. In this blog, we will explore techniques to improve retrieval and make pre-retrieval processes, such as

What is RAG? A comprehensive guide to retrieval-augmented generation in AI

What is RAG? A comprehensive guide to retrieval-augmented generation in AI

What is a RAG? Retrieval-augmented generation (RAG) is a process designed to enhance the output of a large language model (LLM) by incorporating information from an external, authoritative knowledge base. This approach ensures that the responses generated by the LLM are not solely dependent on the model's training

KNOWHALU: Hallucination detection via multi-form knowledge-based factual checking

KNOWHALU: Hallucination detection via multi-form knowledge-based factual checking

What is "hallucinations" in LLM? Significant advancements in Natural Language Processing (NLP) have been achieved with the advent of Large Language Models (LLMs). These models excel in generating coherent and contextually relevant text. However, they are prone to "hallucinations." In the context of LLMs, hallucinations refer

LongRAG

LongRAG

What is LongRAG? What's the buzz around this? In this blog post, we are taking references from the LongRAG research paper to provide a detailed explanation of LongRAG. LongRAG (Long Retrieval-Augmented Generation) is a framework that uses long-context language models to enhance traditional Retrieval-Augmented Generation (RAG) methods. In

Using a Jury of LLMs Instead of a Single Judge to evaluate LLM generations

Using a Jury of LLMs Instead of a Single Judge to evaluate LLM generations

As LLMs advance, evaluating their quality is increasingly complex. Using a single large model like GPT-4 as a judge is costly and biased. A research paper by the Cohere team suggests using a Panel of LLM evaluators (PoLL) with smaller models for more accurate, unbiased, and cost-effective assessments. What’s