Traditional RAG vs. HyDE, explained visually.

One critical problem with the traditional ​RAG system​ is that questions are not semantically similar to their answers.

Consider you want to find a sentence similar to "What is ML?".

It is likely that "What is AI?" is more similar to it than "Machine learning is fun."

Due to this semantic dissimilarity, several irrelevant contexts get retrieved during the retrieval step.

HyDE solves this.

The following visual depicts how it differs from traditional RAG.

Here's how it works:

• Use an LLM to generate a hypothetical answer H for the query Q (this answer does not have to be entirely correct).

• Embed the answer using a contriever model to get E ​(Bi-encoders​ trained using contrastive learning are famously used here).

• Use the embedding E to query the vector database and fetch relevant context (C).

• Pass the hypothetical answer H + retrieved-context C + query Q to the LLM to produce an answer.

Done!

Now, of course, the hypothetical generated will likely contain hallucinated details.

But this does not severely affect the performance due to the contriever model—one which embeds.

More specifically, this model is trained using contrastive learning and it also functions as a near-lossless compressor whose task is to filter out the hallucinated details of the fake document.

This produces a vector embedding that is expected to be more similar to the embeddings of actual documents than the question is to the real documents.

Several studies have shown that HyDE improves the retrieval performance compared to the traditional embedding model.

But this comes at the cost of increased latency and more LLM usage.

Author

AiUTOMATING PEOPLE, ABN ASIA was founded by people with deep roots in academia, with work experience in the US, Holland, Hungary, Japan, South Korea, Singapore, and Vietnam. ABN Asia is where academia and technology meet opportunity. With our cutting-edge solutions and competent software development services, we're helping businesses level up and take on the global scene. Our commitment: Faster. Better. More reliable. In most cases: Cheaper as well.

Feel free to reach out to us whenever you require IT services, digital consulting, off-the-shelf software solutions, or if you'd like to send us requests for proposals (RFPs). You can contact us at [email protected]. We're ready to assist you with all your technology needs.

© ABN ASIA