• Inference Scaling for Long-Context RAG

  • Oct 20 2024
  • Length: 12 mins
  • Podcast

Inference Scaling for Long-Context RAG

Listen for free

View show details

  • Summary

  • 🗓 Inference Scaling for Long-Context Retrieval Augmented Generation

    This research paper explores the effectiveness of inference scaling for retrieval augmented generation (RAG), a technique that enhances large language models (LLMs) by incorporating external knowledge. The authors introduce two strategies, demonstration-based RAG (DRAG) and iterative demonstration-based RAG (IterDRAG), for effectively scaling inference computation. They demonstrate that increasing inference computation, when optimally allocated, leads to nearly linear gains in RAG performance. Furthermore, they develop a computation allocation model to predict the optimal test-time compute allocation for various tasks and scenarios, showcasing its effectiveness in achieving performance gains and aligning with experimental results.

    📎 Link to paper
    Show more Show less
Show more Show less
activate_Holiday_promo_in_buybox_DT_T2

What listeners say about Inference Scaling for Long-Context RAG

Average customer ratings

Reviews - Please select the tabs below to change the source of reviews.

Audible.com reviews

Amazon reviews
No Reviews are Available