Index-Time RAG vs Real-Time RAG: Choosing the Right Retrieval Strategy

Index-time RAG prepares and embeds data before anyone asks a question, so retrieval is fast but each answer reflects the index as of its last update; real-time RAG reads the source systems when the question arrives, so answers reflect the records as they are now, at the cost of each source's response time and rate limits. For SaaS products built on customer data in CRMs, ticketing systems and file storage, a hybrid fits: index content that changes slowly, and read records that change often or carry per-user permissions at request time. Unified.to supplies both sides across 1651 integrations: change events to keep an index current, and reads at request time, with no customer records at rest.

Key takeaways

• Index-time RAG trades currency for speed: retrieval is a similarity search against an index built ahead of time.
• Real-time RAG trades speed for currency: each question triggers reads against the source APIs.
• Keeping an index current takes change detection, re-embedding and deletes; when those lag, answers go stale without any error.
• Permissions are the sharpest difference: an index needs its own copy of who can see what, while a request-time read gets the source's current access rules for the connected account.
• A hybrid system decides, per data type, which path it takes.

How do index-time and real-time RAG compare?

Index-time and real-time RAG differ in when context is prepared, what retrieval returns, and where a copy of the data lives.

What is index-time RAG?

Index-time RAG is a retrieval strategy that does most of its work before anyone asks: content from your sources is cleaned, chunked, tagged with metadata such as object type, owner and timestamps, embedded, and stored in a vector or hybrid index. At query time, the system embeds the question and runs a similarity search against that index.

It works well for large bodies of content that change slowly, such as documentation, policies and knowledge-base articles, because the work of preparing them is done once and every query reuses it.

The cost is keeping the index current. Tickets close, deals change stage, files are edited or deleted and permissions change, and each change has to reach the index through change detection, re-embedding and deletes. When that lags or fails, the system keeps answering from the old version without any error to show it. Keeping Your RAG Index in Sync with Live SaaS Data covers how to run that process.

What is real-time RAG?

Real-time RAG is a retrieval strategy that reads the source systems when the question arrives, applies the source's access rules for the connected account plus any per-user filtering your product adds, optionally embeds or reranks the results, and passes the current records to the model.

It fits questions about current state: "What's the status of this ticket?", "Which deals moved stages today?", "Which files can this user open?". The answer reflects the source at the moment of the read, and access follows the source's current rules.

The trade-offs come from the sources themselves. Each read takes as long as the source API takes to respond, paginated results and rate limits have to be handled, and a question that touches many records costs more calls. Real-time RAG suits targeted lookups better than searches across a whole corpus.

Which data should you index, and which should you read at request time?

Index data that is large and changes slowly, and read at request time data that changes often, carries per-user permissions, or will be acted on.

How do permissions and compliance differ?

An index holds its own copy of your customers' data and its own copy of who may see it, while a request-time read relies on the source's authorization and leaves the data in the source.

With an index, every chunk needs the access rules of the record it came from, stored as metadata and checked at query time, and those rules have to change when the source's permissions change. The index is also another store of customer data for retention, deletion and security reviews. Permissions, Security, and Compliance in RAG Pipelines covers filtering and propagation in detail.

With a request-time read, the source checks access on each call for the account the customer connected, your product applies any per-user rules on top, and nothing is added to your stores unless you choose to keep it.

How does Unified.to support both strategies?

Unified.to supplies the two inputs a hybrid RAG system needs from customer SaaS data: change events to keep an index current, and reads at request time, through one API per category across 36 categories.

• Keeping an index current. Native webhooks are forwarded as the integration sends them; where an integration sends none, virtual webhooks detect changes and deliver events at an interval you set, as often as every minute on paid plans. Events carry the changed records, so your embedding service usually needs no follow-up fetch, and a check that finds nothing isn't billed. Point webhooks or Database Sync at the service that chunks and embeds your records. Support varies by integration and object; check each integration's Feature Support tab.
• Reading at request time. Each call goes to the source API on the customer's connection and is mapped onto the category's model, so the same code reads records across CRMs or ticketing systems; filtering support varies by integration. raw returns the source's original fields when the model doesn't cover one.
• No stored copy at Unified.to. Unified.to stores no customer records at rest; connection credentials and operational metadata are stored, encrypted at rest (privacy policy). The index you build sits in your infrastructure.
• Agents. Unified MCP runs on the same connections, so an agent can read current records and write back. permissions and tools limit what it can call, and hide_sensitive strips names, emails and phone numbers before results reach the model. For how MCP and RAG divide the work, see MCP vs RAG.

For a worked implementation, see How to Build a RAG Pipeline for Live SaaS Data; for the basics, What Is a RAG Pipeline?.

Which retrieval strategy should you choose?

Choose by how the data changes and how the answer will be used; a product that both searches history and answers questions about current records needs a hybrid, with a clear rule for each data type.

→ Book a demo

Written by Mallory Greene.

About the author: Mallory Greene is Head of Marketing at Unified.to. She writes about integration infrastructure, unified APIs, and MCP for technical teams. Based in Toronto.