Real-Time Data Pipelines vs ETL: What Modern SaaS Systems Actually Need

ETL moves data into a warehouse in scheduled batches for analytics and reporting; a real-time data pipeline delivers each change as an event, or reads the source system at the moment your application asks. For SaaS products whose features act on customer data in other apps, such as a CRM, HRIS or accounting system, the second pattern fits, and Unified.to provides both halves of it: reads at request time across 1594 integrations, and change events through native and [virtual webhooks](/blog/what_are_virtual_webhooks_how_they_differ_from_sync_based_webhooks) on supported integrations, with no customer records at rest.

Key takeaways

• ETL copies data into a warehouse on a schedule, so the warehouse is as current as the last load.
• CDC and streaming ingestion move changes more often, but they still copy data into a stream, table or warehouse that consumers read.
• Event-driven pipelines deliver a change when it happens, through webhooks; request-time reads return what the source holds when you call it.
• ETL stays the right choice for analytics, historical reporting and large queries across your own systems.
• Product features and AI agents that act on a record need the record as it is in the source, not as it was at the last sync.

How do ETL and real-time data pipelines compare?

ETL and real-time data pipelines differ in how data moves, how current it is when something reads it, and where it rests.

What is ETL?

ETL (extract, transform, load) is, in Microsoft's definition, "a data integration process that consolidates data from diverse sources into a unified data store". Microsoft's Azure Architecture Center describes ETL and ELT as operating "on datasets in scheduled batches" (page updated 30 October 2025), so the warehouse is as current as the last load.

For how ETL compares with iPaaS and unified APIs as ways to build integrations, see ETL vs iPaaS vs Unified API.

What do ELT, CDC and streaming ingestion change?

CDC and streaming ingestion shorten the gap between a change and the copy, and ELT moves where the transformation happens; all three still load data into a stream, table or warehouse that consumers read.

• ELT moves the transformation step: "In the ELT pipeline, the transformation occurs in the target data store" (Microsoft). The data is loaded first and reshaped in the warehouse.
• Change data capture (CDC) reads changes from the database's own log instead of re-extracting tables. Debezium "records all row-level changes within each database table in a change event stream" (as of 19 October 2026). Airbyte's CDC reads the log "from the previously synced position in the logs up to the start time of the sync", so each sync picks up where the last one stopped (as of 19 October 2026).
• Streaming ingestion loads data as it arrives rather than on a schedule: Snowpipe "continuously loads data from files into standard tables and Apache Iceberg™ tables as the files arrive in a stage" (as of 19 October 2026).

CDC depends on access to a database's change log, as Airbyte's docs note: "Many common databases support writing all record changes to log files for the purpose of replication." That works for databases you can connect to directly. A customer's SaaS apps give you their APIs, and on some apps webhooks, rather than access to their database logs, so data from those apps reaches you through API calls and events.

What is a real-time data pipeline?

A real-time data pipeline delivers a change as an event when it happens, or reads the source system when your application asks, instead of reading a copy loaded on a schedule. It takes two forms, and a product can use both.

• Event-driven. The source system sends a webhook when a record changes, and your code reacts. Where a system sends no webhooks of its own, a service checks it on an interval and sends the event instead; Unified.to's virtual webhooks do this.
• Request-time reads. Each call goes to the source API and returns what it holds at that moment. A record your product just updated reads back with the new value, as the source returns it. The Pass-Through vs. Sync-Based Unified APIs guide covers this model in detail.

Neither form needs a stored copy, though an event-driven pipeline often writes the events it receives to your own database, which is a copy you choose to keep.

How current is the data in each approach?

How current the data is depends on how a change reaches you: on a schedule it waits for the next run, a native webhook arrives when the source sends it, and a request-time read returns what the source holds when you call it. For anything on a schedule, with changes arriving evenly, the average wait is about half the interval (our calculation).

When is ETL the right choice?

ETL is the right choice when the reader is an analyst or a report rather than a product feature: analytics across your company's systems, historical reporting, and large transformations or joins across datasets.

• Analytics and reporting. A warehouse holds history from many systems in one place for queries and dashboards.
• Historical data. Point-in-time reporting, long-term storage and datasets for training models need a copy that keeps history, which a source API may not.
• Cross-system transformations. Large joins and transformations across datasets run better in a warehouse than across live API calls, which are paginated and rate-limited by each source.

When does a product need an event-driven or request-time pipeline?

A product needs an event-driven or request-time pipeline when a feature shows or acts on a customer's record in another app, and a stale copy would show the wrong value or trigger the wrong action.

• In-app views of customer data. A user who updates a deal in their CRM expects your product to show the new stage.
• Automation on changes. Creating a task when a candidate applies, or starting onboarding when an employee is hired, depends on the event arriving.
• Writes back to the source. A write needs the current record; a read from a day-old copy can overwrite a newer change.
• AI agents that act. An agent that reads, decides and writes in one loop acts on the record as it was at the last sync if it reads a copy. Pipelines that summarize or index many records for retrieval can run on a copy, as long as webhooks keep it current.

How does Unified.to deliver changes and current reads?

Unified.to reads each source at request time and delivers changes as events, through one API per category across 36 categories, and stores no customer records at rest.

• Request-time reads and writes. Every call goes to the source API, and the response is mapped onto the category's model in memory. Connection credentials and operational metadata are stored, encrypted at rest; API call logs record endpoint, timing, status and workspace, never payloads (privacy policy).
• Native webhooks. Unified.to forwards events as the integration sends them.
• Virtual webhooks. Where an integration sends no webhooks, virtual webhooks detect changes and deliver events at an interval you set, as often as every minute on paid plans (60 minutes on the free trial). Events carry the changed records, so there's usually no follow-up fetch, and a check that finds nothing isn't billed.
• Coverage. Webhook support, native or virtual, varies by integration and object; check each integration's Feature Support tab.
• Database Sync, for when you do want a copy. Database Sync writes records into your own MongoDB, MySQL, Postgres, MSSQL, MariaDB, Supabase or Snowflake database, through the same webhooks, on Grow, Pro and Scale. It keeps current only the objects that emit change events, and include_all loads history first where the integration supports an initial sync. The copy sits in your infrastructure, which is how a warehouse can sit downstream of a request-time API.

Which should you use?

Choose by who reads the data: a report can read a copy, while a feature that acts on a customer's record should read the source or react to its events. A product can use both: a warehouse for reporting, and events and request-time reads for its features.

→ Book a demo

Written by Mallory Greene.

About the author: Mallory Greene is Head of Marketing at Unified.to. She writes about integration infrastructure, unified APIs, and MCP for technical teams. Based in Toronto.