Unified.to
Blog

Understanding native and virtual webhooks


March 8, 2023

What is a webhook?

A webhook is a mechanism that allows a server to send a real-time notification to another server when a specific event occurs. It is sometimes called a "web callback" or "push API". This can include events such as an update to a candidate in an ATS or new deal creation in a CRM.

Unlike REST APIs that are used to "poll" a server for data, a webhook is sent by the data server back to your application whenever new data is available.

Webhooks can make your synchronization strategy less complicated since your application will receive updated data directly.

Polling for updates (via APIs)

The traditional method of retrieving new data from an API (without a webhook) is by calling the List API with the updated_gte filter, you can get data that has been updated since that date. The frequency that you poll would be determined by how important that data (and that customer) is to your application. You'll also need to handle rate-limiting and other network errors that you could encounter. If the data isn't time-sensitive, then this strategy might be sufficient. But you need to take care of retry strategies when the integration's API returns back a Rate-Limit error, which can make this method more complicated than using a webhook.

Pushed updates (via webhooks)

Modern APIs and applications leverage webhooks for synchronization because of its simplicity and timeliness. You, the application developer, do not have to create a retry mechanism to actively poll the server for new data and deal with rate-limiting and other network errors. The hard work is done by the third-party API provider to send you the update when their data changes.

Webhooks are much more efficient than polling by reducing any unnecessary API requests and eliminating the need for your app to retain a polling state. According to a survey conducted by Zapier, "82% responded that they preferred webhooks over polling."

How do webhooks work?

You first subscribe to receiving the data that you want to monitor for changes, by registering a webhook with the URL of your server's API endpoint.

Your server then listens to that URL and once it gets a message, it parses it to figure out which data has changed.

What is a virtual webhook?

Webhooks sound great, right? However, there's an important caveat we have to mention.

Not all software vendors support webhooks. So instead of having to support a mix of webhooks and polled API requests, Unified.to has standarized this for you through virtual webhooks, where our system mimics real webhooks for all vendor integrations. That means that you, the application developer, do not have to handle both polled and pushed data updates.

Our proprietary technology will monitor your customers' connections for update to data objects and then notify your webhook URL of those changes. The actual data will be sent to your webhook URL so that you then do not have to call another API request to retrieve it.

You subscribe to and manage virtual webhooks exactly the same way as you would with real webhooks. There is no difference with how to handle webhooks on the Unified.to platform, regardless if the integration support webhooks natively or not. The Unified.to platform will manage all of the polling, rate-limiting, and state management so you don't have to. Of course, because of our strict security policies, we are still not storing any of your customers' data on our servers or data bases.

How to subscribe to a webhook

Once a customer authorizes an integration for your application with the Unified.to authorization system, you will get a connection_id that represents that new connection.

You can then call the CreateWebhook API

POST /unified/webhook?include_all=true

{
    hook_url: `${my_webhook_url}`,
    connection_id: `${connection_id}`
    object_type: 'candidate',
    event: 'updated',
    interval: 60,
  fields: ''
}

That API request will return a webhook object with an id which you can then use to delete the webhook when it is no longer needed. If you delete the connection, any associated webhooks will also be deleted.

You can also see active webhooks at https://app.unified.to/webhooks or via the ListWebhooks API .

Synchronization strategy

When you first integrate with a customer's third-party account, you'll want to use Unified.to unified API to retrieve the relevant information. The first step is to initially retrieve all of the existing data and then the next step is to retrieve the updated data when it is changed.

  1. Initial retrieval When you create a webhook with the include_all parameter, Unified.to will retrieve all of the existing requested data. Our system will send your webhook that data in chunks of 100 entries until complete. Depending on the amount of data, the integration's API's rate-limiting rules, and your webhook server's performance, this could take a very long time. If you do not want to get all of the existing data (for example, you already have it, or don't need it), then do not send in the include_all parameter.
  2. Updated data retrieval Anytime that updated data is available, your webhook will get called with a list of up to 100 entries of that requested object. If there are more than 100, then your webhook will be called multiple times.
  3. Triggering a retrieval If you don't want to wait the whole hour, while testing webhooks on a Tester plan, you can use a manual trigger to re-schedule webhook to run right away. Simply click the "Trigger" icon next to your webhook. This can also be accomplished in the API.

Handling Errors

  • If your webhook in unavailable or returns an error when Unified.to sends data to it, we will try up to 3 times, waiting 1 second in between. If the error persists, we will back-off and retry based off of a Fibonacci delay sequence, starting with 1 minute.
  • If the integration's API is unavailable or returns an error (such as a Rate-Limiting warning), we will back-off and retry based off of a Fibonacci delay sequence, starting with 1 minute.
  • The Connection Logs will show all webhook attempts and errors.

Additional Webhook Information

  • health status indicator with possible values of Healthy and Unhealthy visible on the webhook page.
  • Light audit trail that is limited to the last 100 activities
  • Last-time that the webhook was run/checked

Blog