ht
Documentation
WelcomeConcepts

Get Started

OverviewCreate a sourceCreate a modelCreate a destinationCreate a sync

Working with data

Working with syncsUsing the Live Debugger
Documentation/Working with data/Working with syncs
ht
Documentation

Working with syncs

Table of Contents
Overview

Overview

The Sync page provides an overview of your syncs along with their status, which model they use, the destination they write to, the schedule, last run, and when they were created.

Sync Status

Your sync's status is based on the latest information available on a run:
  • Healthy: a recent run completed
  • Pending: no runs have been scheduled yet
  • Disabled: the sync has been disabled through the UI
  • Querying: a sync is querying your model
  • Preparing: a sync is preparing your data for your destination
  • In Progress: a sync is writing to your destination
  • Incomplete: a sync had a temporary interruption, it will be resumed in a new run.
  • Aborted/Failed: there was an error in the sync progress, check the errors on the sync to see why the sync failed.

Run Details

When viewing a specific sync, you can view detailed run-level information. A run is a particular invocation of a sync, either triggered manually, via an API request, through an Airflow/dbt integration or via a schedule.Each run provides the number of records processed, added, changed, removed, and rejected. If there were any errors, click the View Error button to view details related to the run.For the added, changed, and removed columns the number of records successfully sent to the destination appears in green as the first column. The number of records attempted but did not successfully completed are in the second red column.For example, if a record was removed from the model and it was successfully removed from the destination, you will see a green 1 under removed. If the record was not removed due to an error, you will see a red 1 instead under the same column.

Configuration

The configuration section exposes destination-specific configuration options. Check out the Destinations page for the particular destination you're using for more details

Schedule

Resync Full Query

By default, Hightouch keeps track of changes to your data between syncs. Consequently, an initial run of a sync will show many records 'added' and subsequent runs will, in many cases, show fewer records 'added' and instead, show many records have 'changed'. If you wish to reset the sync's tracking to its initial state -- as if it were syncing for the first time again -- you can select 'Resync full query' from the menu on your sync's main page:

Alerting

You can set alerts to inform you of problems with your syncs. Hightouch supports sending alert notifications through SMS, Slack, email, and PagerDuty.Setting up alerting requires two steps:Step One:To begin, you first need to visit the Alerts section under your workspace's main settings to set up your workspace's default alerts. Click on 'Add alerts' and input the data required for your choice of alert: SMS, Slack, email, or PagerDuty. A 'fatal error' is an error communicating with either a source or destination. It is 'fatal' because the sync can't even run.A 'row error' means there's a problem with syncing a row (or record). The sync works, but a row has failed.Setting the 'Minimum Alert Interval' allows you to receive alerts at spaced intervals; once every hour, for instance.You can create multiple alerts for different types of severity. For instance, perhaps you use PagerDuty for fatal alerts and Slack for row alerts. Once you have default alerts set up in your workspace's main settings, you are ready for step two, which is to enable an alert (or alerts) in your sync.Step Two:Visit the 'Alerts' tab in your sync's settings area:The default alerts you set up in your workspace's settings will automatically be listed.From here, you can modify the default behavior of an alert by selecting or deselecting if it's active for a 'fatal error' or a 'row error' or both. You will also see a record of when the alert was last attempted.You can also set an alert threshold so that you're only notified when the sync errors are above a certain percentage of either total queried rows or total synced rows. This feature is useful when you expect that a certain number of rows/records will regularly fail and you don’t wish to be notified unless the number of errors is significant.Once you've configured your alert settings in the sync, don't forget to click 'save':