Sync Observability Settings

This guide explains the different observability settings that let you specify:

• How RudderStack retains the sync log and snapshot tables for your Reverse ETL connection.
• Whether RudderStack should continually retry sending the failed records in the future syncs.

Sync settings

You can access the following sync settings in the Settings tab of your Reverse ETL connection:

Storing sync logs and snapshot tables may incur additional warehouse costs.

Sync logs

RudderStack supports storing Reverse ETL sync logs in your warehouse. These logs are stored in a certain format in the RudderStack schema (used for storing the state of each sync). You can explore these logs using SQL or any BI tool to debug any issues or failures during syncs.

Storing the sync logs in your warehouse will incur additional costs.

Sync log table schema

The sync log table schema is as follows:

Snapshot table schema

RudderStack generates a snapshot table in the RudderStack schema for each sync. It contains rows (barring invalid rows like duplicate primary keys and null primary keys) from the data source like a warehouse table, view, SQL model, or audience. It also includes the details of the changes that happened in each row during the sync.

Storing the snapshot tables in your warehouse will incur additional costs.

The snapshot tables are stored in the snapshot_<connection_id>_<sync_run_id> format, where:

• <connection_id> is the Reverse ETL connection ID.
• <sync_run_id> is the unique identifier for the sync run.

The snapshot table schema is as follows:

Debug issues with sync logs

This section lists some queries you can run on your sync logs for debugging issues in your Reverse ETL syncs.

• Check how a particular primary key has changed over time for a Reverse ETL connection:

select * from _rudderstack.sync_log where primary_key='<primary_key>' and connection_id='<connection_id>';

• Check the data for failures in the sync run:

SELECT *
FROM   _rudderstack.sync_log sl
       LEFT JOIN
       _rudderstack.snapshot_<connection_id>_<sync_run_id>
       sn
              ON sl.primary_key = sn.<primary_key>
WHERE  sl.sync_run_id = '<sync_run_id>'
       AND sl.status = 'failed'; 

Make sure to replace the placeholders in the above queries with the actual values.

Retry failed records

The Retry failed records toggle in the sync settings lets you determine whether RudderStack should continually retry sending the failed records in the previous syncs to the downstream destination.

If a failed record has undergone any changes, RudderStack retries syncing the updated record instead.

Questions? Contact us by email or on Slack