RudderStack requires some warehouse-specific permissions to sync data from it. These permissions are listed in the source documentation, for example, see BigQuery permissions.
Make sure to grant these permissions before you set up the source.
Connect destination
Once you have created a Reverse ETL source, connect it to an existing or new RudderStack destination.
RudderStack supports connecting multiple Reverse ETL sources to a destination.
Note that this feature is available only for the Facebook Custom Audience destination currently. Support for more destinations is coming soon.
To connect a destination:
Click Use existing destination or Set up a new destination depending on your requirement.
Other destinations that do not support Visual Data Mapper have the Map with JSON option by default, with the Map with Visualizer option greyed out.
The steps to set the data mappings (using JSON) are listed below:
Select the Sync mode to specify how RudderStack syncs each record to the destination. RudderStack supports two sync modes:
Sync mode
Description
Upsert
In this mode, RudderStack inserts new records and updates the existing records in the destination while syncing the data.
Mirror
In this mode, RudderStack ‘mirrors’ the source by keeping the destination data the same as the source data. It performs insertion, deletion, and updates to the records while syncing the data.
RudderStack does not support track event type for this mode.
Select the Event Type - RudderStack supports sending the synced data to your downstream destinations as identify or track events.
If you select track as the Event Type, you also need to set the event name that is sent to the downstream destination. This is because every track event requires a name. You can also set the column name as the event name by turning on the Lookup event name by column toggle and selecting the column from the dropdown:
Set at least one user identifier from user_id or anonymous_id in the Choose user identifier setting.
Preview the resulting data and make the necessary changes before proceeding.
Schedule syncs
You can configure the schedule settings and sync frequency at the connection level. This is because a source might be connected to multiple destinations and the sync interval and frequency might need to be configured differently for each connection.
RudderStack determines how and when to run a sync based on the sync schedule you set for your Reverse ETL connection.
Schedule type
Description
Basic
Run syncs at a given time interval and specified time (in UTC).
CRON
Run syncs based on a specified CRON expression (in UTC).
Manual
Run syncs manually.
Configure advanced settings
This option is available only if you have set up a new destination.
Once you create a connection successfully, you can configure the optional advanced settings to receive the data correctly in your destination.
Click Configure your destination.
Specify the advanced settings and click Save.
Activate connection
Activate the destination by turning on the toggle.
Turn on the connection to activate the connection.
You will be redirected to the connections page.
RudderStack will send data to your destination as per the specified sync frequency. Click Sync now to manually trigger a new sync.
Click Turn on the connection to activate the connection.
You will be redirected to the connections page.
RudderStack will send data to your destination as per the specified sync frequency. Click Sync now to manually trigger a new sync.
Stop data syncs for connection
Go to the connection page and turn off the connection toggle to stop data syncs for the connection.
Note that the Sync Now button is greyed out and the connection status is Paused until you turn on the connection again.
Update mapping configuration
Go to the Schema tab of your connection and click Update to change the mapping configuration and update your column selection. Then, click Save to update and save the configuration.
The Object (for VDM mapping), Sync mode, Event type, and User identifier fields are not editable.
Update connection settings
Go to the Settings tab of your connection to:
Get the Connection ID.
Change the sync schedule and frequency.
Specify the settings to retain sync logs in your warehouse and retry syncing failed records. See Syncs Observability for more information.
Delete the connection permanently.
View all connections for Reverse ETL source
Go to the Overview tab of your Reverse ETL source. Here, you will see all the destinations connected to that source. Click the required destination or click View > Connection details to go to the connection’s page.
RudderStack automatically redirects you to the Syncs tab where you can view the latest sync details or check past syncs.
View sync details
Go to the Syncs tab to see detailed metrics on the latest sync. You can also view details of the past syncs (up to 1 month). These details include:
Metric
Details
Status
The sync status. It can be one of the following:
Aborted
Completed, with failures
Completed, no failures
Sync mode
Defines how RudderStack syncs data to the destination. It can be one of the following:
Upsert
Mirror
Sync type
Determines the scope of the sync. It can be one of the following:
Incremental: RudderStack syncs only the newly added data in the warehouse since the last sync.
Full: RudderStack syncs all the data irrespective of whether it was synced to the destination previously.
Trigger
Determines how the sync was triggered. It can be one of the following:
RudderStack provides an intuitive visual graph detailing the records sent to the destination over time. The X-axis represents the time of the day and the Y-axis represents the number of records classified as:
Successfully synced records
Dropped records
Records that failed to sync
Switch to the Retries tab to get a graphical view of the retries over time. The X-axis represents the retry time of the day and the Y-axis represents the number of retry attempts.
This site uses cookies to improve your experience while you navigate through the website. Out of
these
cookies, the cookies that are categorized as necessary are stored on your browser as they are as
essential
for the working of basic functionalities of the website. We also use third-party cookies that
help
us
analyze and understand how you use this website. These cookies will be stored in your browser
only
with
your
consent. You also have the option to opt-out of these cookies. But opting out of some of these
cookies
may
have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This
category only includes cookies that ensures basic functionalities and security
features of the website. These cookies do not store any personal information.
This site uses cookies to improve your experience. If you want to
learn more about cookies and why we use them, visit our cookie
policy. We'll assume you're ok with this, but you can opt-out if you wish Cookie Settings.