Braze Deduplication

Braze charges its consumption based on data points it ingests for every user property and events like session starts and ends, purchases, etc. These data points are accumulated whenever you update a user profile or when the users perform specific actions.

Braze counts all data points irrespective of whether you send any duplicates. It considers every property (except the standard ones) as a data point and counts it against your billing.

With the deduplication feature, RudderStack prevents duplicate user traits in the identify and track calls from being sent to Braze, thus saving data points and ultimately avoiding billing overages in Braze.

This feature is particularly useful in Reverse ETL scenarios where a large number of rows are sent with duplicate columns.

Get started

To use the Braze deduplication feature, follow these steps:

1. Log in to your Braze dashboard and go to Settings > Developer Console.
2. Generate a new REST API key with the users.export.ids permission. You can find it in the User Data section of permissions.

You also need to include the permissions listed in the Braze destination’s connection settings.

3. Use this REST API key to set up your Braze destination.
4. Toggle on the Deduplicate Traits (Beta) option in the Braze destination configuration settings:

5. Enable the connection and start sending your events to Braze.

To verify if you are receiving deduplicated traits in Braze, see FAQ.

You can use the deduplication feature when sending data to Braze from your Event Stream sources via both cloud mode and device mode.

It also works for Reverse ETL sources.

How deduplication works

Device mode

RudderStack stores the previous identify object and uses it for comparison - this check occurs directly in the Braze SDK.

Cloud mode

RudderStack uses the /users/export/ids endpoint (leveraging the users.export.ids scope in the REST API key used for setting up the Braze destination) to export data from a user profile by specifying a user identifier. It then compares the user profile against the incoming user traits to determine any deltas.

If a delta is found, RudderStack sends only the updated user traits to Braze and drops the others.

RudderStack implements the deduplication logic such that it will not surpass Braze’s API limits in most cases. However, in scenarios where this limit is surpassed, RudderStack defaults to sending all events without deduplication until the volume returns below the permissible rate limits.

The following example highlights how RudderStack sends deduplicated data to Braze:

user_attributes: {
  name: "Alex Keener",
  age: 30,
  location: {
    city: "New York",
    area: "Manhattan",
    country: "US"
  },
  interests: ["hiking", "soccer"]
}

user_attributes: {
  name: "Alex Keener",
  age: 30,
  location: {
    city: "San Francisco", // delta
    area: "California", // delta
    country: "US" // delta
  },
  interests: ["hiking", "soccer", "cycling"] // delta
}

user_attributes: {
  location: {
    city: "San Francisco",
    area: "California",
    country: "US"
  },
  interests: ["hiking", "soccer", "cycling"]
}

Note that RudderStack deduplicates only the billable Braze attributes. It does not deduplicate the following non-billable attributes:

See the Braze documentation for more information on the billable and non-billable data points.

FAQ

Which RudderStack Cloud plans support the Braze deduplication feature?

The Braze deduplication feature is available in RudderStack Cloud Growth and Enterprise plans.

How do I verify if deduplicated traits are being sent to Braze?

1. Set up your Braze destination in RudderStack by following these steps.
2. Create a test identify or track payload with a new test userId in Postman that includes several user traits.

You can use the RudderStack HTTP API to send the payloads.

1. Import the Postman collection using this URL.
2. Edit the variable source_write_key with the write key of the source connected to your Braze destination.
3. Edit the variable data_plane_url with your data plane URL.
4. Use Basic Authentication for authenticating all HTTP requests.

See the RudderStack HTTP API documentation for more details.

3. If you don’t have a test payload, you can use the following snippet:

{
  "type": "identify",
  "sentAt": "2023-04-15T12:07:41.050Z",
  "traits": {
    "first_name": "John",
    "last_name": "Doe",
    "email": "johndoe@example.com",
    "attribute_a": "value_a",
    "attribute_b": "value_b",
    "myObj": {
      "test": "value"
    },
    "myArr": [
      "value1",
      "value3"
    ]
  },
  "userId": "john@doe",
  "context": {
    "library": {
      "name": "rudder-analytics-php",
      "version": "1.0.1",
      "consumer": "LibCurl"
    }
  },
  "rudderId": "86a09397-07e3-4cc4-9bd3-1fa262a5b521",
  "messageId": "fb203095-df9d-40bd-bcf3-97eb8084fea4",
  "timestamp": "2023-04-15T13:07:39.000+01:00",
  "receivedAt": "2023-04-15T12:07:40.222Z",
  "request_ip": "212.129.29.104",
  "originalTimestamp": "2023-04-15T12:07:41.050Z"
}

4. Go to the Braze destination in your RudderStack dashboard and click the Live Events button. Also, log in to your Braze dashboard.

5. Send the test payloads - you should be able to see them in your Live Events viewer. Also, confirm that the attributes show up as expected in your Braze dashboard by going to the User Search tab.
6. Add or update a few attributes while retaining the rest of the event payload. If you used the event payload in Step 3, you can use the following modified payload to test the impact of the attribute changes:

{
"type": "identify",
"sentAt": "2023-04-15T12:07:41.050Z",
"traits": {
  "first_name": "John",
  "last_name": "Doe",
  "email": "johndoe@example.com",
  "attribute_a": "value_a.1",
  "attribute_b": "value_b",
  "myObj": {
    "test": "value_changed"
  },
  "myArr": [
    "value1",
    "value3"
  ]
},
"userId": "john@doe",
"context": {
  "library": {
    "name": "rudder-analytics-php",
    "version": "1.0.1",
    "consumer": "LibCurl"
  }
},
"rudderId": "86a09397-07e3-4cc4-9bd3-1fa262a5b521",
"messageId": "fb203095-df9d-40bd-bcf3-97eb8084fea4",
"timestamp": "2023-04-15T13:07:39.000+01:00",
"receivedAt": "2023-04-15T12:07:40.222Z",
"request_ip": "212.129.29.104",
"originalTimestamp": "2023-04-15T12:07:41.050Z"
}

7. Resend your event payload. You should see the new event with only the added/updated attributes. You can also confirm in the Braze dashboard that the attributes were added/updated (by going to the User Search tab for the same test user ID).

How do I observe the difference in data point billing due to deduplication?

In your Braze dashboard, go to Subscriptions and usage and check the metrics by the appropriate date filters (before and after enabling deduplication) to observe the data point differences over that period.

Does RudderStack support deduplication in case of nested arrays?

RudderStack supports the deduplication of both simple and nested arrays stored as properties.

However, if there are any changes within the nested array or object, RudderStack considers the entire array or object as changed before sending it to Braze.

For example, if you send the below data that is already sent previously to Braze, RudderStack deduplicates it and prevents additional data point consumption.

{
  "attributes": [{
    "external_id": "userId",
    "pets": [{
        "id": 1,
        "type": "dog",
        "breed": "beagle",
        "name": "Tommy"
      },
      {
        "id": 2,
        "type": "cat",
        "breed": "calico",
        "name": "Garfield"
      }
    ]
  }]
}

However, if you make some changes to the above data, as shown:

{
  "attributes": [{
    "external_id": "userId",
    "pets": [{
        "id": 1,
        "type": "cat",
        "breed": "street",
        "name": "Toby"
      },
      {
        "id": 2,
        "type": "cat",
        "breed": "calico",
        "name": "Garfield"
      }
    ]
  }]
}

In that case, RudderStack identifies the entire object as new data and sends it to Braze without any deduplication.

Questions? Contact us by email or on Slack