RudderStack Transformations Overview

RudderStack Transformations allow you to code custom JavaScript and Python functions to implement specific use cases on your event data. Transformations run in real-time, after data collection and before delivery to a destination. You can use them on any data you send through RudderStack – via Event Stream, ETL, or Reverse ETL pipelines. Transformations enable a variety of use cases, such as:

• Filtering/sampling events
• Enriching events
• Cleaning/aggregating data
• Data masking or removing sensitive PII information
• Implementing external actions on events using an API

Transformation templates and libraries

Every transformation you write is reusable. When you write a transformation once for use in a specific destination, it’s made available for use across all of your other destinations. Transformations are also modular – you can save your transformation code in a Transformation Library to reuse in other transformations.

RudderStack also provides over 18 prebuilt transformation templates, so you don’t have to start from scratch for common use cases.

Transformations API

You can build and manage your transformations using your existing workflow via our Transformations API. The Transformations API enables you to programmatically create, read, update, and delete transformations and libraries by making HTTP calls. It also gives you the flexibility to reuse and maintain different versions of your transformation code.

With the Transformations API, you can:

• Programmatically add and remove transformations to your platform via HTTP API calls
• Create an organization-wide sandbox where your team can store transformations
• Define Libraries
• Version Control your transformations
• Publish multiple Transformations in a single operation
• Check for successful compilation and execution of new transformations

Transformations use cases

Because of the inherent flexibility of Transformations, you can use them to implement a wide range of use cases based on your business requirements. Write custom transformations to handle specific requirements, or use one of our transformation templates to handle common use cases. Use cases fall into four broad categories:

• Data cleaning
• Data privacy
• Data enrichment
• Data filtering

Below, we’ll detail some of the most common use cases for transformations and provide the corresponding templates.

Data cleaning

Transformations provide a powerful mechanism for cleaning your event data before it reaches its destination. You can use our prebuilt data cleaning transformations to:

• Remove all properties with Null values to reduce the number of unnecessary fields generated in downstream destinations.
• Rename event properties to conform with the naming convention expected by the destination.
• Remove Auth0 UserID prefixes

Data privacy

Transformations allow you to replace or mask any sensitive data in your payloads on a per-destination basis. You can use our prebuilt data privacy transformations to:

• Replace or mask sensitive personal data to reduce the risk of accidentally disclosing PII. You can specify the properties you want to mask in the code.
• Anonymize IP addresses to protect the privacy rights of your users and reduce the risk of accidentally disclosing their IP addresses.
• Hash PII using the SHA256 encryption to

Data enrichment

Transformations allow you to automatically enrich your events with other available data from internal and external APIs. You can use our prebuilt data enrichment transformations to:

• Enrich events with third-party data like location, employment, or social media details using the Clearbit API and the user's email address.
• Enrich events with geolocation data using an IP-to-geolocation API. This lets you easily query your events based on the geolocation data, for example, country or city.
• Enrich events with user agent information, including the user’s browser, engine, OS, device, and CPU-related info so that you can perform downstream actions, like filtering events from mobile devices, later.
• Programmatically extract URL query parameters to enable data to be processed for specific destinations.

Data filtering

Transformations make it easy to filter your event data in flight so that only relevant, necessary data reaches each destination. You can use our prebuilt data filtering transformations to:

• Allow only events containing properties included on an allowlist to reach downstream destinations. For example, you could allow only certain event names like Product Added or Order Completed.
• Block events containing properties included on a denylist. For example, you could block all Product Added and Order Completed events or certain event types (identify, track, etc.).
• Send only a representative sample of events, which can be anonymized, to a downstream destination so you can work with a small, manageable amount of data in the destination.

Check out our Transformation Templates documentation for a complete list of prebuilt templates. While these cover common use cases, the applications for Transformations are extensive. You can create transformations from scratch to solve for your unique business requirements.

Demo and additional resources

Embed video: https://youtu.be/C11Na9oIv-E

Visit the following links for more information on RudderStack Transformations:

• Transformations documentation
• Transformations API documentation
• Transformations FAQs
• Sample transformations Github repo