Data Warehousing is Not a Strategy: How to Maximize ROI from Your Warehouse Investment
Modern data warehouses are incredibly powerful. Cloud warehouses deliver on-demand scalability, extreme flexibility, and they underpin incredible business results. According to Gartner/Tellius data driven companies have achieved as much as 19x profitability, are 23x more likely to acquire new customers, and can get a 20-30% EBITDA boost. Most companies, though, struggle to generate these kinds of returns on their warehouse investments.
One reason for this is that it’s easy to focus on perfecting data storage and lose sight of the big picture. Merely storing data in a warehouse can take a lot of work and feel valuable, but it doesn't automatically create any business advantage – it’s not a strategy. To create value, it’s crucial to ensure that data and accompanying insights from the warehouse are accessible to business users who use them for strategic action.
Establishing collaboration between your data team and business stakeholders, however, is challenging. It’s tempting for data teams to stick to their roles and responsibilities and consider their jobs complete once the data pipelines are built. But if you want to position your team as a driver and value creator you must bridge this gap.
With a strategic approach, you can transform the perception of your data team from cost center to business engine. The key lies in bringing the insights derived from the data together with the decision-makers themselves to translate them into game-changing actions.
So, how do you empower business users to utilize warehouse data for decision-making? What processes should be implemented to foster a sense of partnership rather than departmental silos? I’ll answer these questions in this post.
Data warehousing: A brief history
Data warehouses first emerged in the 1980s as centralized repositories for structured data from transactional systems and other sources. Typically built using SQL databases, they could store current and historical data in one place for business reporting and analytics. These early warehouses focused on business intelligence and predefined reports.
Fast forward to today, and we have modern warehouses that combine vast cheap storage with separated scaleable cloud computing. This combination enables more advanced analytics and data science (like running Python notebooks on Databricks or Snowflake). With these added capacities and capabilities come greater expectations and costs. Companies now expect their data to provide more business impact from things like predictive models, accurate forecasting, and game-changing insights. The reality, though, is that many companies simply have a warehouse full of data without the business impact.
Data warehousing is not the final product
Economic pressures of recent years have turned up the heat on data teams. Once concerned with data warehousing as a requirement, company leadership is now more aware of data infrastructure costs than ever, and they’re eager for results. For data leaders, this new pressure can be unsettling and even lead to concerns about job security. The good news is, the data in your warehouse is arguably the most valuable and hardest-to-copy asset your company has. When you unlock it for the business, results will follow. It won’t come all at once, but orchestrate a series of small wins, and the value your team provides will become apparent.
The question is: How do you go beyond fulfilling the requirement of having a data warehouse and reach a place where your team, and the entire company, is utilizing data from the warehouse to create maximum business value?
The answer lies in creating interfaces between business and technical teams and aligning tooling and processes to create valuable datasets, analytics models, and semantic layers rather than data vanity projects that don’t get used or create value.
You’ll have to shift from working in silos and viewing data warehousing as the end to working cross-functionally and viewing data warehousing as a means to competitive business advantage. Do the work to make data warehousing an integral part of a broader data strategy. Yes, your warehouse needs to be built with architecture and modeling best practices, but more importantly, it’s got to fulfill specific business needs.
Similar to development teams, your data team should build MVDPs (Minimum Viable Data Products), create quick feedback loops, and iterate. Interstingly, Microsoft Excel is a great tool for this process because it can act as a universal language between data teams and business users – since they are often both familiar with it. Once you set requirements with Excel, data engineering and warehousing can happen much more effectively.
The role of the analytics engineer
To help close the gap between data and business teams, a new role emerged in the early 2010s: The analytics engineer. This hybrid role brings together the builder skills of a data engineer (naturally closer to the infrastructure) and the analytics skills of a data analyst (naturally closer to the business). We’ve previously written about the difference between data engineers and analytics engineers if you’d like to do a deeper dive. For this post, I’ll focus on how analytics engineering can help you bridge the gap between your data team and the business.
Companies like Netflix and Spotify were among the first to recognize the importance of this role in making data-driven decisions. They understood that it required a combination of engineering skills and a deep understanding of analytics. The advent of data stacks and tools like dbt (data build tool) further influenced this role by enabling advanced data modeling and transformation.
Analytics engineers are exposed to, or actively play a role in each stage of the data lifecycle:
- Data collection: Building and maintaining data pipelines to ingest data from various sources into a centralized warehouse.
- Data unification: Processing data so that it can be effectively used to bring together data from various sources, cleaning it up, and transforming it into various other formats.
- Data activation: Translate the objectives of the company into models, semantics layers, visualizations, and KPIs/metrics that transform data into valuable insights.
Because of their understanding of and involvement on both the data side and the business side, analytics engineers can translate between teams and provide much-needed alignment.
While this sounds like a solution to the problem, I’ve noticed that many analytics engineers are more engineer than analyst. Perhaps because the role came out of tech companies that highly value engineering skills. Whatever the cause – the important thing to note is that the key to success as an analytics engineer is a deep understanding of the business problems.
With this understanding and a broad knowledge of data work, an analytics engineer can help you educate business teams on possible use cases and drive data work toward higher business value projects. However, you won’t be able to deliver without the right processes and technology in place.
Practical solutions for processes and technology
When surveyed, 72% of companies said they had not yet forged an effective data culture. To improve your data culture, there are several practical steps you can take to go from just storing data to unlocking valuable business insights from your datasets.
- Business-driven data unification: Prioritize creating curated and unified views of the data by dumping various datasets into warehouses for a purpose (not just because it's possible). By consolidating data sources with the contextual understanding (for example a unified customer profile), you can achieve better results through more complete, accurate, and useful data.
- Internal team alignment: fostering collaboration between business and technical teams is essential for breaking down barriers. A practical step here is to include all the relevant business stakeholders and data team members in planning discussions, conversations about metrics, and discussions about business objectives. You’ll likely have to assert yourself to make this happen, but it will go a long way in helping you promote functional alignment.
- Analytics engineering: Introducing roles in engineering that combine technical skills like SQL and statistics with business knowledge and effective communication can be highly impactful in driving the business forward with data-driven decision making. These roles will be responsible for preprocessing stored data and generating tailored insights.
- Feedback loops: Lastly, to drive progress, you must push to establish feedback loops where data-driven insights enhance business strategy in a tight cycle. The OODA (Observe, Orient, Decide, Act) loop is one such loop. Feedback loops will help teams shift from reporting to optimization, and they’ll allow your team to show off their full capabilities – once initiated, there are often lots of “Oh, I didn’t know we could do that!” statements from business users.
Through the alignment of people, processes, technologies, and more people with hybrid skill sets, You can drastically increase the ROI on your data warehouse and turn it into a powerful engine for competitive advantage.
Conclusion: emphasizing the holistic data lifecycle
Data warehousing plays a role as a capability, but it should not be viewed as a strategy. To unlock value, you must integrate warehousing into a comprehensive data strategy that encompasses every stage of the data activation lifecycle, from initial collection to proactive activation.
As you build your data team, prioritizing roles that combine data skills with business acumen is essential. This means seeking out individuals who can analyze data, apply methods, and possess an understanding of the industry. When you blend these skills, build collaborative processes, and leverage the right technology, you turn your data into more than just an asset at rest. It becomes an active driver of strategic goals.
When data drives tangible business impact, your team's value will become apparent, paving the way for continuous success on increasingly powerful initiatives.