Non-Functional Requirements in Data Warehousing
The enterprise data warehouse is more than just reports and dashboards. There are actually more technical components, also know as non-functional requirements, to be built.
In this article:
Why Non-Functional Requirements are important
In our trainings and consulting practice, we often pitch the idea of “focusing on the business value” to the audience. Business value in enterprise data warehousing is defined as “something of value to the business” (believe it or not, we believe it should be said sometimes).
Typically, the reason businesses setup a budget for the enterprise data warehouse is that they want some reports or dashboards with actionable information. On the other hand, the most important counter-argument is that the enterprise data warehouse is more than just reports and dashboards. There is actually a lot of more technical components (non-functional requirements) to be built, including but not limited to:
- Infrastructure and storage
- Networking setup and security
- Multiple environments
- Performance tuning
- Maintaining security
- Privacy obligations require the deletion of consumer records
- Documentation
- Automated Continuous deployment
- Testing strategy is required
- Auditability and data lineage
- Operational aspects
- Data quality and data cleansing (good, bad and ugly data)
- Data and information governance
- Metadata management
- Data warehouse automation
This is all right, and we fully agree with that. However, all these non-functional requirements are of secondary priority for the business user. They (typically) don’t care that the enterprise data warehouse is developed with a fancy new technology or that the deployment is performed in a fully automated fashion.
Who cares? I do, because I’m a tech person. But should business actually care? The Short answer is “well, maybe”.
No, because it is not their primary intent why they came up with the idea to introduce an enterprise data warehouse in the first place (I bet because of the need for reporting and dashboarding).
Yes, because they are interested in a solid solution, documented, tested, stable, etc. So having implemented all these non-functional requirements has priority, but sorry, just secondary priority.
And nobody expects that these non-functional requirements are implemented in the first sprint – instead we typically have more time to implement them. And yes, everybody expects the availability of the functionality eventually. But again, not after the first sprint.
But the regular delivery of business value, again, reports or dashboards, demonstrates project progress to the business user and makes it more transparent what (and how much) they pay for. That’s why we have this focus on delivering actual business value in our projects but without losing our view on the non-functional requirements, too. But just as a second priority.
Conclusion
In conclusion, while delivering immediate business value through reports and dashboards is essential in enterprise data warehousing, it’s equally important to address non-functional requirements such as infrastructure, security, performance tuning, and data governance. These elements, though often secondary in priority, are crucial for building a robust, scalable, and maintainable data warehouse that meets long-term business objectives. Balancing the development of functional outputs with the implementation of these technical components ensures a comprehensive solution that aligns with both immediate and future business needs.
– Michael Olschimke (Scalefree)
