Subsetting Business Keys: Modeling Shared Entities in Raw vs. Business Vault
Managing heterogeneous business objects stored within a single operational table presents a classic data modeling dilemma for enterprise data teams. Operational software vendors often build flexible schemas to accommodate broad usage patterns, placing diverse entities like customers, suppliers, contract holders, and injured parties into a single generic table such as a partner database. When designing an enterprise analytical platform, data architects must decide whether to split these entities during the initial load or maintain source alignment. Within the framework of Data Vault 2.0, the resolution lies in establishing a clear boundary between the Raw Data Vault and the Business Vault.
Understanding how to model shared business keys requires examining the foundational principles of Data Vault architecture. Operational systems frequently force distinct business concepts into shared structures due to software constraints or legacy design decisions. However, analytical environments demand a representation that reflects the actual semantic model of the enterprise. By applying soft rules in the Business Vault while keeping the Raw Data Vault strictly aligned with source systems, organizations can maintain auditability while delivering flexible, domain-specific data structures to business users.
To master these architectural patterns and ensure your enterprise warehouse scales effectively, explore our comprehensive Data Vault 2.1 Training & Certification programs designed for data engineers and solution architects.
In this article:
- The Challenge of Mixed Business Objects in Source Systems
- The Raw Data Vault Perspective: Absolute Source Fidelity
- The Business Vault Perspective: Applying Soft Rules for Semantic Separation
- Handling Satellites and Hash Keys in Subsetted Entities
- Architectural Boundaries for Upstream Filtering
- Watch the Video
The Challenge of Mixed Business Objects in Source Systems
Operational software packages—ranging from enterprise resource planning systems like SAP to custom legacy applications—frequently consolidate multiple real-world entities into a single relational table. A common example occurs in insurance core systems, where a single table records all individuals or organizations involved in a policy or claim. Within this single structure, the record could represent a policyholder, a beneficiary, an injured party, or a third-party driver. Similarly, commercial ERP systems often manage both customers and vendors inside a unified party or partner table.
This consolidation happens because software vendors prioritize operational efficiency, generic software design, and database normalization over business semantics. The operational application might use a unified primary key generator across all partner types, meaning that customers and suppliers share the same ID space and column attributes. While this architecture serves the immediate transactional workflow of the application, it creates significant friction when business users attempt to query, report on, or analyze specific business entities.
When enterprise data teams ingest these unified tables into an analytical environment, they encounter a fundamental semantic mismatch. Business stakeholders rarely ask questions about a generic party; instead, they query specific business concepts, such as customer lifetime value or vendor lead times. If data architects attempt to resolve this mismatch by filtering and splitting data prior to storage, they risk compromising the integrity and auditability of the underlying data platform.
The Raw Data Vault Perspective: Absolute Source Fidelity
The primary objective of the Raw Data Vault is to capture an unmodified, historical record of source system data. To preserve absolute auditability, data must be loaded into the Raw Data Vault without applying business filters, conditional transformations, or selective routing. The only condition permitted during a Raw Data Vault load is a delta check to determine whether a business key or satellite record has already been ingested.
When processing a source table that contains mixed business objects, the Raw Data Vault design must mirror the source structure. If the source system maintains a single partner table, the Raw Data Vault must implement a single partner Hub (h_partner) along with one or more corresponding partner Satellites (s_partner). Attempting to split the partner table into separate customer and supplier Hubs within the Raw Data Vault violates the core principles of Data Vault modeling.
+-----------------------------------------------------------------------------------+
| RAW DATA VAULT |
| |
| +--------------------+ +--------------------+ +---------------+ |
| | stg_partner | ====> | h_partner | ====> | s_partner | |
| | (Source Data Set) | | (Raw Partner Hub) | | (Raw Sat) | |
| +--------------------+ +--------------------+ +---------------+ |
+-----------------------------------------------------------------------------------+
Loading data as-is ensures that the data platform retains a complete and indisputable representation of what existed in the operational environment at any given point in time. If a source system contains misclassified records, overlapping key ranges, or changing attribute definitions, those realities are captured faithfully in the Raw Data Vault. Applying filter conditions or inner joins during the raw loading process obscures the lineage and makes it impossible to reconcile the data warehouse against source audit logs.
Furthermore, operational source systems often change unexpectedly or suffer from misuse over time. In enterprise settings, legacy systems frequently lack formal fields for new operational requirements, leading staff to repurpose existing text or phone fields for notes and alternative identifiers. If raw ingestion pipelines depend on specific attribute values to route data into distinct Hubs, minor operational changes in the source system can silently corrupt or break downstream data pipelines.
The Data Vault Handbook:
Core Concepts and Modern Applications
Build Your Path to a Scalable and Resilient Data Platform
The Data Vault Handbook is an accessible introduction to Data Vault. Designed for data practitioners, this guide provides a clear and cohesive overview of Data Vault principles.
The Business Vault Perspective: Applying Soft Rules for Semantic Separation
While the Raw Data Vault remains strictly source-aligned, the Business Vault provides the mechanism for restructuring data to match the enterprise semantic model. The Business Vault sits between the Raw Data Vault and the consumption layer (Information Marts), serving as the dedicated layer where soft business rules, complex transformations, and domain-specific logic are applied.
In scenarios where a single raw Hub contains multiple business concepts, architects can create virtualized Hubs in the Business Vault—such as bv_h_customer and bv_h_supplier—by applying filtering logic over the underlying Raw Data Vault constructs. These Business Vault Hubs isolate specific subsets of business keys using defined business rules without modifying or duplicating the underlying raw storage.
+-----------------------------------------------------------------------------------+
| BUSINESS VAULT |
| |
| +--------------------------+ |
| +---> | bv_h_customer | |
| | | (Filtered Business Hub) | |
| +-------------+ | +--------------------------+ |
| | h_partner | -+ |
| +-------------+ | +--------------------------+ |
| | s_partner | +---> | bv_h_supplier | |
| +-------------+ | (Filtered Business Hub) | |
| +--------------------------+ |
+-----------------------------------------------------------------------------------+
The logic required to distinguish between entities like customers and suppliers can range from basic attribute filtering to multi-table relational analysis:
-
Attribute-Based Logic: Filtering records based on categorical values stored in the raw Satellite, such as a partner type flag, status code, or geographic region (e.g.,
WHERE status IN (1, 2, 3) AND country = 'DE'). -
Behavior-Based Logic: Identifying entities based on their participation in transactional relationships. For instance, a partner may be classified as a customer only if their key exists in a non-historized transaction Link representing sales orders.
-
Composite Logic: Combining descriptive attributes, organizational boundaries, and historical activity to dynamically categorize entities as business rules evolve.
Because business definitions change far more frequently than source system structures, encapsulating these rules in the Business Vault offers significant architectural flexibility. If executive leadership redefines what constitutes an active customer, data teams can update the Business Vault view logic without reloading historical data or altering the physical Raw Data Vault schema. For deeper insights into designing scalable architecture and managing business rules effectively, refer to our guide on Enterprise Data Warehouse Solutions.
Handling Satellites and Hash Keys in Subsetted Entities
A common technical question when splitting a single Hub into multiple Business Vault Hubs is whether the associated Satellites must also be physically split or re-hashed. In most implementations, physically splitting Satellites or generating new hash keys is unnecessary and discouraged.
When a Business Vault Hub is instantiated as a virtualized view over the raw Hub, it preserves the original hash keys derived from the source business keys. Because the hash key remains unchanged, downstream queries can join the virtualized Business Vault Hub directly back to the original Raw Data Vault Satellite.
+-----------------------------------------------------------------------------------+
| VIRTUALIZED JOIN MECHANISM |
| |
| +--------------------+ INNER JOIN ON +----------------------------+ |
| | bv_h_customer | ======================> | s_partner | |
| | (Filtered Hash Keys| HASH_KEY | (Raw Satellite Deltas) | |
| +--------------------+ +----------------------------+ |
| |
| Result: Automatically retrieves ONLY satellite deltas matching customer keys. |
+-----------------------------------------------------------------------------------+
This join mechanism automatically filters the descriptive records. When joining bv_h_customer to s_partner on the shared hash key, the query naturally restricts the returned descriptive deltas to those belonging to customer keys. The remaining partner records belonging to suppliers or third parties are excluded by the join condition itself, eliminating the need to physically duplicate or split Satellite tables.
Retaining original hash keys yields several distinct operational benefits:
-
Storage Efficiency: Prevents duplicate physical storage of descriptive attributes across multiple derived Satellites.
-
Query Performance: Avoids dynamic hash re-computation inside views, allowing query engines to leverage existing hash indexes and distribution keys.
-
Simplified Maintenance: Keeps the lineage clear and direct between the raw ingestion layer and the virtualized business layer.
However, an exception to this rule occurs when business keys must be parsed or split into multi-part keys. For example, if a Vehicle Identification Number (VIN) contains embedded sub-keys representing the manufacturer, plant, and engine type, separating these components requires generating new hash keys. In such cases, materializing the resulting Business Vault Hub rather than calculating hashes on the fly inside a view ensures optimal query performance.
Architectural Boundaries for Upstream Filtering
When operational requirements dictate that certain records must be excluded from the analytical platform entirely—for example, due to legal restrictions, privacy regulations, or strict scoping—data architects must determine where that filtering should occur.
A fundamental rule of Data Vault design is that filtering logic must never be embedded within the staging or raw ingestion pipelines of the data warehouse. If data enters the staging area, it must flow into the Raw Data Vault to maintain complete auditability. If an organization requires that certain records (such as non-supplier partners) never be stored within the data warehouse, the filter must be implemented at the source export layer before the data is delivered to the ingestion boundary.
+-----------------------------------------------------------------------------------+
| FILTER LOCATION ARCHITECTURE |
| |
| +------------------+ Source Export +-----------------+ |
| | Operational Source| Filter (If needed) | Staging / Raw | |
| | System Database | =================> | Data Vault | |
| | | [WHERE type='S'] | (Full Audit) | |
| +------------------+ +-----------------+ |
| | |
| v |
| +-----------------+ |
| | Business Vault | |
| | Filter Logic | |
| +-----------------+ |
+-----------------------------------------------------------------------------------+
The data platform team is responsible for auditing the data delivered to the platform, not for auditing the operational rules of source applications. If the source system delivers a dataset, the Raw Data Vault must store it completely. If business requirements specify that subsetting should occur after ingestion, that logic belongs exclusively in the Business Vault.
By maintaining a clear division between source-aligned raw ingestion and logic-driven business modeling, organizations build resilient data platforms capable of adapting to changing operational landscapes. To explore how your organization can implement these patterns or evaluate your current system architecture, learn more about our specialized Data Vault Consulting Services.
The Data Vault Handbook:
Core Concepts and Modern Applications
Build Your Path to a Scalable and Resilient Data Platform
The Data Vault Handbook is an accessible introduction to Data Vault. Designed for data practitioners, this guide provides a clear and cohesive overview of Data Vault principles.