Skip to main content
search
0
CartClose Cart
In Data Vault Friday

(Logical) Information Marts in Data Vault

Watch the Video

In our continuous Data Vault Friday series, our CEO Michael Olschimke addresses a question from our audience that delves into the intricacies of the CDVP2 training.

“We are having trouble understanding the attached slide 28 of the CDVP2 training.

– What is the difference between Business DV Pits & Bridges and Pits & Bridges?
– We are confused about why Business Vault and Info Mart are put into one logical wrapper. Why does physical and logical wrapper differentiate?”

In this elucidating video, Michael provides clarification on the distinctions between “raw” and “business” Point-in-Time (PIT) and bridge tables. The question prompts a discussion on understanding the nuances of these components within the Data Vault methodology.

Michael shares insights into the reasoning behind grouping Business Vault and Info Mart into one logical wrapper while emphasizing the differentiation between physical and logical wrappers. The discussion provides valuable context for participants seeking clarity on the CDVP2 training material.

Meet the Speaker

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

In Intermediate

Why Data Vault 2.0 Is the Best Data Model for Automation

Watch the Webinar

Why Data Vault 2.0 Is the Best Data Model for Automation

Many data teams worry that automation won’t work on their specific data and technology stack. They’ve learned the hard way that automation doesn’t always stand up to the complexity of different source data models, taxonomies, and tech stack components.
Join this webinar to understand how Data Vault 2.0 is designed to focus on models and logic, not complex code so that it’s rapidly becoming the DWH standard.

We’ll explain how Data Vault has taken the best of the more traditional modeling
approaches, such as Inmon or Kimball, to provide the level of abstraction, quality, and agility that automation requires.

You’ll learn how the Data Vault model and its methodology and architecture leverage
automation. And how we use integration templates based on Data Vault standards to pave the way to fully automated data loading.

This webinar takes you from theory to practice.

Watch Webinar Recording

Webinar Agenda

1. The pros and cons of different data modeling techniques.
2. The prerequisites for automation.
3. Why Data Vault works best.
4. How to create abstractions in data warehousing.
5. Demo: how it’s applied in VaultSpeed.

In Data Vault Friday

Supersetting in Data Vault

Watch the Video

In our ongoing Data Vault Friday series, our CEO Michael Olschimke engages with a thoughtful inquiry from our audience.

“Dear Scalefree team, we receive data from the source for multiple company forms (like HoldingCompany, JointVenture), and we want to know if it’s recommended to save them in different entities (e.g., HoldingCompany_h/s, JointVenture_h/s) or one big entity (Company_h/s).

If we split them, we will have for each company form (e.g., Holding Company) about 10 links; If we store everything in one Company entity, we may face the situation that different company forms have different master data in the future, besides, it violates the Data Vault 2.0 rule that we should save the data as delivered by the source.”

In this insightful video, Michael delves into the strategic considerations of applying sub-setting and super-setting in the context of Data Vault 2.0. The question prompts a discussion on where to employ these techniques and the potential exceptions that might arise from the default strategy.

Michael provides practical insights and recommendations for effectively handling diverse company forms within the Data Vault framework, ensuring compliance with Data Vault 2.0 principles while addressing the complexities of master data variations.

Meet the Speaker

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

In Data Vault Friday

Reference Table Vs. Reference Hub in Data Vault

Watch the Video

In this week’s Data Vault Friday, our CEO Michael Olschimke addresses an intriguing question from our audience regarding the difference between a Reference Table and a Reference Hub.

“If I need to historize the reference table, I can use the Satellite pattern. Ok, I have now a Reference Satellite table. But what about the Reference Hub table? Is it effective to create a table with just one column?”

In this informative video, Michael explores the concept of historizing reference tables within Scalefree‘s Data Vault 2.0 projects. The question specifically focuses on the efficiency and effectiveness of creating a Reference Hub table with just one column.

Michael shares insights into the considerations and scenarios where creating a Reference Hub table with a single column can be a viable and effective approach. The discussion provides practical guidance for handling reference tables within the Data Vault 2.0 methodology.

Meet the Speaker

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

In Data Vault Friday

Calculating Hash Keys in Business Vault

Watch the Video

In our ongoing Data Vault Friday series, our CEO Michael Olschimke delves into a thought-provoking question from our audience.

“When calculating hash_key in links in Business Vault, it sometimes can be quite expensive to join all hubs to get the business keys, etc. In many cases, we keep those hash_keys to keep the standards only. And even for any case where you may need to build a satellite for that link, that means you would have the same granularity. So is it still a no-go to generate the link hash_key from the hub hash_keys to prevent expensive joins in some cases? If so, what do you suggest?”

In this insightful video, Michael addresses the considerations and challenges related to calculating hash keys in links within the Business Vault. The question prompts a discussion on the trade-offs between keeping hash keys for standards and the potential expense of joins, especially when dealing with multiple hubs.

Michael shares his expertise on hashing practices in Data Vault 2.0 links, offering recommendations and considerations to optimize the balance between standards and performance in the Business Vault.

Meet the Speaker

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

In Beginner, Salesforce

Top 10 Salesforce Features – 2023 (German)

Watch the Webinar

Top 10 Salesforce Features – 2023 (German)

Entdecke die neuesten Entwicklungen für Salesforce mit dem Spring ’23 Update! Unser Team hat die Release-Notes genau durchgearbeitet, um dir die besten neuen Funktionen vorzustellen, die jetzt in deiner Organisation verfügbar sind. Komm an Bord und erfahre, wie du diese Tools nutzen kannst, um deine Arbeitsabläufe zu optimieren und deine Effizienz zu steigern. Nutze die Chance, um dein Wissen über Salesforce zu erweitern und deine Fähigkeiten nachhaltig zu verbessern.

Watch Webinar Recording

Webinar Agenda

1. Top 10 bis 4
2. Top 3 im Detail
3. Ausblick und Q & A

Meet the Speaker

Picture of Markus Lewandowski

Markus Lewandowski

Markus Lewandowski hat mehr als 6 Jahre Salesforce Erfahrung und ist ein zertifizierter Salesforce Berater bei Scalefree. Er hilft Kunden in ganz Europa, Salesforce Umgebungen zu implementieren, zu verbessern und in ihren Tech-Stack zu integrieren.

In Data Tools, Intermediate

Speed Up Your Data Vault 2.0 Implementation with Turbovault4DBT

TurboVault4dbt Logo

TurboVault4dbt

Scalefree released TurboVault4dbt, an open-source package to automate model generation using DataVault4dbt-compatible templates based on your sources’ metadata.

TurboVault4dbt currently supports metadata input from Excel, GoogleSheets, BigQuery, and Snowflake and helps your business with:

  • Speeding up the development process, reducing development costs, and producing faster results
  • Encouraging users to analyze and understand their source data

Speed up Your Data Vault 2.0 Implementation – with TurboVault4dbt

This webinar delves into TurboVault4dbt, an open-source tool by Scalefree that speeds up Data Vault 2.0 implementation. It automates dbt model creation using your source metadata, saving time and costs while encouraging better data analysis.

TurboVault4dbt works with metadata inputs like Excel, Google Sheets, BigQuery, and Snowflake, generating models for hubs, links, and satellites automatically. Just set up your metadata tables, connect the tool, and watch it do the heavy lifting!

Watch webinar recording

In this article:

‘Isn’t every model kind of the same?’

Datavault4dbt is the result of years of experience in creating and loading Data Vault 2.0 solutions forged into a fully auditable solution for your Data Vault 2.0 powered Data Warehouse using dbt.

But every developer who has worked with the package or has created dbt models for the Raw Vault must have come across one nuisance:

Creating a new dbt model for a table means taking the already existing template and providing it with specific metadata for that table. Doing this over and over again can be quite a chore. This is why we created TurboVault4dbt to automate and speed up this process.

From CTRL+C AND CTRL+V to a simple mouse-click

How many times has everyone pressed CTRL+C then CTRL+V and corrected a few lines of code when creating new dbt-models for the raw vault?

Instead of trying to figure out what the names of your tables and business keys are or what hashing order you want your Hashkey to be generated in, TurboVault4dbt will do all of that for you. All TurboVault4dbt needs is a metadata input where you capture the structure of your data warehouse.

TurboVault4dbt

TurboVault4dbt currently requires a structure of five metadata tables:

  • Hub Entities: This table stores metadata information about your Hubs,
    e.g. (Hub Name, Business Keys, Column Sort Order for Hashing, etc.)
  • Link Entities: This table stores metadata information about your Links,
    e.g. (Link Name, Referenced Hubs, Pre-Join Columns, etc.)
  • Hub Satellites: This table stores metadata information about your Hub Satellites,
    e.g. (Satellite Name, Referenced Hub, Column Definition, etc.)
  • Link Satellites: This table stores metadata information about your Hub Satellites,
    e.g. (Satellite Name, Referenced Link, Column Definition, etc.)
  • Source Data: This table stores metadata information about your Sources,
    e.g. (Source System, Source Object, Source Schema, etc.)

By capturing the metadata in those five tables above, TurboVault4dbt can extract necessary information and generate every model that is based on a selected source but also, as a user, encourage you to analyze and understand your data.

Conclusion: Lean back, relax and let TurboVault4bdt take over!

Create and fill your metadata tables, connect them to TurboVault4dbt, and enjoy your free time for another cup of coffee. Give it a try, or give us your feedback by visiting TurboVault4dbt on GitHub!

Stay updated on TurboVault4dbt through our marketing channels as great features lie ahead!

In Data Vault Friday

PIT Table Structure in Data Vault

Watch the Video

In our continuous Data Vault Friday series, our CEO Michael Olschimke engages with an insightful question from our audience.

“Is it possible to add business keys and/or descriptive attributes to a Point-in-Time (PIT) table to improve performance when filtering or joining data in the information mart?”

In this concise yet informative video, Michael delves into the consideration of enhancing the performance of filtering or joining data in the Information Mart by incorporating business keys and descriptive attributes into a PIT table. The question prompts a discussion on the circumstances and scenarios where denormalizing these elements into a PIT table may be beneficial.

Michael shares practical insights and considerations, providing clarity on when and how the inclusion of business keys and descriptive attributes in a PIT table can contribute to improved performance in data retrieval and analysis within the Information Mart.

Meet the Speaker

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

In Data Vault Friday

Bridge Table and Zero Code Impact in Data Vault

Watch the Video

In our ongoing Data Vault Friday series, our CEO Michael Olschimke addresses a pertinent question from our audience.

“We are currently implementing a bridge table over a series of sprints. The table prepares a fact entity with many measure values that are added sprint by sprint. Some measures are based on other measures in the bridge table. Our issue is that the code to load the bridge table is already complex due to the many measures. It exceeds 800+ lines of code and requires constant reengineering when additional measures are added. Is there a more agile approach with less, maybe zero change impact on the existing code?”

In this insightful video, Michael explores strategies for building a bridge table in an agile and incremental fashion. The question prompts a discussion on addressing the complexity of the loading code and finding approaches that minimize change impact, ensuring a more flexible and adaptive development process.

The video offers practical insights and recommendations for streamlining the implementation of a bridge table, enhancing agility, and reducing the challenges associated with code maintenance in evolving data models.

Meet the Speaker

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

In Intermediate

Boost ROI of Data Infrastructure with Automation

Watch the Webinar

Boost ROI of Data Infrastructure with Automation

Generating returns from a modern data infrastructure is tough. First, creating a central repository for easy data access requires much upfront, traditionally manual work to set up data ingestion, mapping, metadata management, etc. Changes in sources, tech stack, and taxonomies require more work. Or someone new comes on board and proposes building an entirely new model to answer the same business question. Typically, all this pushes the data team to take shortcuts to regain lost time, creating technical debt. In this webinar, we’ll explain how automation done right, following Data Vault 2.0 standards, will not only cut manual work but solve problems of agility, uncertainty, and output quality, to ultimately provide the return you expect. Learn about what can go wrong — and how to get it right.

Watch Webinar Recording

Webinar Agenda

1. Common pitfalls in data management.
2. How the problems were solved in the past: what worked and what didn’t
3. How Data Vault methodology combined with automation brings new solutions…
4. … And how this will save you time, and money.

Meet the Speakers

Profile picture of Michael Olschimke

Michael Olschimke

Michael has more than 15 years of experience in Information Technology. During the last eight years he has specialized in Business Intelligence topics such as OLAP, Dimensional Modelling, and Data Mining. Challenge him with your questions!

blank

Dirk Vermeiren

Dirk Vermeiren is CTO at VaultSpeed. His lifelong experience in data management stretches over 25 years. He used Data Vault as the driving methodology for building large data warehouses. Along this path, he was one of the driving forces behind a Data Vault automation framework that gradually evolved into the product: VaultSpeed.

In Data Vault Friday

Zero Key Concepts in Data Vault

Watch the Video

In our ongoing Data Vault Friday series, our trainer Marc Finger delves into an intriguing question posed by the audience.

“In Hubs, we add two ghost records: one with 0s (unknown/zero key) and another with f’s (sometimes called error key). In the loading of the stage, in which cases should we replace the generated hash key with the error key instead, and how? Right now, if the Business Key (BK) or combination of BKs is null, we are always replacing it with the zero key. My question is in which cases should we use the ffff key instead.”

In this informative video, Marc explores the usage and value of zero keys when loading links within the Data Vault framework. The question prompts a discussion on the considerations and scenarios where replacing the generated hash key with the error key, represented by ‘ffff,’ is beneficial.

The video provides practical insights and recommendations for optimizing the handling of ghost records and error keys, contributing to a more robust and efficient Data Vault implementation.

Meet the Speaker

Marc Winkelmann

Marc Finger

Marc is working in Business Intelligence and Enterprise Data Warehousing (EDW) with a focus on Data Vault 2.0 implementation and coaching. Since 2016 he is active in consulting and implementation of Data Vault 2.0 solutions with industry leaders in manufacturing, energy supply and facility management sector. In 2020 he became a Data Vault 2.0 Instructor for Scalefree.

In Data Vault Friday

Realtime Architecture in Data Vault

Watch the Video

In our continuous Data Vault Friday series, our CEO Michael Olschimke addresses a thoughtful question from our audience.

“What additional steps are there in a Real-Time loading pattern on top of the batch loading pattern?”

In this concise yet informative video, Michael focuses on the nuances of incorporating real-time loading patterns into the Data Vault 2.0 architecture. The question prompts a discussion about the specific steps that distinguish real-time loading from the traditional batch loading pattern.

Michael shares insights into the additional considerations and steps required to ensure the effectiveness of real-time data integration. The discussion provides valuable guidance for those looking to enhance their understanding of real-time loading within the context of the Data Vault 2.0 framework.

Close Menu