One of the leading suppliers for military and commercial airplane structures and components sought an alternative to their data architecture as the one they had in place came with a significant cost.
About the Client
So the problem can be summarized as:
- Expensive legacy tool
- Lack of expertise within the organization with this particular tool
- Data privacy and regulatory compliance was not fully met
- Complex process and stream of data
- Lack of resources
- Highly regulated and sensitive data information
Therefore the project went through an extensive process of planning, implementation and testing to create a solution to the initial problem.
- Implementation of Enterprise Service Bus using Kafka message queues in Confluent
- Use of a tool called Mapforce for generating the data transformations
- Proposed construction of an EDW using Data Vault 2.0 to leverage NoSQL and agile development
Tangible Results for the Client
The major takeaway from the proof of concept and project implementation was the feasibility of the use of Kafka and Confluent to handle message queues in the Enterprise Service Bus which would significantly reduce the costs attributed to the architecture.
- Kafka + Confluent