Simply Magical Patterns
Good solutions for common technical data and analytics problems
Attribute Splitting Pattern in Data Warehousing
TD;LR One powerful concept in physical data modeling is the Attribute Splitting Pattern. This pattern is particularly useful for handling rapidly changing attributes within your data. In this article we’ll delve into...
Reference Data – Table
How do we model reference data, when using a Data Vault modeling approach.
AgileData Patterns
AgileData Patterns are a widely used concepts to describe good solutions to recurring technical problems using an common language
Patterns are conceptual solutions that can be applied in concrete use cases regardless of used technologies, such as architecture, software, or programming languages. Patterns are useful because they:
-
- represent proven solutions to common problems;
- organise solutions into a standard and easily referenced format;
- can be adopted and implemented by most practitioners who work in that pattern domain;
- can be used to ensure consistency in how systems are designed and built
- reduce the level of effort required to solve specific problems;
- can be selected prior to the implementation of the system;
- provide a shared language for specific problems and potential solutions.
The notion of a pattern language originated in building architecture.
Attribute Splitting Pattern in Data Warehousing
TD;LR One powerful concept in physical data modeling is the Attribute Splitting Pattern. This pattern is particularly useful for handling rapidly changing attributes within your data. In this article we’ll delve into...
Reference Data – Table
How do we model reference data, when using a Data Vault modeling approach.
Reference Data – Hubs & Sats
How do we model reference data, when using a Data Vault modeling approach.
Header-Detail – Multiple Links
How do we model transactions which have Header and Detail records (i.e Order and Order Line) using a Data Vault model
System of Capture > Persistent History > Data Vault > Consume
How do we store data centrally, in a way that provides context and absorbs changes, while also making it quickly available for use with minimal upfront work.
Golden Record – Combined Hub & Same as Link
How do we model data from multiple systems when they store copies of the same business concept (Customer, Person, Organisation, Product etc), using a Data Vault modeling approach.
Golden Record – Combined Hub
How do we model data from multiple systems when they store copies of the same business concept (Customer, Person, Organisation, Product etc), when using a Data Vault modeling approach.
Golden Record – Unique Hubs & Combined Hub
How do we model data from multiple systems when they store copies of the same business concept (Customer, Person, Organisation, Product etc), when using a Data Vault modeling approach.
Source > Data Lake > Data Vault > Consume
How do we store data centrally, in a way that provides context and absorbs changes, while also making it quickly available for use with minimal upfront work.
Source > Data Vault > Consume
How do we store data centrally, in a way that provides context and absorbs changes.
Extract Load and Transform (ELT)
How do we leverage the compute power of our target data repository to transform data.
Extract, Transform and Load (ETL)
How do we change data as we move it between data repositories.
Change Data Capture (CDC) – Log Mining
How do we capture all data changes in a source applications database as they happen, so that any changes in records can be easily collected, loaded and retained in another data repository.
Change Data Capture (CDC)
How do we capture all data changes in a source application as they happen, so that any changes in records can be easily collected, loaded and retained in another data repository.
Have a pattern to share
Get in touch and help us publish more useful patterns