Image by Pete Linforth from Pixabay
Imagine you’re keeping a diary about your favourite ice cream shop. One day, they introduce a new flavour. The next week, they rename the “Choco Fudge” to “Double Chocolate Delight.” Later, they move to a bigger location. How do you keep track of all these changes in your diary?
That’s where Slowly Changing Dimensions (SCDs) come into play. In the world of data, they help us track changes to important information over time. Let’s break it down in a way that’s easy to understand.
What Are Dimensions in Data?
Think of dimensions as the “descriptive stuff” in a database. For example:
- A customer’s name, address, and email.
- Details about a product, like its name, price, and category.
Dimensions are the context behind the raw numbers. If you’re looking at sales data, dimensions tell you who bought something, what they bought, and where.
Why Do Dimensions Change?
People and things change.
- A customer moves to a new city.
- A product gets rebranded.
- A store changes its opening hours.
If we don’t track these changes, we might lose important details. For example, what if you wanted to know how many items were sold at the old store location before they moved?
What Are Slowly Changing Dimensions (SCDs)?
SCDs are techniques to manage these changes. They’re like different ways of writing down updates in your diary.
Here are the main types of SCDs:
SCD Type 1: Overwrite the Old Information
Imagine you erase the old name of the ice cream flavour in your diary and replace it with the new one.
- Pro: It’s simple and saves space.
- Con: You lose the history.
Example:
Yesterday: “Choco Fudge”
Today: Erased and replaced with “Double Chocolate Delight”
SCD Type 2: Keep a History of Changes
Instead of erasing, you add a new entry every time something changes.
- Pro: You have a full history.
- Con: It takes more space.
Example:
- Jan 1: “Choco Fudge”
- Jan 8: “Double Chocolate Delight” (new row in your database)
SCD Type 3: Add a New Column for the Change
You add a “before” column in your diary to track just the last change.
- Pro: You keep some history without adding too much complexity.
- Con: It’s limited. You can’t track all the past changes.
Example:
- Current Name: “Double Chocolate Delight”
- Previous Name: “Choco Fudge”
Why Are SCDs Important?
Let’s say you’re running a business. You need to know:
- What products were popular before and after a rebranding.
- How a customer’s location affects their buying habits over time.
- How your shop’s sales changed when you moved to a new location.
Without SCDs, this kind of analysis becomes messy or even impossible.
How Are SCDs Used in Real Life?
- Netflix
They track what shows you’ve watched and how your preferences change over time. If you suddenly start watching sci-fi instead of rom-coms, they update your profile to suggest better recommendations. - E-Commerce (like Amazon)
Your address history matters! If you ordered something last year to an old address, they need to know where to send the package today. - Healthcare
Doctors need to track your medical history. If you change medications or move to a new doctor, they can still see your past treatments if your details are tracked in a database.
Challenges of SCDs
While they sound amazing, SCDs come with some hurdles:
- More Data: Tracking history means more storage.
- Complexity: Updating databases can get tricky.
- Performance: More data can slow down your system.
Why Should You Care?
Understanding SCDs is like knowing how to keep a diary that doesn’t lose important details. Whether you’re tracking your favourite ice cream flavours or running a business, knowing how to handle changes helps you make better decisions.
So, the next time you wonder how Spotify remembers your favourite songs or how Amazon keeps track of your address history, you’ll know it’s all thanks to Slowly Changing Dimensions!