Time doesn't literally go faster, it just feels like it does. That's because your perspective on time changes.
Think about it, when you are five years old, a year seems like forever because it only represents one/fifth of your life. So, if a year is equivalent to 20% of your life, than it feels slower due to your perspective.
Now, when you are ten, a year represents one/tenth, or 10% of your life, which is significantly less than 20%. Your perspective has shifted, and time seems to move quicker.
At least, that's how I've come to understand it.