What is Adjusted R-squared?
When you build a regression model, one of the first questions you might ask is:
“How good is my model?”
To answer this, people often use something called R-squared. But there’s a more helpful version of it, especially when you’re working with more than one variable: Adjusted R-squared.
If you need, I have written an article about R-squared here.
Let's get into it.
Let’s say you’re predicting someone’s income based on their years of education.
- If R² = 0.80, it means your model explains 80% of the variation in income based on education.
- The remaining 20% is due to other factors (like experience, number of skills, location, etc.) that your model doesn’t include.
What happens if you decide to add to your regression model another variable. You know, to better explain your data. Maybe you decide that including number of skills is a good idea. So you to do that.
You created a new model with another variable: number of skills. Your R-squared increases and is now 0.81. You might think: “this is great, the bigger the R-squared the better. You might be wrong.
Let's say you randomly add another variable, a column with people’s favorite pizza topping. Weirdly, your R² might go up to 0.85, even though that new variable is completely unrelated. And now you think to yourself: “what is happening?!”
That’s where Adjusted R-squared comes in.
Adjusted R-squared is a smarter version of R². It penalizes your model if you add variables that don’t actually help.
Why is that useful?
Because it helps you avoid overfitting. That means making your model too complex by adding variables that fit the training data well, but don’t generalize to new data.
Get Michal Mikulasi’s stories in your inbox
Join Medium for free to get updates from this writer.
Here is a scary-looking formula:
Where:
- R² = regular R-squared
- n = number of data points
- p = number of predictors (independent variables like above mentioned number of skills)
The more variables (p) you add, the bigger the adjustment.
If the new variable helps, adjusted R² goes up.
If it doesn’t help enough, adjusted R² goes down.
Now you see why it might be a better option to use Adjusted R-squared. If you have a model where you need not one, but several variables(number of skills, location, age, etc..), you need to adjust for those.
When Should You Use Adjusted R-squared?
Use Adjusted R-squared when:
- You have multiple independent variables
- You want to compare models with different numbers of variables
- You’re doing feature selection (choosing which predictors to keep)
- You care about generalization, not just fitting training data
Use regular R-squared only for:
- Simple models with just one or two variables
- Quick, rough evaluations.
Now you can understand not only what Adjusted R-squared is, but also the difference between R-squared and Adjusted R-squared. I hope it helps!
Thanks for reading!
If you are happy that I can dedicate my time to this, please consider supporting me on:
https://buy.stripe.com/fZu14nffsgn85p67o43Ru00
Or by buying this book: https://amzn.to/44YX518
