The Glitch in the Nest: How I found Duolingo’s secret A/B test
What I Found
For the last 6 or 7 months, I have been using Duolingo on my mobile phone with their new Energy system. Instead of hearts, I have a battery bar with a total count of 25. I had grown used to this system, but last night when I logged into Duolingo account on my laptop after a while I found the same old Heart system.
On the web version, the Energy system was nowhere to be found. Instead, it was showing the old Heart system with a maximum of 5 hearts.
My Initial Thoguhts
When I saw the two different displays, I didn’t think Duolingo had two separate backends. Instead, I suspected that the internal logic in the backend was still based on the original 5-heart system.
Initially I thought that Duolingo’s database still stores my “life” as 5 units, but they updated the code to perform a calculation before sending the data to the mobile app. I thought the phone was simply multiplying that internal “heart” value by 5 to show 25 energy units, while the web version was showing the raw, unmultiplied value. I believed the “Energy” was just a new way of calculating and displaying the same underlying data.
Putting it to the Test
To check this, I compared the numbers. On my laptop, I had 3 hearts. On my phone, I had 21 energy.
Immediately, the math didn’t add up. If it were a simple 5:1 ratio, 3 hearts should have been 15 energy. Even more confusing, the timers were different:
the Laptop said 5 hours for a refill, while the Phone said 3 hours and 11 minutes. This suggested the two systems weren’t just looking at the same number in different ways.
AI’s Takedown:
Duolingo is likely running an “A/B test” or a slow rollout. The backend database probably stores a high-resolution number (like “Total Mistakes Allowed” or “Energy Points”), and the Web team hasn’t updated their UI to show the 25-point bar yet, so they just “map” that number back to the old 5-heart icons.
Basically, the phone is showing you the “true” granular value, while the laptop is giving you a simplified, lower-resolution version of that same data.
The Practical Experiment
I decided to do a “live test” on my laptop to see how it affected my phone. I intentionally made mistakes in a lesson on the web.
- I made 3 mistakes in 3 different problems.
- On the laptop, I lost all my hearts. The counter hit 0.
- I immediately checked my phone. To my surprise, the phone still showed 22 energy.
AI’s Observation:-
1. The “Platform-Specific Logic” Theory
Get Manas Jha’s stories in your inbox
Join Medium for free to get updates from this writer.
It appears Duolingo is not just changing the UI; they are testing a new monetization/retention model specifically for mobile users.
- Web (Legacy Logic): High stakes. 1 mistake = 1 full heart lost. This is designed to be punishing to encourage “careful” learning or a subscription.
- Mobile (New “Energy” Logic): Low stakes. It seems your mobile app is currently decoupled from the mistake penalty on the web. You lost 3 hearts on your laptop, but your phone didn’t “subtract” 15 energy points to match.
2. The “State Synchronization” Failure
Normally, when you do an action on one device, the Backend API sends an update to the database, and other devices “poll” that data to stay in sync.
- The Glitch: Because the “Energy” system is likely a brand-new feature being A/B tested, the developers might not have written the “translation layer” yet.
- The Result: Your “Heart Count” in the database is 0, but your “Energy Count” is a separate column in the database that hasn’t been touched by your laptop activity.
3. Why would a company do this? (The “Business” Logic)
From a product perspective, there are two big reasons:
- A/B Testing: They want to see if people on phones (who usually have shorter attention spans) play longer if they have “25 lives” instead of “5 lives.” They keep the laptop version old-school to use as a “control group” for the experiment.
- Granularity: As you noticed, the laptop is “all or nothing” one mistake on a new problem kills a heart. The 25-energy system is likely designed to eventually take 1 or 2 energy points per mistake, allowing for ~15 mistakes before you’re blocked, instead of just 5.
What I researched about the “Energy” Logic
I looked up the official Duolingo blog to understand the rules of the Energy system. I found out that it works very differently from Hearts.
- Energy is a “learning battery” that powers your activity.
- You lose 1 energy for every single problem you solve, even if your answer is correct. (This is concluded through practical)
- You can gain energy back through “correct answer streaks.”
- It’s a limit on how many total questions you can answer, rather than a penalty for being wrong.
What Came to my mind later
I thought that because both the application versions (web app and mobile phone) are managing their own life/energy systems independently, they must be using different databases, or maybe branching of database, just like we do in git.
AI’s takedown:-
1. The “Shadow Account” or Split-State Logic
- Instead of a single “Stamina” column that gets translated, it is very likely that Duolingo has created a completely separate table or service for the new Energy system.
- Legacy State: Lives in the original database. It tracks hearts_remaining.
- Energy State: Lives in a new microservice database. It tracks energy_points.
When you log in on the Web, the backend only looks at the Legacy State. When you log in on Mobile, it only looks at the Energy State. Because Duolingo’s team are A/B testing, they have purposely disabled synchronization between these two fields to see which system performs better in isolation. If they synced them, they wouldn’t know if your behavior was caused by the “Hearts” or the “Energy.”
2. Why they haven’t “Reflected” each other?
- In software engineering, this is often called Feature Branching at the Database level.
If Duolingo is planning to kill the Heart system entirely, they don’t want to write complex “sync” code that they are just going to delete in three months. Instead, they just let the two systems run in parallel.
Conclusion
After these tests and research, I realized that my initial theory was wrong. The systems aren’t just the same logic with different math; they are completely separate data silos. Basically Duolingo is running an A/B test where the Legacy Heart system and the New Energy system operate on totally different tracks in the backend.They probably keep these systems separate to collect “clean” data on which one performs better.
For me, this means I essentially have two separate sets of “lives” wohoo!🙌🏻. If I run out of energy on my phone, I can just switch to my laptop and keep learning because the two systems don’t talk to each other.
