Why can’t ChatGPT count to 1 million? And what does it reveal?
As I was scrolling through TikTok on a Sunday morning while in my bed… like a typical Gen Z person would do, I scrolled upon one interesting video where a person tries to make ChatGPT count to 1 million. The first thought that came into my mind was, “surely! ChatGPT can count to 1 million, right?” but as the video progressed, no matter how hard the person tried to make ChatGPT count, the AI model would give some kind of excuse to dodge the counting and tried to change the topic into another direction.
After seeing this video, I was really keen to know by myself, “is this actually true?”. So, I rolled myself out of the bed and got hold of my laptop and went straight into ChatGPT to search, “Can you count to 1 million?”, and these were the answers that the AI model gave me.
“Hey! 😄 Technically, yes — I can count to 1,000,000… but it wouldn’t be very practical to list them all here (you’d be scrolling forever 😅)”
“So, before I unleash the infinite scroll monster — can you tell me what you actually want to do by “count to 1 million”?”
As I was trying forcefully to get the AI to count from 1 to Million, the assertiveness really gets stronger and stronger and eventually it tries to get away from the real problem by giving some other options like a downloadable text file with numbers from 1 to million or a python script which runs numbers until 1 million. This baffled me with the thought in my mind, “Is this almighty… all-powerful tool incapable of counting to 1 million?”. So, I started to do my research and found that the real question is not “Why can’t ChatGPT count to 1 million?” and rather “Why won’t ChatGPT count to 1 million?”. (7news, 2025)
To find an answer to this problem, we have dive deep into the core of Artificial Intelligence and model training. Most AI models including ChatGPT are developed with information that is publicly available on the internet, information that third-parties provide or the information that the users provide or generate. The model analyzes the data and learns patterns and relationships among them. Then they use that understanding to predict the next most likely word, pixel or frame. Moreover, ChatGPT is not a single model. It is a collection of several highly trained models for specific tasks. GPT-4o and GPT-4.5 are the latest models used to generate text, process and understand images and reply quickly. Models like o1-mini have advanced reasoning capabilities. The famous DALL·E 3 model generates images. (OpenAI, 2025)
The GPT of ChatGPT stands for Generative Pre-trained Transformer. A massive part of developing process of these models involves “Training”. In the earlier days, when AI just made its way to the public, mostly supervised learning was used in training models. Vast amounts of data which were manually labeled like, databases of trees with pictures of different trees paired with a description of that tree, were fed into the system so that system can identify the patterns and predict data. With the unimaginably massive amounts of data available on the internet, supervised learning was like trying to empty the Indian ocean with a teaspoon. It was a futile task. As a result, the unsupervised learning was put into use. Instead of giving pre-labeled data, the model was provided with data and few ground rules and left unsupervised to crunch through all this data and develop its own understanding.
Another point which cannot be left out when finding an answer to the question “Why won’t ChatGPT count to 1 million?”, is the Architecture. Like any other software, these AI tools also have architecture. But they are not the typical software architecture that we learn during our software architecture course unit module. The ChatGPT specifically uses a Transformer Architecture (T in GPT) that is the heart of the model. These transformers are the real heroes behind the scenes, which allows the computations to be parallelized, making AI models bigger, better, faster and cheaper. However, these transformers do not work with words. Rather they work with “Tokens”, which are chunks of text or an image encoded as a vector (that means it has a value and a position/direction). GPT-3 model — the first publicly introduced model of ChatGPT was trained using around 500 billion tokens. This allows the models to easily assign values and meanings which results in a more realistic prediction. Once a user inputs a prompt, the data is tokenized. The maximum number of tokens that the model can handle in a single operation, including both input and the generated output is known as the Context Window. As an example, an AI model with 40,000 Token context window should fit everything — the input (prompt), system instruction and the generated output within that window. This can explain why ChatGPT tries to limit its answers. (Rishav Paul, 2025)
Get Association of Computing Students (ACS)’s stories in your inbox
Join Medium for free to get updates from this writer.
From a different standpoint, it could be argued that the AI/ChatGPT doesn’t actually understand things, it is not general intelligence or doesn’t have common sense. It simply is a manmade tool, which learns from data and previous experiences to give an output which best matches to the given input. You could think of ChatGPT as a very well-born person whose name is James, but sadly with some profound brain issues. James’s family has spent hundreds of millions of dollars getting James to the point looking and sounding good. But he has never ridden a bike or made a sandwich. Imagine you are meeting James over for dinner, and James seems extremely impressive. Your natural instinct would be to think that James is a smart, impressive guy. But actually, if you ask James to make a sandwich, it would be an impossible task for him since he has never learned or come across making sandwiches. Same logic applies to the situation of you asking ChatGPT, “Hey, can you count to 1 million?”.
Therefore, this question may have more than one correct answer. ChatGPT like models are trained in a way that they build conversation and provide quick and short answers. These models are restricted by their own architecture, rather than by their intelligence. Also, these models lack general intelligence or common sense. Instead, what it tries to do is a game of “finish the sentence”. It takes your prompt, breaks it down to tokens and using the transformer based neural network architecture it figures out what you are really looking for and generates an appropriate token sequence, relying on what it has previously learned. That is why ChatGPT won’t count to 1 million.
This brings us to the crucial idea of AI still has a lot of limitations. The architecture, lack of understanding and common sense, resource intensiveness, data dependency are few major factors that limit the scope of Artificial Intelligence. (Mark Levis, 2024). AI may provide you with what you need, fluently and confidently. But still, fluency is not proportional to accuracy. That is why it states “ChatGPT can make mistakes. Check important info” in the bottom of your ChatGPT window. Some people ask questions about when AI will take over the world from humans, or will it replace the livelihood of everyone. Still, it is very early for those questions to be answered because AI still has a lot of room for improvement.
Simply AI is a derivative of numbers, patterns and algorithms which operate within the limits drawn by data. (Eliezer Yudkowsky). It does not yet have the capabilities of creativity, emotion or reasoning as humans do. So, comparing human intelligence to AI is not a fair task. It is a fact that technological advancements are on fast track, and with the increased computational power through quantum computing, the line between human and AI may have started to become finer. But more importantly, it is the combination of AI’s efficiency with human’s creative thinking that drives us to the future. Therefore, the future needs people who can effectively use AI, and also people who can think beyond AI.
Article by: Praveen Seneviratne
22/23 Batch
Faculty of Computing
University of Sri Jayewardenepura
References
7news Australia. (2025 August 25). Asking ChatGPT to count to 1 million. https://www.youtube.com/shorts/jIYjzQdSA5k
OpenAI. (2025 August). How ChatGPT and our foundation models are developed. https://help.openai.com/en/articles/7842364-how-chatgpt-and-our-foundation-models-are-developed
Rishav Paul. (2025 February 10). Tokenization in GPT: A Deep Dive for Engineers. Retrieved from medium.com: https://paulrishav.medium.com/role-of-tokenization-in-llms-85a086b04fa9
Eliezer Yudkowsky. The fine line between AI and human creativity. https://www.unswbsoc.com/blogs/the-fine-line-between-ai-and-human-creativity
Mark Levis. (2024 January 16). Understanding The Limitations Of AI (Artificial Intelligence). Retrieved from medium.com: https://medium.com/@marklevisebook/understanding-the-limitations-of-ai-artificial-intelligence-a264c1e0b8ab
