Let’s admit that ChatGPT took the world by surprise. Of course, many books and movies predicted AI, and we knew a technology like this was coming someday, but most thought “someday” was at least a decade away. Well, it wasn’t, and now we have a difficult task ahead of us: explaining how ChatGPT works.
If you’re one of the 100 million record-breaking users already benefiting from ChatGPT, you know it’s a game-changer. But are you curious about what makes it so powerful? Then, keep reading! We’re about to dive into how ChatGPT works, and you don’t want to miss it.
ChatGPT: How does it work?
It would be bad journalistic practice—not to mention rude—to write a story about someone or something without letting them share their point of view. So, let’s start by asking ChatGPT how it works:
Víctor: Hello ChatGPT, how do you work?
ChatGPT: Hello! I work as an AI language model by using deep learning algorithms. I’ve been trained on a vast amount of text data to understand and generate human-like responses. When you interact with me, I analyze your input, search for patterns and relevant information, and generate a response based on what I’ve learned during training. My goal is to provide helpful and coherent information based on the input you provide.
So, ChatGPT analyzes user input, searches for patterns and relevant information, and generates a response. It’s like an old-and-well-known chatbot—the “Chat-” in its name—but way more powerful (that must be the “-GPT”).
GPT stands for Generative Pre-trained Transformer. Transformers are a technology that helps computers understand and produce human-like language. They do so by:
- Weighting the relationships between words or tokens, assigning importance to each one.
- Finding patterns and predicting which words should come after others.
Here’s the process:
- The software breaks down the prompt into tokens (words or subwords).
- ChatGPT uses its training to identify patterns, relationships, and associations between tokens to comprehend the input.
- Finally, it generates a response by predicting the most likely sequence of words based on the context and knowledge acquired during training, spiced with a bit of randomness to keep it fresh.
Where does ChatGPT get its data?
ChatGPT gets its data from its training. Its current model (GPT-3.5 or GPT-4 for subscribers) was trained by “reading” books, websites, articles, and other publicly available written material with the help of human AI trainers.
OpenAI, the company behind ChatGPT, did not share how much information it used. However, the training corpus likely consisted of hundreds of gigabytes of text and maybe 500 billion tokens. “Basically, it was allowed to crunch through the sum total of human knowledge,” says specialized writer Harry Guinness in his article about how ChatGPT works.
One crucial thing to notice is that ChatGPT has no real-time access to the internet, and its knowledge was cut off in September 2021. Therefore, any information about events that occurred after that timeframe is out of its scope.
Why won’t ChatGPT work? Top 3 limitations and how to fix them
While it’s a powerful tool capable of amazing feats, ChatGPT is a work in progress that still has its limitations. Here are its top weak points and how to make it work better:
1. No common sense
Data is central to how ChatGPT works, but data is not knowledge. Moreover, ChatGPT lacks one key feature humans have: common sense. In other words, ChatGPT can produce human-like and grammatically correct responses but can’t guarantee they are nonsense-free. For instance, it might claim the Berlin Wall fell before WWII without blinking.
How to fix this:
- There’s no way to correct this directly. Just remember ChatGPT is a super-powerful chatbot rather than the ultimate source of truth.
- Always double-check ChatGPT output with reliable sources when necessary.
2. Out-of-scope knowledge
ChatGPT was trained using an incredible amount of information, but there’s always a limit. The more specific the topic, the less likely ChatGPT has read enough about it to provide a reasonable answer. Additionally, ChatGPT’s knowledge is capped at September 2021, so it can’t access recent events.
How to fix this:
- Stick to what ChatGPT does best: generating creative content, explaining concepts, translating and summarizing texts, brainstorming ideas, etc.
3. Not so great for long-form content
Many users have complained about ChatGPT struggling with long-form content. While there’s no specific word limit, ChatGPT aims to provide concise and informative answers. This often means responses don’t exceed ~500 words, and longer outputs might lack quality.
How to fix this:
- Break longer content into smaller sections (e.g., write an article by focusing on its subtitles individually).
- Use ChatGPT to generate lists and brief explanations, then use them in follow-up prompts.