OpenAI’s new chatbot has garnered attention for its impressive answers, but how much of it is believable? Let’s explore the darker side of ChatGPT.
ChatGPT is a powerful AI chatbot that is quick to impress, yet plenty of people have pointed out that it has some serious pitfalls.
From security breaches to privacy concerns to the undisclosed data it was trained on, there are plenty of concerns about the AI-powered chatbot, yet the technology is already being incorporated into apps and used by millions of users, from students to company employees.
With no sign of AI development slowing down, the problems with ChatGPT are even more important to understand. With ChatGPT set to change our future, here are some of the biggest issues.
What Is ChatGPT?
ChatGPT is a large language model designed to produce natural human language. Much like having a conversation with someone, you can talk to ChatGPT, and it will remember things you have said in the past while also being capable of correcting itself when challenged.
It was trained on all sorts of text from the internet, think Wikipedia, blog posts, books, and academic articles. Alongside responding to you in a human-like way, it can recall information about our present-day world plus pull up historical information from our past.
Learning how to use ChatGPT is simple, and it’s easy to be fooled into thinking that the AI system performs without any trouble. However, in the months following its release, key problems emerged around privacy, security, and its wider impact on people’s lives, from jobs to education.
1. Security Threats and Privacy Concerns
In March 2023, a security breach meant some users on ChatGPT saw conversation headings in the sidebar that didn’t belong to them. Accidentally sharing users’ chat histories is a serious concern for any tech company, but it’s especially bad considering how many people use the popular chatbot.
As reported by Reuters, ChatGPT had 100 million monthly active users in January 2023 alone. While the bug that caused the breach was quickly patched, the Italian data regulator demanded that OpenAI stop all operations that processed Italian users’ data.
The watchdog organization suspected that European privacy regulations were being breached. After investigating the issue, it requested that OpenAI meet several demands to reinstate the chatbot. OpenAI eventually resolved the issue with regulators by making several significant changes. For a start, an age restriction was added, limiting the use of the app to people 18+ or 13+ with guardian permission. It also made its Privacy Policy more visible and provided an opt-out Google form for users to exclude their data from training ChatGPT and delete it entirely if they want.
These changes are a great start, but the improvements should be extended to all ChatGPT users.
This isn’t the only way that ChatGPT poses a security threat either. It’s just as easy to accidentally share confidential information as a user. One good example is how Samsung employees shared company information with ChatGPT several times.
2. Concerns Over ChatGPT Training and Privacy Issues
Following the massively popular launch of ChatGPT, many people have questioned how OpenAI trained its model in the first place.
Even with improved changes to OpenAI’s privacy policies following the incident with Italian regulators, it may not be enough to satisfy the General Data Protection Regulation (GDPR), a data protection law that covers Europe.
It’s highly likely that OpenAI scooped up personal information when it trained ChatGPT. While the laws in the United States are less definitive, European data laws still protect a person’s personal data, whether they post that info publicly or privately.
Similar arguments against training data are being waged by artists who say they never consented for their work to train an AI model. At the same time, Getty Images sued Stability.AI for using copyrighted images to train its AI models.
Unless OpenAI publishes its training data, the lack of transparency makes it difficult to know whether it was done lawfully. For example, we simply don’t know the details about how ChatGPT is trained, what data was used, where the data comes from, or what the system’s architecture looks like in detail.
3. ChatGPT Generates Wrong Answers
It fails at basic math, can’t seem to answer simple logic questions, and will even go as far as to argue completely incorrect facts. As people across social media will attest, ChatGPT can get it wrong on multiple occasions.
OpenAI knows about this limitation, writing that: “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.” This “hallucination” of fact and fiction, as it’s been referred to, is especially dangerous regarding things like medical advice or getting the facts right on key historical events.
ChatGPT doesn’t use the internet to locate answers, unlike other AI assistants like Siri or Alexa. Instead, it constructs a sentence word by word, selecting the most likely “token” that should come next based on its training. In other words, ChatGPT arrives at an answer by making a series of guesses, which is part of why it can argue wrong answers as if they were completely true.
While it’s great at explaining complex concepts, making it a powerful tool for learning, it’s important not to believe everything it says. ChatGPT isn’t always correct—at least, not yet.
4. ChatGPT Has Bias Baked Into Its System
ChatGPT was trained on the collective writing of humans across the world, past and present. Unfortunately, this means that the same biases that exist in the real world can also appear in the model.
ChatGPT has been shown to produce some terrible answers that discriminate against gender, race, and minority groups, which the company is trying to mitigate.
One way to explain this issue is to point to the data as the problem, blaming humanity for the biases embedded on the internet and beyond. But part of the responsibility also lies with OpenAI, whose researchers and developers select the data used to train ChatGPT.
Once again, OpenAI knows this is an issue and have said that It’s addressing “biased behavior” by collecting feedback from users and encouraging them to flag ChatGPT outputs that are bad, offensive, or simply incorrect.
With the potential to cause harm to people, you could argue that ChatGPT shouldn’t have been released to the public before these problems were studied and resolved. But a race to be the first company to create the most powerful AI model might have been enough for OpenAI to throw caution to the wind.
By contrast, a similar AI chatbot called Sparrow—owned by Google’s parent company, Alphabet—was released in September 2022. However, it was purposely kept behind closed doors because of similar safety concerns.
Around the same time, Facebook released an AI language model called Galactica, intended to help with academic research. However, it was rapidly recalled after many people criticized it for outputting wrong and biased results related to scientific research.
5. ChatGPT Might Take Jobs From Humans
The dust is yet to settle after the rapid development and deployment of ChatGPT, but that hasn’t stopped the underlying technology from being stitched into a number of commercial apps. Among the apps which have integrated GPT-4 are Duolingo and Khan Academy.
The former is a language learning app, while the latter is a diverse educational learning tool. Both offer what is essentially an AI tutor, either in the form of an AI-powered character that you can talk to in the language you are learning. Or as an AI tutor that can give you tailored feedback on your learning.
This could be just the beginning of AI holding human jobs. Among the other industry jobs facing disruption are paralegals, lawyers, copywriters, journalists, and programmers.
On the one hand, AI could change the way we learn, potentially making education more accessible and the learning process a little bit easier. But on the other, a huge cross-section of human jobs face going away at the same time.
As reported by The Guardian, Education companies posted huge losses on the London and New York stock exchange, highlighting the disruption AI is causing to some markets as little as six months after ChatGPT was launched.
Technological advancements have always resulted in jobs being lost, but the speed of AI advancements means multiple industries are facing rapid change all at once. There’s no denying that ChatGPT and its underlying technology are set to reshape our modern world drastically.
6. ChatGPT Is Challenging Education
You can ask ChatGPT to proofread your writing or point out how to improve a paragraph. Or you can remove yourself from the equation entirely and ask ChatGPT to do all the writing for you.
Teachers have experimented with feeding English assignments to ChatGPT and have received answers that are better than what many of their students could do. From writing cover letters to describing major themes in a famous work of literature, ChatGPT can do it all without hesitation.
That begs the question: if ChatGPT can write for us, will students need to learn to write in the future? It might seem like an existential question, but when students start using ChatGPT to help write their essays, schools will have to think of an answer fast.
It’s not only English-based subjects that are at risk either; ChatGPT can help with any task involving brainstorming, summarizing, or drawing intelligent conclusions.
It’s no surprise that students are already taking it upon themselves to experiment with AI. The Stanford Daily reports that early surveys show a significant number of students have used AI to assist with assignments and exams. In response, some educators are re-writing courses to get ahead of students using AI to skim through classes or cheat on exams.
7. ChatGPT Could Cause Real-World Harm
It wasn’t long before someone tried to jailbreak ChatGPT, resulting in an AI model that could bypass OpenAI’s guard rails meant to prevent it from generating offensive and dangerous text.
A group of users on the ChatGPT Reddit group named their unrestricted AI model Dan, short for “Do Anything Now.” Sadly, doing whatever you like has led to hackers ramping up online scams. ArsTechnica also reports that hackers are selling rule-less ChatGPT services that create malware and produce phishing emails.
Trying to spot a phishing email designed to extract sensitive details from you is far more difficult now with AI-generated text. Grammatical errors, which used to be an obvious red flag, aren’t there because ChatGPT can fluently write all kinds of text, from essays to poems to dodgy emails.
The spread of fake information is a serious concern too. The scale at which ChatGPT can produce text, coupled with the ability to make even incorrect information sound convincingly right, makes everything on the internet questionable and amplifies the dangers of deepfake technology.
The rate at which ChatGPT can produce information has already caused problems for Stack Exchange, a website dedicated to providing correct answers to everyday questions. Soon after ChatGPT was released, users flooded the site with answers they asked ChatGPT to generate.
Without enough human volunteers to sort through the backlog, it would be impossible to maintain a high level of quality answers. Not to mention, many of the answers were simply not correct. To avoid the website being damaged, a ban was placed on all answers that were generated using ChatGPT.
8. OpenAI Holds All the Power
With great power comes great responsibility, and OpenAI holds a lot of power. It’s one of the first AI companies to truly shake up the world with not one but multiple generative AI models, including Dall-E 2, GPT-3, and GPT-4.
As a private company, OpenAI selects the data used to train ChatGPT and chooses how fast it rolls out new developments. As a result, there are plenty of experts out there warning of the dangers posed by AI, but little sign of things slowing down.
On the contrary, the popularity of ChatGPT has spurred a race between big tech companies competing to launch the next big AI model; among them are Microsoft’s Bing AI and Google’s Bard. Fearing that rapid development will lead to serious safety problems, a letter was penned by tech leaders worldwide asking for development to be delayed.
While OpenAI considers safety a high priority, there is a lot that we don’t know about how the models themselves work, for better or worse. At the end of the day, most of us have to blindly trust that OpenAI will research, develop, and use ChatGPT responsibly.
Whether we agree with its methods or not, it’s worth remembering that OpenAI is a private company that will continue developing ChatGPT according to its own goals and ethical standards.
Tackling AI’s Biggest Problems
There is a lot to be excited about with ChatGPT, but beyond its immediate uses, there are some serious problems.
OpenAI admits that ChatGPT can produce harmful and biased answers, hoping to mitigate the problem by gathering user feedback. But its ability to produce convincing text, even when the facts aren’t true, can easily be used by bad actors.
Privacy and security breaches have already shown that OpenAI’s system can be vulnerable, putting users’ personal data at risk. Adding to the trouble, people are jailbreaking ChatGPT and using the unrestricted version to produce malware and scams on a scale we haven’t seen before.
Threats to jobs and the potential to disrupt education are a few more problems that are piling up. With brand-new technology, it’s difficult to predict what problems will arise in the future, but unfortunately, we don’t have to look very far. ChatGPT has produced its fair share of challenges for us to deal with in the present.
