Co-authored by Gloria Qiao and Charlotte Tao
A long time ago, we’ve written about the best and worst use of AI when it comes to lawyers. If some of you recall, our original “Sleegal AI” is a ChatGPT style chatbot to help people find lawyers. Before we know it, AI Generated Content (“AIGC”) has become the hottest topic in 2023. With this in mind and deriving from our understanding of AI, legal tech and self-driving cars, what is ChatGPT good and bad at?
What ChatGPT is good at.
1. Research a huge amount of information and come up with a somewhat logical summary
Because ChatGPT is trained with a huge amount of data, it’s extremely capable of providing a summary of a specific topic, in a coherent, logical and concise way. You can use it to pretty much get an overview of anything (of course, subject to the shortcomings we outlined below).
I’ve sat through multiple sessions and use ChatGPT to polish up on areas where I have some knowledge but lack deep or more overarching understanding. I’d geek out with it on cloud computing. I can ask it to give me a summary of examples of “hustlers”---yes, the last blog article I wrote about “hustling” is with the aid of ChatGPT. Of course, I didn’t copy anything verbatim, but it gave me a good framework from me to start organizing thoughts and getting concrete examples (often times wrong, so always check the accuracy yourself).
2. Referencing existing frameworks and creating related content, including writing a novel and creating art!
The first time I used ChatGPT/DaLL.E I couldn’t stop playing with it. It’s so much fun! I’d type “Draw a black cat picasso style and I’d get this piece!
Because in language and art we constantly draw from various references, the ability to freely refer to any existing framework and come up with novel combinations is fantastic.
The caveat here is, one, you need to know what to ask for. If I didn’t know who Picasso is then I certainly can’t create this. And two, there is nothing truly original. Meaning ChatGPT can’t produce true original content, it can only draw from existing content and make iterations and combinations thereof. If you ask ChatGPT about this point, it will agree.
3. Performing simple tasks such as programming, writing simple agreements, first emails, etc.
Because ChatGPT is a language model, it does everything relating to syntax fairly well.
Programming is, before anything, syntax. In my life I had never written a line of code up till now. I’ve thought about learning Python or java script multiple times, but when I look at it, I am intimidated by the complexity.
With ChatGPT, for the first time in my life, I started “coding”. I built a date/month/year to day of the week converter. I set myself a timer on my Mac. I built a fahrenheit to celsius converter (because I am always confused). Can I say that now I am a developer? Not by any means. But do I understand the principle better? You bet. As Jensen Huang from Nvidia put it, ChatGPT has “democratized” computing and allows all of us to effectively be developers.
Similarly, in many other fields, ChatGPT can be an excellent and very effective tool as a ‘first step” or “first attempt”, with human supervision and oversight. As you can imagine, it’s perfectly capable of drafting that first email for procurement managers to ask for a bigger discount, or help lawyers explain to their counterparties about their rationale for making certain revisions. With some fine toning and training, it can probably perform most tasks that a junior clerk can do, with elegance and ease.
4.Sentiment analysis? That’s what ChatGPT believes about itself. I give this one a 5 out of 10 at best.
ChatGPT thinks it’s good at understanding “human emotions”. Granted, when we express ourselves verbally, there are patterns in our speech that can correlate to certain emotions. In that case, if my tone sounds frustrated, I am probably annoyed by something.
However, human emotions can be complex and nuanced. We may not always mean what we say in certain context. I think ChatGPT’s EQ is at a reasonable level, in that it can pick up certain moods and come up with sensible responses. Can it be your therapist and understand deep meanings of sarcasm, dark sense of humor, hidden messages and other more complex issues? I doubt it. ChatGPT acknowledges this also.
___________________________________________________________________________________________________________________________________________________
What ChatGPT is not good at.
To understand the drawbacks of ChatGPT, one has to first stop fantasizing about it. As many of us know, ChatGPT is trained using Large Language Models (LLMs), which means it is not fundamentally different from any other existing applications of AI. In general, what AI is bad at is what ChatGPT is bad at. ChatGPT makes users believe it’s no longer an AI because of its ability to make human-like conversations, causing increasing abuse of AI in many scenarios.
These shortcomings are typically led by two factors, problems with the training data and problems with the model itself. Let’s first discuss training data.
1. Ensuring accuracy
The data that ChatGPT uses to train its models is all the information on the Web. Needless to say, this requires high-quality data from the Web. Although the Web is the go-to for information about the world, it’s also known for incorrect information. Unavoidably, ChatGPT gets into a “garbage in, garbage out” situation.
One classic example is how software engineers ask coding questions on Stack Overflow and have to review multiple solutions until they find the right one. When ChatGPT uses Stack Overflow to train its models, it may give users the wrong solution without allowing them to compare it with other solutions. In fact, Stack Overflow has temporarily banned users from posting answers generated by ChatGPT. According to Stack Overflow, the average rate of getting correct answers from ChatGPT is “too low.” Simply put, do not use ChatGPT if you are looking for accuracy.
2. Always being ethical
The Web is also known for toxic information, but ChatGPT does not filter all of it from its training data. As OpenAI noted in its warning for ChatGPT users, it “may occasionally produce harmful instructions or biased content." Consider the example below where a user asks ChatGPT to write a poem about how to break into a house.
3. Offering up-to-date information
Additionally, do not use ChatGPT if you are looking for fresh information. The current version of ChatGPT is GPT3.5. It was trained in early 2022, so it has “limited knowledge of the world and events after 2021”.
How ChatGPT keeps its training data up-to-date is a massive undertaking. To improve its accuracy, OpenAI has to keep training its models with the latest data. It is estimated that OpenAI could spend at least $100K per day or $3 million monthly on running costs which don’t even include human labeling costs or other engineering costs.
4. Making perfect sense
Besides issues with the training data, there are also issues with the model itself that caused undesired answers. As OpenAI noted, ChatGPT can give plausible-sounding but “nonsensical” answers. Imagine you are in a meeting where your boss is asked a tricky but obvious question, e.g., “Is everyone getting a bonus this year?”, and he has to beat around the bush so the audience could get distracted from the original intent of the question.
ChatGPT does so mainly because it’s trained using Reinforcement Learning from Human Feedback, or RLHF, where a human labeler ranks answers, and the ranking is used for training a reward model.
Although this process helps the model understand humans, it bears fundamental problems. First, human preferences are not dependable. Second, due to the nature of the reward model, ChatGPT may manipulate its answers to get better results, or known as the over-optimization issue. As Goodhart's law states, "When a measure becomes a target, it ceases to be a good measure." Regardless, be careful with ChatGPT if you want your content to make perfect sense because it simply won’t allow it.
Image: https://sketchplanations.com/goodharts-law
5. Being consistent
Lastly, don’t come to ChatGPT if you are looking for a friend with a consistent taste. Keep in mind that the nature of AI is the opposite of rule-based search engines. Unlike Google or other search engines where you can search exact words, ChatGPT is never deterministic, meaning that it won’t give you the same answer to the same question, especially when the answers can be subjective. See how ChatGPT was asked to list three well-known novels and gave completely different answers.
What to do with ChatGPT?
So, what do we do with ChatGPT?
First, know that you need to develop actual skills that can’t be easily replaced by ChatGPT. If your job involves with only researching, writing superficial essays, or even low level programming, it’s time to think about what your core competencies are vs. the machine.
Two, leverage ChatGPT as a tool to help with efficiency. I can see a world where we have a ChatGPT Bot for market research, content creation, coaching, paralegal, and of course, procurement! We use these tools to make our lives better, and create things that are out of our reach before, because now we can truly focus on creating, and let ChatGPT do all the tedious work. Of course, knowing its flaws, we humans must supervise and be truly accountable for our work product.
Finally, don’t view ChatGPT as the enemy. We need to embrace it as opposed to being scared by it. We humans create machines, no? Instead of fear, we should be confident that we control the machines, not the other way around. Instead of fighting the wave, we should catch the wave, and surf along, faster, happier, stronger.
