ChatGPT is Way Better Than Ask Jeeves
Why new AI language models should blow your mind, but may not dramatically change your educational experience
DALL-E OpenAI Photo picture of a robot reading an ancient text confusedTech Joy
Do you remember the first time you used Google?
I sure do. My friend Spencer showed it to me on an old laptop in my now-demolished first-year residence hall.
It was remarkable how quickly Google found answers and how it effectively sorted the best content. I Googled for hours on end, enjoying the luxury of information at my fingertips, delivered at sweet, sweet, residence hall T1 speeds 1 It felt like magic. 2
I had a similar, but perhaps less enthusiastic enjoyment when I first got to Ask Jeeves questions rather than clicking through the endless, non-searchable Yahoo directory which, was a giant improvement over, well…nothing at all. Later on in the timeline, my mind was also blown the first time I used Napster, watching Ken Jennings get schooled on Jeopardy by IBM’s Watson, and then having that voice-to-action tech brought to my house in the form of Amazon Echo Dot
This past weekend, people from all over the world experienced similar “tech joy” from the the new OpenAI tool ChatGPT. For those who haven’t give it a test, ChatGPT is an AI model that was trained to take in semantic, chat-like, requests, and then generate new (or remixed) outputs based on those requests. Unlike previous iterations of language-based models, ChatGPT has benefited from detailed human tuning, which will hopefully prevent it from becoming a racist jerk-bot. Even if it manages to avoid the worst issues of earlier language models, we should still be cautious: early reports indicate that factual information from ChatGPT seem awesome, but may be gloriously, confidently, wrong.
GPTChat’s specification and reward system also allows it to generate “…followup questions, admit its mistakes, challenge incorrect premises, and reject inappropriate requests.” This creates a far more interactive experience, akin to a conversation, or as some have suggested, a Socratic dialogue with a professor.
It is a remarkable innovation which, naturally, spurred discussions in two spaces where I happen to sit: data science and higher education.
AI and the End of Higher Education
Among the most immediate discussions about ChatGPT was whether it might destroy our higher education model as we know it. After all, a conversational bot that can iteratively walk you through a problem sounds a lot like an inexpensive, scalable instructor. Right?
The death of the academy is a common refrain when technology moves forward.3 It’s been predicted for decades, perhaps centuries. And, it continues to this day. While the modern volleys often hail, from Silicon Valley techno-Utopians and ed-tech VCs, there are many of us in the education policy and finance spaces who openly recognize the challenges to the overall operating model.
Given higher ed’s slow pace of change, substantial fixed costs, and imperfect decision-making, massive exogenous shocks (like COVID-19 or AI) could accelerate those changes. But is the current model of higher education doomed by a highly sophisticated AI model?
My sense (which is biased, limited, and probabilistic) is that no, higher education as we know it isn’t very likely to end or be substantially different over a medium time horizon.
The rationale is this: a liberal arts, technical, or creative education isn’t just about figuring out the answers to well-structured questions, it’s about learning how to ask good questions. Indeed, one of the ironies of the ChatGPT discourse is that well-educated, experienced people are having their minds blown by detailed answers to questions that are useful to them now without considering or remembering an earlier, intellectual tabula rasa.
Put another way: It’s easier for people who are traditionally educated to query “Build me a Python tutorial” and self-learn from there, than it is for someone who isn’t quite sure what they want to learn, or what problems Python might solve in the first place. Furthermore, those who are traditionally educated are more likely to detect logical flaws, inaccuracies, or factual holes. Self-learners and the traditionally educated will benefit tremendously from AI-generated educational pathways, but the rest of the population may still need some guidance.
I don’t want to suggest that these tools aren’t educational groundbreaking or that they might not substantially change the way that we teach and learn. Furthermore, I don’t want to suggest that the patches-on-jackets experience will be the model forever. Higher education demands faster, cheaper, skill-based credentialing options that help educate more students, without the limiting fixed costs of a synchronous, classroom environment. Educators and institutions would be wise to leverage AI for its speed, flexibility, and economies of scale. ChatGPT has so much promise, so let’s figure out the best ways we could use it, without the histrionics about fully upending everything.
Generative Language AI and Changes to Pedagogy
If the assumption above holds and our millennium-old concept of higher education is not fully destroyed, then how will colleges and universities adopt these new technologies into our existing practices? This seems to be a much more interesting question since ChatGPT and it’s improved successors can do a lot of what we ask students to do: write essays, answer questions, generate code, apply for admissions, and much more.
If students can use ChatGPT to generate seemingly original work, then how are we going to teach students to think critically, write convincing arguments or not pass off AI-generated work as their own?
Although I will openly admit to not having perfect answers, allow me to humbly propose a few ways in which educators can utilize modern AI tools while also developing graduates capable of thinking in a complex world. Please note that I am generally not into “futurism” and recognize that predictions are routinely wrong. I am not trying to indicate that this what the world will look like, but instead, presenting some ideas about ways in which generative language bots like ChatGPT could benefit the academic enterprise, without completely tearing it down.
Become AI Adversaries
ChatGPT has been shown to be confidently wrong. This provides an incredible opportunity to turn student end users into fact-checkers. One can imagine an exercise where a professor creates a small series of AI-generated responses, and then asks students, in class, to find ways in which the responses are inaccurate, incomplete, biased, or misrepresented. This is exactly the type of inquiry that took place when Wikipedia first came out, and before it was as widely trusted as it is today.
Create an AI “Write-a-Thon”
ChatGPT can produce workable text. However, the examples (even if correct) are bland and generic. One pedagogical option might be to set up something akin to an extemporaneous speech events at debate tournaments. For example, one might generate an AI response based on a known prompt. Then, students could be give a limited amount of time to write a better, more convincing, more engaging speech or paper.
Encourage Use of AI for Knowledge Discovery and Detecting Gaps in the Literature
AI is powerful and has every opportunity to improve our lives especially in the context of repetitive tasks that can be automated. This, in theory, leaves more time for the deeper creative and analytic work at which humans excel. This is already happening in a variety of industries.
One example that comes to mind is the literature review and knowledge discovery process that must happen before developing a novel research question. To be clear, this is a critical skill that should be taught (and not automated) in graduate programs. Still, I would argue that this is the type of work that could benefit from AI. The systematic Googling and library searches, reading through paper abstracts, chasing earlier work through the citations, and determining places of agreement or disagreement in the existing literature can be pretty repetitive and exhausting.
If AI can reduce the repeitive task of reading legal documents, that isn’t too far from “reading” academic papers and having them rapidly summarized. One might use ChatGPT (or its improved successors) for generating literature review summaries, course reading lists, article graphs, or bibliographies. If AI were able to rapidly synthesize the current state of the literature, it may also, detect or even recommend studies that would fill existing gaps, or surface discrepancies between similar studies that are ripe for reproduction.
I am guessing this idea won’t be popular among scholars who believe that the searching and the reading are part of the discovery process. Nevertheless, we have to ask ourselves whether some work that has previously been viewed as “critical” exists because we didn’t have an easier, more efficient way. If something like this were to be available, then maybe the “disruption” to academic research is actually a way to have less overlap in existing work, more novel research, and expedited timeline from idea to knowledge.
Use Students as Human AI Trainers
One of the best parts of ChatGPT is that it has clearly been tuned with the help of human beings who understand language and context. As AI becomes more accessible to build and deploy, surely one of the best educational use cases is to have students learn to build, train, and validate their own AI models. As much as we worry about it changing education, it is pretty clear that new technologies will precipitate new jobs and industries.
Inconclusive Conclusion
ChatGPT and similar AI models are almost certainly going to change the ways in which we live, work, and study. With better data sets, and improved human tuning, we may enter a world where some of the tasks that we previously took on, can now be automated or semi-automted.
That being said, I am not yet convinced that these AI tools can so dramatically demolish the academic enterprise. This is largely because education is so much more that completing assignments and getting grades. Students will continue to pursue the educational experiences that allow them opportunities to the big, thorny, difficult problems that require more than a mediocre AI-generated essay.
If you believe that education is some sort of home school workbook whereby correct answers to existing questions can be checked off in step wise fashion until some badge is unlocked, then ChatGPT or similar tools can and will be an acceptable substitute. If you believe that education is an exploration of ideas, and method by which you learn how to learn then ChatGPT can be a remarkable discovery tool, without replacing a human-centered educational experience.
Either way, we don’t have to create a false choice between AI and traditional forms of education. Our best bet as educators, technologists, and policy-makers is to find ways to ensure that students are learning how to learn, and thinking about ways in which we can pass off low-value, boring tasks to the machines, while keeping the really engaging, human-problems front of mind.
Example
If social media is any indicator, many of you spent considerable time with ChatGPT geting prompts answered. Here is a personal favorite, just for fun.
Me: write a bluegrass song describing breakfast menu items
AI:
Verse 1:
I woke up this morning, feeling fine
Thinking about what I’d like to dine
I headed down to the kitchen with a smile
To make myself a bluegrass breakfast worth the while
Chorus:
Bluegrass breakfast, full of flavor
Makes my heart sing and my taste buds savor
Bacon sizzling, eggs a-frying
Grits a-boiling, coffee a-pouring
Verse 2:
I started with a stack of pancakes hot and sweet
Topped with butter and a drizzle of maple syrup treat
I fried up some eggs and crispy bacon too
And stirred up a pot of creamy grits, just the thing to do
Chorus:
Bluegrass breakfast, full of flavor
Makes my heart sing and my taste buds savor
Bacon sizzling, eggs a-frying
Grits a-boiling, coffee a-pouring
Verse 3:
I poured myself a mug of steaming hot black coffee
To wash down all the tasty food, that’s the way to be
I sat down at the table, and dug in with delight
A bluegrass breakfast feast, a beautiful sight
Chorus:
Bluegrass breakfast, full of flavor
Makes my heart sing and my taste buds savor
Bacon sizzling, eggs a-frying
Grits a-boiling, coffee a-pouring
A Quick Note on the Photo Above
The photo above was created using DALL-E, another generative AI model from open.ai. It was generaged using the prompt “Picture of a robot reading an ancient text confused.”
For people in my age group, you will recall that getting email and having T1 were real perks of higher education.↩︎
This was in 1999, before Google had fully monetized ads giving rise to SEO, marketing automation, and the awful ad chum bucket we experience on today’s web.↩︎
So too is the “AI will become sentient and destroy us all unless Sarah Connor gets off her ass” narrative.↩︎
Brad Weiner
My research interests include higher education policy, data science, enrollment management, and institutional advancement.