AI in Education – Chat-GPT and a Serious Concern

While there has been considerable attention recently about Artificial Intelligence, AI, development in the field is not new. In 1956 Dartmouth college coined the term during a summer research project organized by Dr. John McCarthy.


Since then, “artificial intelligence” has been used in one way or another in education for years. Search engines, personal assistants on phones, assistive technology to increase accessibility, and other technology all use some form of applied artificial intelligence. However, what most people are concerned about is the advances OpenAI technology found in GPT-4 and ChatGPT. To address this, let's briefly look at how ChatGPT works and a few of the issues they pose through their architecture.

CHAT-GPT: How it Works

  Unlike Deep Blue, that uses “brute force” to check all possible outcomes to determine an optimal answer, ChatGPT uses an artificial neural network that trains on data sets from billions of webpages from the internet, with trillions of words, and selects a word that statistically will follow its predecessor. Artificial Neural Networks have been around for a long time, and they are excellent at pattern matching. In the case of ChatGPT, the program weights all the words that have been connected to the input and selects the word that has the highest probability the weights have assigned to it. The program then looks for the next word the same way.

GPT-3 (the program the runs ChatGPT) used over 3000 HGX A100 servers with over 28,000 GPS to train on over 570 GB of text data to train and assign weights to words. With months of training, it created over 170 billion connections between all the words and added weights representing the importance of each connected relative to the word. This level of computing was very costly, at about $500,000/day and using more electricity than 150,000 people use in a month (23M KWh). The trained Artificial Neural Network then had researchers ‘tweak’ the weights to help rule out odd responses and was trained until the responses were consistent to what the programmers want.

This results in a program that looks for the next -token prediction from a list of rated words as the word that will likely appear next. There is no semantic grounding, set knowledge base of information based on confirmed scientific data, or any criterion of truth other than that of the statistically probability of another work following your prompt given its large training set. For example, when you use CHATGPT you:

1. Give it a prompt
2. ChatGPT looks at the last word of the prompt and assigns a number to encode it.
3. It then multiplies that number by the connections of everything it learned how words are associated with each other (embedded) which creates a 12,000-dimensional matrix
4. Attention transformers that identify which words in the prompt should have more attention that others (such as nouns over adverbs)
5. “normalize” output to make it seem more like a matrix again
6. Feed results forward to another layer of attention transformers (repeat 95 times)
7. Produce one word
8. ChatGPT repeats steps 2-7 for the rest of the words in the prompt.

At no point does ChatGPT know what the question is.

There is no knowledge-base being consulted.


It just works assign weights to nodes in a matrix representing the probability of a word that is most likely to follow its predecessor and selecting the one with the highest value. These weights are assigned through a training algorithm, that includes a ‘randomness” to ensure that the text appears “fresh”, working with a training set, as well as researchers “tweak” the weights to help create desired results.

It seems clear that one of the desired results is to create a program that appears intelligent regardless of knowledge or being accurate. This might not be too surprising when we remember that ChatGPT is a product of a private company where perception of a successful product is key for profit and increasing stock prices. Moreover, there is a lot of pressure to sell these systems. The current costs of these AI systems are creating a money pit for investors while the product seems to be a solution looking for a problem

What Could Go Wrong?

Misinformation and the Lack of Truth

ChatGPT may be designed to present its information in a convincing way. Since the first chatbot, ELIZA, people tend to anthropomorphize and place trust in these software applications. This can be dangerous with a program that can produce misleading statements. While there is a disclaimer to ChatGPT, OPEN AI benefited from the hype while not warning of issues with their product.

ChatGPT is capable of generating a considerable amount of nonsense, such as:

“Some experts believe that the act of eating a sock helps the brain to come out of its altered state as a result of meditation.”

While this is noticeably absurd, other statements may not be obvious. ChatGPT has already been documented to fabricate information and to adamantly defend these fabrications. Often these cases are referred to as “hallucinations”, as the chatbot produces responses as though they are correct. Apparently ChatGPT has been so convincing, that already a lawyer has been caught citing cases hallucinated by the program. This was a career ending mistake.

OPENAI recommends that users check the output of ChatGPT. However, not producing accurate information seems to be a serious flaw of the tool. Instead, ChatGPT is producing vast amounts of text with a variable level of “truthiness”. Another term for its output would be fiction or misinformation. Others argue that its output conforms with the technical term coined by Harry Frankford – the term ‘bullshit ’. It is named this in reference to the game where actors deliberately try to convince others of a statement with no regard to its truth.

In small doses misinformation on the web is not a problem, however AI has already produced more text than humans have since the Guttenberg printing press. Each day ChatGPT produces approximately 4.5 billion words a day. This flood of information will make it harder to find accurate and truthful information on the web. This is harmful on many levels, including contributing to undermining democracies. If colleges are going to promote the whole scale adoption of this technology, we do have to consider the increase in misinformation and resources that this will produce. It may be wise to have a more tempered approach.

Whenever a new language model like ChatGPT comes out, it gets a lot of hype. However, how should we proceed? We are not faced with the dilemma of promoting the misinformation spread by encouraging it or banning all AI reminiscent of human history in Herbert’s book Dune. Another option includes employing a measured approach where we carefully employ AI to illustrate its flaws and prepare students for the future. This may include how to combat a vast amount of misinformation and should certainly include highlighting the importance of information literacy and research librarians, who are regularly under-utilized by students.

ChatGPT may be designed to present its information in a convincing way. Since the first chatbot, ELIZA could encourage people to anthropomorphize and place trust in the software application. This can be dangerous when a program that can produce misleading statements. While there is a disclaimer to ChatGPT, OPEN AI benefited from the hype while not warning of issues with their product.

ChatGPT is capable of generating a considerable amount of nonsense, such as:

“Some experts believe that the act of eating a sock helps the brain to come out of its altered state as a result of meditation.”

While this is noticeably absurd, other statements may not be obvious. ChatGPT has already been documented to fabricate information and to adamantly defend these fabrications. Often these cases are referred to as “hallucinations”, as the chatbot produces responses as though they are correct. Apparently ChatGPT has been so convincing, that already a lawyer has been caught citing cases hallucinated by the program. This was a career ending mistake.

OPENAI recommends that users check the output of ChatGPT.  However, not producing accurate information seems to be a serious flaw of the tool. Instead, ChatGPT is producing vast amounts of text with a variable level of “truthiness”.  Another term for its output would be fiction or misinformation. Others argue that its output conforms with the technical term coined by Harry Frankford – it is  ‘bullshit’.  

In small doses misinformation on the web is not a problem, however AI has already produced more text than humans have since the Guttenberg printing press. Each day ChatGPT produces approximately 4.5 billion words a day. This flood of information will make it harder to find accurate and truthful information on the web. This is harmful on many levels, including contributing to undermining democracies. If colleges are going to promote the whole scale adoption of this technology, we do have to consider the increase in misinformation and resources that this will produce. It may be wise to have a more tempered approach.

Whenever a new language model like ChatGPT comes out, it gets a lot of hype. However, how should we proceed? We are not faced with the dilemma of promoting the misinformation spread by encouraging it or banning all AI reminiscent of human history in Herbert’ book Dune. Another option includes employing a measured approach where we carefully employ AI to illustrate its flaws and prepare students for the future. This may include how to combat a vast amount of misinformation and should certainly include highlighting the importance of information literacy and research librarians, who are regularly under-utilized by students.

Reference  

Berry, D (2018) "Weizenbaum, ELIZA and the End of Human Reason". In Baranovska, Marianna; Höltgen, Stefan (eds.). Hello, I'm Eliza: Fünfzig Jahre Gespräche mit Computern [Hello, I'm Eliza: Fifty Years of Conversations with Computers] (in German) (1st ed.). Berlin: Projekt Verlag. pp. 53–70.

Bobrowsky, M (2023) The Metaverse is Quickly Turning into the Meh-taverse , The Wall Street Journal. Mar 29.

Colom R, Karama S, Jung RE, Haier RJ. (2010) Human intelligence and brain networks. Dialogues Clin Neurosci. 12(4):489-501

DeGeurin, M (2023) 'Thirsty' AI: Training ChatGPT Required Enough Water to Fill a Nuclear Reactor's Cooling Tower, Study Finds. Gizmodo. May 10.

Dotan, T (2023) Big Tech Struggles to Turn AI Hype Into Profits. The Wall Street Journal. Oct 9.

Ellingrud, K., Sanghvi, S. Dandona, G, Madgavkar, A, Chu, M, White, O and P. Hasebe (2023) Generative AI and the Future of Work in America. McKinsey Global Institute. July 25.

Estreich, G (2019) Fables and Futures: Biotechnology, Disability, and the Stories We Tell Ourselves, Cambridge, MA: MIT Press.

Frankfurt, H. G. (1988) “On Bullshit.” The Importance of What We Care About: Philosophical Essays, pp. 117–133. Cambridge: Cambridge University Press (originally published in the Raritan Quarterly Review, 6(2): 81–100, 1986; reprinted as a book in 2005 by Princeton University Press).

Gebru, T, Morgenstern J, Vecchione, B, Wortman Vaughan, J, Wallack, H, Daume, H, and K. Crawford (2022) Excerpt from Datasheets for Datasets. In Ethics of Data and Analytics: Concepts and Cases. Martin, K. New York: Auerbach Publications

Hicks, M (2023) No, ChatGPT Can’t be Your New Research Assistant. Chronicle of Higher Education. Ag 23.

Hetzner, C (2023) ChatGPT moves to cash in on its fame as OpenAI launches plan to charge monthly fee for premium subscribers. Fortune. January 23.


Hopfield, J. J. (1982). "Neural networks and physical systems with emergent collective computational abilities". Proceedings of the National Academy of Sciences. 79 (8): 2554–2558

Knight, W (2022) ChatGPT’s Most Charming Trick Is Also Its Biggest Flaw. Wired. Dec 7.

Korhonen, P (2023) AI is a solution in search of a problem. UXDesign. Feb 2

Ludvigsen (2023) ChatGPT’s Electricity Consumption, Part II Medium. Mar 5

Magesh, V (2024) AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries, HAI. Stanford University. May 23.

McDermott, D (1976) Artificial Intelligence Meets Natural Stupidity. SIGART Newsletter, 57:4-9

McMurtrie, B (2023) Teaching: Want your students to be skeptical of ChatGPT? Chronical of Higher Education. Sep 21

Merken, S (2023) Lawyer who cited cases concocted by AI asks judge to spare sanctions. Reuters. June 8.

Mollick, E (2023) Centaurs and Cyborgs on the Jagged Frontier. One Useful Thing, Sep 16.

Morgan, S. (2018) Fake news, disinformation, manipulation and online tactics to undermine democracy, Journal of Cyber Policy, 3:1, 39-43.

Newell, A. (1963). A Guide to the General Problem-Solver Program GPS-2-2. RAND Corporation, Santa Monica, California. Technical Report No. RM-3337-PR.

Newell, A.; Shaw, J.C.; Simon, H.A. (1959). Report on a general problem-solving program. Proceedings of the International Conference on Information Processing. pp. 256–264.

Norvig, Peter (1992), ELIZA: Paradigms of Artificial Intelligence Programming, San Francisco: Morgan Kaufmann Publishers

Olson, P. (2023) There’s No Such Thing as Artificial Intelligence. The Washington Post. Mar 26.

Ouyang, L, et al (2022) Training language models to follow instructions with human feedback. OpenAI

Pawar, S (2023) ChatGPT Costs a Whopping $700,000/Day to Operate, Says Research Analytics Drift. April 25.

Roose, K. (2023) Don’t ban ChatGPT in Schools. Teach with it. The New York Times. Jan. 12

Ropek, L (2023) So Far, AI Is a Money Pit That Isn't Paying Off Gizmodo. Oct 10.

Vincent, J (2021) OpenAI’s text-generating system GPT-3 is now spewing out 4.5 billion words a day. The Verge. Mar 29.

Valdez, E (2023) How much is invested in Artificial Intelligence Exoinsight. Mar 23.

Wolfram, S. (2023) What is ChatGPT Doing … and Why Does It Work? Stephen Wolfram Writings.

Cheating the Benefits of Cheating with AI

With the popularity and extraordinary, and dubious, claims of the ability of AI (Narayanan & Kapoor, 2024), a heightened concern about students cheating has occurred. Naturally, proctoring and anti-plagiarism companies benefit from this fear and the sales of their ‘solutions’ increase.  The problem with this policing solution is that beyond a point, teachers get into an ‘arms race’ between AI and AI-detection software where the real loser is the institution footing the bill.  One might think that the companies are happy to ‘stir the pot’ to increase anxiety for the benefit of their shareholders. Meanwhile, research has indicated that the level of academic dishonestly has not changed with the prevalence of AI (Lee, et al, 2024) Yet the focus on student policing places instructors in an adversarial relationship with the students, instead of an instructive one. 

 

A Better Solution

 

Low-stakes assignments have the added benefit of deterring academic dishonesty.  Because there is less risk involved in the assignment, there is less incentive to cheat, which could bring about sever consequences.  Why take the risk, when there is so little to gain?

 

Low-stakes assignments, such as threaded assignments, thwart using AI that quickly generates content.  The low-stakes assignments act as scaffold that requires further reflection and meta-cognitive skills that are not easily replicated by the stochastic language models modern AI uses. This will force the would-be culprit to reflect and expend so much energy producing something to submit that it isn’t worth the effort to get whatever the AI model can produce.

 

When assignments:

•  offer less stress, 
• supply tools for the students to succeed, 
• clearly express expectations, and 
• encourage the student to take control of their learning, 

the students begin to see the value in what they are learning.  This dissuades the learner from cutting corners instead of easily going through the process.  Not only will this lower academic dishonesty in your classes, low stakes assignments will encourage more of your students to become engaged active learners.

 

Low-stakes assignments engage the students in the learning process.  The best way to eliminate academic dishonestly is to remove the incentive to cheat and be up-front about the rules.  Students are often ‘cheating’ because often they have not received adequate instruction and expectations (Waltzer, Bareket-Shavit & Dahl, 2023).  To solve this, explicitly state the acceptable level of AI usage and you will curb the level of unintended violations to your academic expectations. A group of scaffolded low stakes assignments, often scaffolded to create a large assignment, then undermines the pay-off from cheating.

References

Futterman, K (2024) Zeitgeist 6.0: Results of the Campus sixth-annual student-body survey. The Middlebury Campus. Dec 13.

Lee, V., Pope, D., Miles, S, and R. Zarate (2024) Cheating in the age of generative AI: A high school survey study of cheating behaviors before and after the release of ChatGPT. Computers and Education: Artificial Intelligence. Vol 7, December.

Losey, R. (2024) Stochastic Parrots: How to tell if something was written by an AI or a human? 

Mintz, S (2023) 10 Ways to Prevent Cheating. Inside Higher Ed. February 16.

Mollick, E (2023) Centaurs and Cyborgs on the Jagged Frontier: I think we have an answer on whether AIs will reshape work. One Useful Thing. Sep. 16.

Narayanan, A & S. Kapoor (2024) AI Snake Oil: What artificial Intelligence can do, what it can’t, and how to tell the difference. Princeton University Press: Princeton, NJ.

Waltzer, T., Bareket-Shavit, C., & Dahl, A. (2023). Teaching the What, Why, and How of Academic Integrity: Naturalistic Evidence from College Classrooms. Journal of College and Character, 24(3), 261–284.

Wehlburg, K (2021) Assessment design that supports authentic learning (and discourages cheating) Times Higher Education. Nov 24