While there has been considerable attention recently about Artificial Intelligence, AI, development in the field is not new. In 1956 Dartmouth college coined the term during a summer research project organized by Dr. John McCarthy.
Since then, “artificial intelligence” has been used in one way or another in education for years. Search engines, personal assistants on phones, assistive technology to increase accessibility, and other technology all use some form of applied artificial intelligence. However, what most people are concerned about is the advances OpenAI technology found in GPT-4 and ChatGPT. To address this, let's briefly look at how ChatGPT works and a few of the issues they pose through their architecture.
CHAT-GPT: How it Works
GPT-3 (the program the runs ChatGPT) used over 3000 HGX A100 servers with over 28,000 GPS to train on over 570 GB of text data to train and assign weights to words. With months of training, it created over 170 billion connections between all the words and added weights representing the importance of each connected relative to the word. This level of computing was very costly, at about $500,000/day and using more electricity than 150,000 people use in a month (23M KWh). The trained Artificial Neural Network then had researchers ‘tweak’ the weights to help rule out odd responses and was trained until the responses were consistent to what the programmers want.
This results in a program that looks for the next -token prediction from a list of rated words as the word that will likely appear next. There is no semantic grounding, set knowledge base of information based on confirmed scientific data, or any criterion of truth other than that of the statistically probability of another work following your prompt given its large training set. For example, when you use CHATGPT you:
- Give it a prompt
- ChatGPT looks at the last word of the prompt and assigns a number to encode it.
- It then multiplies that number by the connections of everything it learned how words are associated with each other (embedded) which creates a 12,000-dimensional matrix
- Attention transformers that identify which words in the prompt should have more attention that others (such as nouns over adverbs)
- “normalize” output to make it seem more like a matrix again
- Feed results forward to another layer of attention transformers (repeat 95 times)
- Produce one word
- ChatGPT repeats steps 2-7 for the rest of the words in the prompt.
At no point does ChatGPT know what the question is.
There is no knowledge-base being consulted.
It just works assign weights to nodes in a matrix representing the probability of a word that is most likely to follow its predecessor and selecting the one with the highest value. These weights are assigned through a training algorithm, that includes a ‘randomness” to ensure that the text appears “fresh”, working with a training set, as well as researchers “tweak” the weights to help create desired results.
It seems clear that one of the desired results is to create a program that appears intelligent regardless of knowledge or being accurate. This might not be too surprising when we remember that ChatGPT is a product of a private company where perception of a successful product is key for profit and increasing stock prices. Moreover, there is a lot of pressure to sell these systems. The current costs of these AI systems are creating a money pit for investors while the product seems to be a solution looking for a problem
What Could Go Wrong?Misinformation and the Lack of Truth
ChatGPT may be designed to present its information in a convincing way. Since the first chatbot, ELIZA, people tend to anthropomorphize and place trust in these software applications. This can be dangerous with a program that can produce misleading statements. While there is a disclaimer to ChatGPT, OPEN AI benefited from the hype while not warning of issues with their product.ChatGPT is capable of generating a considerable amount of nonsense, such as:
“Some experts believe that the act of eating a sock helps the brain to come out of its altered state as a result of meditation.”
While this is noticeably absurd, other statements may not be obvious. ChatGPT has already been documented to fabricate information and to adamantly defend these fabrications. Often these cases are referred to as “hallucinations”, as the chatbot produces responses as though they are correct. Apparently ChatGPT has been so convincing, that already a lawyer has been caught citing cases hallucinated by the program. This was a career ending mistake.
OPENAI recommends that users check the output of ChatGPT. However, not producing accurate information seems to be a serious flaw of the tool. Instead, ChatGPT is producing vast amounts of text with a variable level of “truthiness”. Another term for its output would be fiction or misinformation. Others argue that its output conforms with the technical term coined by Harry Frankford – the term ‘bullshit ’. It is named this in reference to the game where actors deliberately try to convince others of a statement with no regard to its truth.
In small doses misinformation on the web is not a problem, however AI has already produced more text than humans have since the Guttenberg printing press. Each day ChatGPT produces approximately 4.5 billion words a day. This flood of information will make it harder to find accurate and truthful information on the web. This is harmful on many levels, including contributing to undermining democracies. If colleges are going to promote the whole scale adoption of this technology, we do have to consider the increase in misinformation and resources that this will produce. It may be wise to have a more tempered approach.
ChatGPT may be designed to present its information in a convincing way. Since the first chatbot, ELIZA could encourage people to anthropomorphize and place trust in the software application. This can be dangerous when a program that can produce misleading statements. While there is a disclaimer to ChatGPT, OPEN AI benefited from the hype while not warning of issues with their product.
ChatGPT is capable of generating a considerable amount of nonsense, such as:
“Some experts believe that the act of eating a sock helps the brain to come out of its altered state as a result of meditation.”
While this is noticeably absurd, other statements may not be obvious. ChatGPT has already been documented to fabricate information and to adamantly defend these fabrications. Often these cases are referred to as “hallucinations”, as the chatbot produces responses as though they are correct. Apparently ChatGPT has been so convincing, that already a lawyer has been caught citing cases hallucinated by the program. This was a career ending mistake.
OPENAI recommends that users check the output of ChatGPT. However, not producing accurate information seems to be a serious flaw of the tool. Instead, ChatGPT is producing vast amounts of text with a variable level of “truthiness”. Another term for its output would be fiction or misinformation. Others argue that its output conforms with the technical term coined by Harry Frankford – it is ‘bullshit’.
In small doses misinformation on the web is not a problem, however AI has already produced more text than humans have since the Guttenberg printing press. Each day ChatGPT produces approximately 4.5 billion words a day. This flood of information will make it harder to find accurate and truthful information on the web. This is harmful on many levels, including contributing to undermining democracies. If colleges are going to promote the whole scale adoption of this technology, we do have to consider the increase in misinformation and resources that this will produce. It may be wise to have a more tempered approach.
Whenever a new language model like ChatGPT comes out, it gets a lot of hype. However, how should we proceed? We are not faced with the dilemma of promoting the misinformation spread by encouraging it or banning all AI reminiscent of human history in Herbert’ book Dune. Another option includes employing a measured approach where we carefully employ AI to illustrate its flaws and prepare students for the future. This may include how to combat a vast amount of misinformation and should certainly include highlighting the importance of information literacy and research librarians, who are regularly under-utilized by students.
Reference
Berry, D (2018) "Weizenbaum, ELIZA and the End of Human Reason". In Baranovska, Marianna; Höltgen, Stefan (eds.). Hello, I'm Eliza: Fünfzig Jahre Gespräche mit Computern [Hello, I'm Eliza: Fifty Years of Conversations with Computers] (in German) (1st ed.). Berlin: Projekt Verlag. pp. 53–70.
Bobrowsky, M (2023) The Metaverse is Quickly Turning into the Meh-taverse , The Wall Street Journal. Mar 29.
DeGeurin, M (2023) 'Thirsty' AI: Training ChatGPT Required Enough Water to Fill a Nuclear Reactor's Cooling Tower, Study Finds. Gizmodo. May 10.
Dotan, T (2023) Big Tech Struggles to Turn AI Hype Into Profits. The Wall Street Journal. Oct 9.
Ellingrud, K., Sanghvi, S. Dandona, G, Madgavkar, A, Chu, M, White, O and P. Hasebe (2023) Generative AI and the Future of Work in America. McKinsey Global Institute. July 25.
Estreich, G (2019) Fables and Futures: Biotechnology, Disability, and the Stories We Tell Ourselves, Cambridge, MA: MIT Press.
Frankfurt, H. G. (1988) “On Bullshit.” The Importance of What We Care About: Philosophical Essays, pp. 117–133. Cambridge: Cambridge University Press (originally published in the Raritan Quarterly Review, 6(2): 81–100, 1986; reprinted as a book in 2005 by Princeton University Press).
Gebru, T, Morgenstern J, Vecchione, B, Wortman Vaughan, J, Wallack, H, Daume, H, and K. Crawford (2022) Excerpt from Datasheets for Datasets. In Ethics of Data and Analytics: Concepts and Cases. Martin, K. New York: Auerbach Publications
Hicks, M (2023) No, ChatGPT Can’t be Your New Research Assistant. Chronicle of Higher Education. Ag 23.
Hetzner, C (2023) ChatGPT moves to cash in on its fame as OpenAI launches plan to charge monthly fee for premium subscribers. Fortune. January 23.
Hopfield, J. J. (1982). "Neural networks and physical systems with emergent collective computational abilities". Proceedings of the National Academy of Sciences. 79 (8): 2554–2558
Knight, W (2022) ChatGPT’s Most Charming Trick Is Also Its Biggest Flaw. Wired. Dec 7.
Korhonen, P (2023) AI is a solution in search of a problem. UXDesign. Feb 2
Ludvigsen (2023) ChatGPT’s Electricity Consumption, Part II Medium. Mar 5
Magesh, V (2024) AI on Trial: Legal Models Hallucinate in 1 out of 6 (or More) Benchmarking Queries, HAI. Stanford University. May 23.
McDermott, D (1976) Artificial Intelligence Meets Natural Stupidity. SIGART Newsletter, 57:4-9
McMurtrie, B (2023) Teaching: Want your students to be skeptical of ChatGPT? Chronical of Higher Education. Sep 21
Merken, S (2023) Lawyer who cited cases concocted by AI asks judge to spare sanctions. Reuters. June 8.
Mollick, E (2023) Centaurs and Cyborgs on the Jagged Frontier. One Useful Thing, Sep 16.
Morgan, S. (2018) Fake news, disinformation, manipulation and online tactics to undermine democracy, Journal of Cyber Policy, 3:1, 39-43.
Newell, A. (1963). A Guide to the General Problem-Solver Program GPS-2-2. RAND Corporation, Santa Monica, California. Technical Report No. RM-3337-PR.
Newell, A.; Shaw, J.C.; Simon, H.A. (1959). Report on a general problem-solving program. Proceedings of the International Conference on Information Processing. pp. 256–264.
Norvig, Peter (1992), ELIZA: Paradigms of Artificial Intelligence Programming, San Francisco: Morgan Kaufmann Publishers
Olson, P. (2023) There’s No Such Thing as Artificial Intelligence. The Washington Post. Mar 26.
Ouyang, L, et al (2022) Training language models to follow instructions with human feedback. OpenAI
Pawar, S (2023) ChatGPT Costs a Whopping $700,000/Day to Operate, Says Research Analytics Drift. April 25.
Roose, K. (2023) Don’t ban ChatGPT in Schools. Teach with it. The New York Times. Jan. 12
Ropek, L (2023) So Far, AI Is a Money Pit That Isn't Paying Off Gizmodo. Oct 10.
Vincent, J (2021) OpenAI’s text-generating system GPT-3 is now spewing out 4.5 billion words a day. The Verge. Mar 29.
Valdez, E (2023) How much is invested in Artificial Intelligence Exoinsight. Mar 23.
Wolfram, S. (2023) What is ChatGPT Doing … and Why Does It Work? Stephen Wolfram Writings.