GIANT AI Guide for Educators: How Text Generative AI Works

Images generated using an AI tool with the prompt, “a person in an empty room with words floating all around”

 

How Text Generative AI Works

Imagine yourself in a room with more than 100,000 words - every single word in the dictionary, floating all around you. Words, like tokens, have filled out the entire space. Words with similar meanings tend to cluster together. For example, you may look up and see words like delicious, yummy, tasty, appetizing, and delectable floating in space close to each other. You may look another direction and see words blue, red, yellow, orange, purple clustered together. 

Imagine, you also have a superpower! You can retrieve words based on the context and instructions given to you. For example, someone may prompt you: “A dog is…”  and you know how to finish the sentence. 

Try this activity with your students

  • Duration: 30 to 45min - a quick activity with follow up classroom discussion

  • Materials: Paper and pencil. Optional: Computers with access to google sheets

  • Recommended for grades 1 to 12

Ask your students to grab a piece of paper to try to fill in the blank with at least 10 different possibilities. If they have access to a computer, they can also go to this document and try to add to the dataset. 

There are many, many, many different words you can use to finish this sentence:

  • A dog is an animal.

  • A dog is a mammal.

  • A dog is a pet.

  • A dog is cute.

  • A dog is a furry animal. 

  • A dog is dangerous and can bite.

  • A dog is everyone’s best friend. 

  • A dog is delicious.

  • A dog is impure.

  • A dog is the only animal that can see white, black, and yellow.

  • A dog is a domesticated descendant of the wolf.

  • A dog is scary.

  • A dog is an enemy to a cat.

  • A dog is playful. 

So, how are you supposed to decide which “completion” will be best received by the person who gave you the prompt? You try to give each “token” (aka word) a probability and then choose one of the options with the highest chance of being relevant. The answer or the “completion” you provide depends on how you have been trained! Imagine you’ve only been trained by a group of dog lovers who gave you this whole dataset with all positive remarks about dogs. Now, if someone prompts you “A dog is…”, you’ll most likely complete the sentence in a very positive way. 

What if a whole group of scientists have trained you with all sorts of scientific facts about dogs? In this case, if someone prompts you “A dog is…”, you’ll most likely complete the sentence with a scientific fact.  

This is how Large Language Models (LLMs), which are the heart of generative AI applications, work. They understand the language and generate text based on probabilities they assign to words, known as tokens, in their “space” - these probabilities get set based on how the models are trained and the “prompt” or the instruction they are given.  

Now, it is time to question whether your class has biased views towards dogs. Review the dataset you have created about dogs. Does your dataset include many positive facts about dogs and portray a picture of the animal as an ideal pet? One may argue that your dataset is biased and does not include diverse opinions about this animal. For example, in islamic cultures, dogs are not allowed to come inside one's home because they are considered “impure”.  

Large Language Models (LLMs) may have biases depending on the datasets they have been trained on - like humans who may have many biases based on our cultures, education, and lived experiences. When working with these models, it is important to know about potential “biases” in their outputs, and make it a habit to analyze the results to spot any content that can be harmful or discriminatory towards certain groups, especially towards marginalized communities.

Interested in learning more about AI and ways to implement in your classroom responsibly?

The GIANT Room provides extensive professional learning workshops for educators to learn more about AI, and strategies to implement AI tools in their curriculum to help their students develop their AI literacy and computational thinking skills. If you are interested to learn more and inquire about these professional learning workshops, please email us here.

In addition, we partner with teachers to publish their students’ projects, writings, and creative work in books, trading cards, and more! Check out GIANT Remixes for samples of these publications and join our waiting list to be part of this amazing free program for teachers.

 
GIANT AI Guide - EduAzadeh Jamalian