Revisiting the Role Comprehension Plays in Preparing Students for Standardized Tests

Apr 23, 2024 | English Language Arts Teaching and Learning

By Dr. Cheryl Sandora 

It can be helpful when students understand that writing for a standardized test is a specific type of writing and not the only type of writing we expect of them. Previously we shared ideas related to teaching test writing as a genre and this article explores a critical and often neglected first step in that process – supporting students in understanding both the prompt and the text it is related to. 
It’s that time of year, again. No, not Opening Day, but standardized testing day. For weeks, teachers have been preparing students for the challenges of standardized testing. Black and brown students, students living in poverty, and emergent multilingual students are more likely than their white peers to receive a steady diet of low-level instruction meant to singularly prepare them for taking the state exam (Booher-Jennings, 2005; Davis & Vehabovic, 2017; Dooley & Assaf, 2009; Valenzuela, 2000). Research has been clear that instruction rooted in low-level, test-centric practices does not help students build the reading and writing skills they need to be successful outside of our classrooms (Santelises & Dabrowski, 2015; Matsumura et al., 2006). However, we can still prepare our students to be successful on exams, especially the writing porton of the test, while engaging them in high-level thinking and reasoning work. In a previous article, we talked about the importance of teaching test writing as a genre, as we want students to understand that writing for the standardized test is a specific type of writing and not the only type of writing we expect of them. To support students in this style of writing, typically, teachers introduce them to various strategies that target the writing portion of the standardized tests. These strategies and approaches often address the overall structure of the essay and provide a general idea of how to respond to the prompt. Although strategies that target the overall structure may be helpful once students begin responding to the prompt, supporting them in understanding both the prompt and the text is a critical first step, and it is this critical first step that is often neglected.

How Can We Support Students with Understanding Both the Prompt and the Text?

For comprehension to occur, students must build an understanding of what they read. This process involves “figuring out what text information we need to pay attention to and connecting that to other information. . .  a learner must actively deal with the information in a text in order to make sense of it” (Beck et al. 6). When considering this perspective in relation to preparing students for standardized tests, it’s important to consider how we can support students to actively deal with and make sense of the information in both the text and the prompt before moving to the structure of the essay. It’s useful for teachers to remind students of the habit of taking stock of what one knows about the major understandings in a text even if the test doesn’t ask about those specific ideas. Although it sounds simple, students often struggle with identifying big ideas in a text. Younger students, in particular, tend to gravitate toward interesting bits of information that are not relevant to the big ideas, so it’s important to work with students on differentiating between interesting and important. However, because students are not permitted to use pre-planned organizers, teachers should consider designing approaches that target the big ideas but are simple enough that students can transfer the support or scaffold when working individually. One approach is to share tips that support comprehension, model for students the process of using those tips to dissect a prompt and identify the big ideas in the text, and then provide students with opportunities to apply learning with a partner or small group before practicing independently

Example from the Field 

One of our partner schools has implemented this approach and has had her students engaging in the process for several weeks. Earlier in the week, students were working in small groups, chunking, annotating, and working to identify the main idea.

As the teacher was circulating, she overheard a student mention what she thought was a big idea and another student responded, “Wait, is that a big idea, or just a fun fact?”

Although the teacher was excited that the student recognized the importance of knowing the difference between the two, she wanted to make sure the students understood, so she asked, “And why is it important for us to know the difference? How can we use that information when responding to the prompt?”

The student responded, “‘cause fun facts are details we like, but that doesn’t mean they’re big ideas.” The teacher asked again, “And how will that help when you’re responding to the prompt?” “‘cause we need big ideas for our claim, evidence, and explanation.”

How Might the Process Play Out in a Classroom?

There are various ways this method may be implemented, but below is one way that you might put the approach into practice.

STEP 1: Begin by introducing students to tips that support them to comprehend the prompt.

Carefully read the entire prompt twice. (Students tend to skim questions and often miss key information.)

Highlight the key words in the prompt that refer to what you are being asked to answer (e.g., main idea, theme, conflict, character traits). Then, think about the meaning of each of those words and the role they play in a text.

STEP 2: Move to tips that support students to comprehend the text.

Read, chunk, and annotate the text.
Identify the main idea in each chunk (Have students guide their thinking with questions such as: What did I
learn in this section? How does this information connect to what I have already learned? What are the big ideas in this section? How do those ideas fit with what I’ve already read? Was anything I read confusing?

If you can’t identify the main idea or were confused by any parts of the section, reread that section and revisit your questions. (Students often give up when confused, so it’s important to remind them that rereading sections is helpful.)

Highlight information in each chunk that reflects the “ask” of the prompt.

Think about
• how the informationhelps to develop a claim.
• provides evidence for your claim.
• allows you to develop an explanation for the connection between your evidence and your claim.

Pay attention to headings and labels. They may give clues to the main ideas.

STEP 3: Model the process for students.

Share the passage “Thursday is Market Day” and the prompt with the students.

Dissecting the Prompt

Begin by reading the prompt: In the passage, a mother and daughter travel a long way to visit a market. Write an essay analyzing why the market is important to Elena and her family. Use evidence from the passage to support your response.  

Next, Think Aloud for Students

Okay, so let me reread the prompt to make sure I didn’t miss any important words the first time I read it. (reread prompt). Now, I need to highlight the key words that let me know what I’m supposed to do and then think about these key words in relation to a text. The first sentence seems to give a brief overview of what the story is about – the mother and daughter going to a market. In the second sentence, I’m going to highlight the word analyze because that’s what I’m being asked to do. I know the word analyze means to examine something closely, so that’s what I’ll be doing. Now, let me see what I’m being asked to analyze. Okay, I have to analyze why the market is important to Elena and her family. So, when I’m reading, I have to understand the big ideas around the importance of this market. Because the last sentence asks me to find evidence from the passage, I’m thinking some of those big ideas may serve as evidence.

Understanding the Text

Okay, now that I understand the prompt, I’m going to begin reading the text, and I will stop and annotate when I’ve come across main ideas and supporting details, paying special attention to the information needed to answer the prompt. I need to remember that supporting details are not just some interesting pieces of information. They have to reflect the big ideas in the section I’m reading.

  • (Read paragraphs 1-3 and stop.) So, I stopped here because we are introduced to the main characters and the setting. I’m going to list the characters and what I’ve learned about them at the side (Elena – 10, Rosita – family llama, Elena’s parents and brother.) I also learned where the story takes place, so I’m going to add “Setting – Peru in South America” at the side with my characters. I’m also remembering the prompt that asks about the importance of the market, and in the 2nd paragraph I learn that it’s Thursday, so the family is getting ready to go to the market, so that’s an important detail. There are other details included in these first three paragraphs, like that the llama has a fluffy neck. Although I like fluffy animals, those kinds of details aren’t important, so I’m not going to include that one.
  • (Read paragraph 4-6 and stop.) I stopped after the next 3 paragraphs because there was a lot of important information that connected to what I already learned. I learned that most people grew potatoes and herded llamas, and I learned more about the market. I learned that it’s in another town (Chupaca), and that people go there to sell their potatoes and blankets to make money. They use this money to buy what they need. This information is important because it connects to the prompt about the importance of the market, so we know one of the reasons the market is important is because it’s where families make money to get what they need. The last sentence mentions that Elena likes to see her friends at the market, so that’s also information that connects to the prompt. The market kind of serves as a social setting for her. Both of these details can serve as evidence when I respond to the prompt, so I’m going to underline them.
  • (Read paragraphs 7-10 and stop) I stopped here because I have more information about the family as they get ready to go to the market and the trip itself. The big idea seems to be that they are carrying quite a load of potatoes and blankets and that the market is pretty far away. Although I was able to pull out the big idea, I don’t think any of that information will be helpful in responding to the prompt. (This is an example of why it’s important to identify big ideas, even if they are not helpful in responding to the prompt. In analyzing sample essays for a partner district, one pattern that emerged was that students included as evidence big ideas that were not relevant to the ask of the prompt.)
STEP 4: Have students continue the process in small groups.

Divide students into groups of 3-4 and have them continue to read the passage, chunk, and annotate. If students are new to the process, provide stopping points and questions for them. More experienced students will be able to identify stopping points, ask unprompted questions, and work together to identify big ideas.

 

STEP 5: Transfer learning to independent writing.

After working with students in small groups on several sample texts and prompts, engage students in the same process independently.

Because standardized tests are an integral part of our education system, it’s important for teachers to provide students with approaches that support them to successfully engage in these tests. Although there are many strategies in place that target the structure of the response, supporting students in understanding the prompt and the text is often overlooked. Designing approaches that address comprehension of both are critical to the work. 

REFERENCES

  • Booher-Jennings, J. (2005). Below the bubble: “Educational triage” and the Texas accountability system.American Educational Research Journal, 42(2), 231–268. https://doi.org/10.3102/00028312042002231
  • Davis, D. S., & Vehabovic, N. (2017). The dangers of test preparation: What students learn (and don’t learn) about reading comprehension from test‐centric literacy instruction. The Reading Teacher, 71(5), 579-588.
  • Dooley, C.M., & Assaf, L.C. (2009). Contexts matter: Two teachers’ language arts instruction in this high-stakes era. Journal of Literacy Research,41(3), 354–391. https://doi.org/10.1080/10862960903133743
  • Matsumura et al. (2006). Measuring reading comprehension and mathematics instruction in urban middle schools: A pilot study of the instructional quality assessment. CSE Technical Report 681. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
  • Santelises, S. B., & Dabrowski, J. (2015). Checking in: Do classroom assignments reflect today’s higher standards? K-12 practice. Equity in motion series. Education Trust.
  • Valenzuela, A. (2000). The significance of the TAAS test for Mexican immigrant and Mexican American adolescents: A case study.Hispanic Journal of Behavioral Sciences, 22(4), 524–539. https://doi.org/10.1177/0739986300224009
    Browse our ELA materials at ShopIFL.com

    IFL Recommends for April 2024

    Apr 9, 2024 | IFL Recommends

    This month’s recommendation includes Soup Club, a soup recipe book and so much more that resulted from a community supporting a cancer fighter, and an NPR article about Hand in Hand, one of the only bilingual Arabic-Hebrew schools in Israel.

    Kristin Klingensmith smiling and posing in front of trees

    Kristin Klingensmith

    Mathematics Fellow

    Kristin says, “Cooking for the people I care about is one of my favorite things to do, and soups are some of my favorite things to make, so when I heard about this book, it was a must get!  The author of this cookbook is Caroline Wright, and the book itself grew out of a story of community during Caroline’s fight with cancer.

    At the time, friends were always asking how they could help, and “soup” was Caroline’s response. Her community delivered, making sure that Caroline had soup on her doorstep every day. She made it through and is incredibly appreciative of her community and soup. In honor of the love and community that she experienced, she launched a monthly soup club to bring her own unique, wholesome soup dishes to her friends’ porches as a token of gratitude. As word of Caroline’s club grew, so did her passion and creative spirit, and before long, she was creating a sizable community of soup lovers who were motivated by her tale.

    Soup Club is unlike any other soup book. It not only contains recipes from across cultures and backgrounds but also artwork, photography, and haiku from her members, which come together to tell a moving story of community and love. Here are just a few of the soups included in this book:

    • Catalan Chickpea Stew with Spinach
    • Jamaican Pumpkin and Red Pea Soup
    • Split Pea Soup with Roasted Kale
    • West African Vegetable Stew

    I highly recommend you try them all!”

    Soup Club
    Caroline Wright

    “Caroline Wright’s inspiring story that began with cancer and ended with community compelled me to make soup for others. What better way to thank people for their kindness. And the recipes are thoughtfully written and work well
    – Deborah Madison, chef and food journalist

    Learn more about this cookbook

     

    soup club book

    Glenn Nolly posing in front of a field

    Glenn Nolly

    Leadership Fellow

    Glenn says, “This article is very similar to our shared experience in the United States of inequality and the transformative power of working together to improve education options for students.”

     

    A school in Jerusalem brings Arab and Jewish kids together to boost understanding
    Eleanor Beardsley, Author

    “We practice empathy and understanding and we practice sharing our feelings and our identities. Not separately, but together.”
    – Efrat Meyer, the Jerusalem school’s principal

     

    Read or listen here

    Teacher Daniel De Shalit writes in Hebrew and Arabic during a class at Hand in Hand.
    Ayman Oghanna for NPR

    We Tried It and It WORKED: Stories from our partnering math teachers

    Mar 19, 2024 | Mathematics Teaching and Learning

    By Laurie Speranzo, Lilia Garcia, Carol Jones, Sara Trail, Dina Conzone & Denise Collier 

    The Institute for Learning works in collaboration with educators to use insights from educational research in their local contexts through the use of professional learning opportunities and materials. After all, it is educators who are doing the work in schools every day! Teachers try new pedagogical moves and get to witness the impact on student understanding and student agency in the classroom. Administrators support teachers and students by encouraging the implementation of research-based practices that lead to deep learning. Read about what teachers have tried recently and how administrators can foster environments of best practice.

    CASE STORY 1: STUDENTS OWNING LEARNING OF THE MATH 

    My name is Lilia Garcia. I have been in education for the last 17 years. Prior to working with the IFL, I felt isolated with limited resources and support. Previous training that was provided was redundant and, at times, not grade-level appropriate.

    Use and Connect Respresentations

    Now, working with the IFL, I have expanded my network with teachers from other districts that have helped guide me into venturing off into the higher-level methods of thinking. The work has provided me with many resources and ideas that I have implemented in my classroom and found them to be very effective. For example, the use of different representations (Lesh, Post, and Behr, 1987). These provide opportunities for students to show mastery of the concept in different ways. I used to underestimate my students. Now, they produce pictures, graphs, and explanations that impress my peers and myself. They take ownership of their learning.

    Another way that my students show ownership of their learning is the opportunity for them to create charts. In the past, I used to create them myself which increased my workload. Now I let the students do it. They are proud of their products. The posters became proof that learning is taking place in my classroom.

    Student posters

    Viewing  Lilia’s classroom with an equity lens:

    By encouraging students to use multiple representations, all students are able to represent their thinking in the way that makes sense to them; they are viewed as mathematicians whose lived math experiences and ways of understanding are seen as assets. When students are asked to create classroom charts, their voice is honored and they are honored as knowers and doers of math.

    CASE STORY 2: IMPLEMENTING HIGH-LEVEL TASKS

    We know that in order for students to be able to leverage the representations that Lilia referenced, students need to be presented with high level tasks. In the City of York, teachers have been focused on the importance of the task the students receive and Dina Conzone shares the work of implementing high-level tasks.

    I have been implementing more and more high-level tasks, like the IFL task Scuba Math. I have found that these are a great way to entice students, to elicit their thinking, and to honor and see what their ways of thinking are. It’s the design of the high-level tasks that provide real-world and mathematical problems that can be solved in many ways. With the increased use of high-level tasks, I have seen an increasing number of students showing their own work and making connections between each other’s solution methods. Students are seeking patterns and explaining the mathematical reasoning behind the different ways of thinking and solving. In the Scuba Math task, the students who used tables and graphs to solve the problem made connections to what justifies using the substitution method that other students used.

    Student Work on Scuba Task

    I have also seen that engaging students in high-level tasks and providing time for students to generate and discuss their solution paths has strengthened the community of problem solvers in my classes.

    Viewing Dina’s classroom with an equity lens:

     The QUASAR project, a pivotal research study in the movement towards conceptual mathematics instruction, lays out that a high level of student learning is only possible from a high-level task; low-level tasks (such as procedural worksheets) only yield a low level of learning. In Dina’s class, the same high-level task was given to EVERY student. Every student had the opportunity to share their solution path with equal validity (because each of them is a valid pathway to solving systems) and every student was afforded the opportunity to engage in productive mathematical discussions. Every student was an honored mathematician.

    CASE STORY 3 and 4: FOSTERING PRODUCTIVE STRUGGLE

    In Morgan County, TN, teachers have been studying ways to support students as they engage in productive struggle around a high-level math task. Sara Trail, a middle school math teacher, shares what happened in her class.

    One of my most enjoyable teaching experiences this year in 8th grade math class was when I allowed the students to struggle as they completed a math task. This sounds really odd, a teacher enjoying watching her students struggle. I had so many doubts and honestly, I didn’t want to lose the instructional time. I also didn’t want the students to become frustrated and just give up. But I will admit, I was curious: would productive struggle work well in the classroom of 8th graders? I seated the students into groups of three and I provided the math task. The only group guidelines were that everyone must contribute and have an opportunity to participate. Here is when the fun began: after several minutes of the students sitting in shock and disbelief, they asked “Is Ms. Trail not going to help get us started with the task?” I have always read the task and reminded the students of their prior learning. This was very difficult for me to let go of the reins and it was difficult for the students to just start the task all on their own. However, after a few minutes of me walking around the classroom, I witnessed students collaboratively sharing ideas, solutions, and strategies as they were completing the math task on transformations. My role in supporting the productive struggle task was to go to the different groups and ask questions with limited assistance. Then, I stepped back and watched the students discuss with one another and allowed conceptual understanding to unfold right in front of my eyes. My students were engaged (even the ones that typically do not contribute), my students were participating, and my students were eager to share with their classmates. The students were still discussing the task and their accomplishments when exiting my classroom. My big takeaway from allowing productive struggle in math class is that the struggle allowed the students to build their confidence and deepen their understanding of math and how to approach problems. 

    Carol Jones, another middle school math teacher in the district, had similar experiences and realizations about her students.

    As I have tried and continue to try supporting students in productive struggle within my classes, I am constantly amazed at the students who CAN actually come up with the answers! Students I knew usually did not engage, are now actually trying and surprising themselves with correct answers! They grin so big, and it makes them want to work harder! It makes my heart happy to see them finally excited about math!

    Viewing Sara and Carol’s classroom with an equity lens:

    …productive struggle comprises the work that students do to make sense of a situation and determine a course of action when a solution strategy is not stated, implied, or immediately obvious. From an equity perspective, this implies that each and every student must have the opportunity to struggle with challenging mathematics and to receive support that encourages their persistence without removing the challenge. (Boston & Dillon, et al., 2017)

    In both classrooms, every student was provided the opportunity to grapple, collaborate, and succeed. But, in addition, the role of the teacher’s expectation that students can and will be mathematicians was crucial. They saw that by not doing the heavy lifting and by requiring students to take on the responsibility of productively struggling with the math, students who may have previously been reticent to engage, defied the passive role and shined!

    THE CASE FOR ADMINISTRATIVE SUPPORT:
    SUPPORTING RISK-TAKING IN TEACHING PRACTICE

    As we have learned from a growing body of research from the Wallace Foundation, school leadership is second only to classroom instruction in affecting student learning, and may matter even more than previously thought (Grissom, et. al., 2021). In other words, what principals do really matters. One important role of school leaders is that of inviting and supporting a school climate that is conducive to learning, both at the student AND the teacher level. Denise Collier, a leadership fellow and former district administrator, shares.

    The impact of principals on the learning culture was made clear to me in a recent conversation with a team of middle school math teachers. When asked what worried them the most about trying out a new instructional strategy—engaging students in collaborative group work around a challenging task—they jointly voiced a concern that their principals wouldn’t understand or support what they were doing, especially if the new strategy wasn’t immediately successful. The teachers raised questions like:

    • Will my principal trust me enough to let me try something new if it takes some time to get good at it?
    • What if my principal comes in for a walk-through or observation and the students are loud or it looks like they are off task?

    So, if one of the risks to continued learning is fear of failure or criticism, one of the key roles of principals should be creating a climate of trust, support for learning, and innovation in their classrooms.

    The experiences that these teachers shared validates that implementing research-based practices works! The teachers spoke to both the learning as well as the excitement and ownership of the math. Taking on new practice served the students, but also the teachers as they learned more about the students’ assets and honored them as doers of math! Thanks to the teachers who take risks to serve their students and the administrators who support them.

    References:

    • Boston, M. D., Dillon, F., Smith, M. S., & Miller, S. (2017). Taking action: Implementing effective mathematics teaching practices in grades 9-12, (p. 208). Reston, VA: National Council of Teachers of Mathematics.
    • Grissom, Jason A., Anna J. Egalite, and Constance A. Lindsay. (2021). How principals affect students and schools: A systematic synthesis of two decades of research. [Research Report] New York: The Wallace Foundation. http://www.wallacefoundation.org/principalsynthesis.  

    Browse our collection of IFL Math Tasks at ShopIFL.com

    IFL Recommends for March 2024 

    Mar 5, 2024 | IFL Recommends

    This month’s recommendation includes three diverse topics—a podcast from psychotherapist Stephen Quinlan that seeks to help anxious children, a list of 25 picture book autobiographies for various ages, and a set of Pi Day resources, including the history of Pi and some activities.

    Beatriz T. Font Strawhun

    Beatriz T. Font Strawhun

    Mathematics Fellow

    Beatriz says, “Hosted by Stephen Quinlan, an author and psychotherapist with over 20 years of experience working with kids and parents of kids with anxiety, this podcast offers some ways to help anxious kids, including book suggestions, strategies, and tips for both parents and kids on what to say and how to say it. With the suggestions and tools that this podcast offers. I have been able to help the teens in my life. Take a listen and see if it may help you help someone in your circle.”

    The Anxious Child Podcast
    Stephen Quinlan, host

    “The first goal should really just be awareness and education so that they can hear words maybe they have not heard of before. Some kids haven’t even heard of anxiety. They don’t really understand what it is.”
    – Ellie Campbell, mom, wife, photographer, writer, copy editor, and consumer of numerous books on anxiety

    Listen here

     

    Anxious Child Podcast Logo

    Kim Rugh

    Kim Rugh

    Project and Communications Manager

    Kim says, “Biographical picture books are one of my favorite genres because they offer an accessible window into the lives of interesting and notable people. While they may often be geared toward early readers, they also provide an easy way for more advanced readers to dive into the stories of influential people and get inspired to learn even more.”

     

    25 Wonderful Picture Book Biographies for Readers of All Ages

    “Picture books are accessible to all levels of students—even high schoolers! Readers get a broad overview in picture book form, and then authors generously include ample back matter for further research and reading.”

    – AuthorAmy, teacher, reader, and writer

    See the list

    Queen of Physics

    Joe Dostilio posing in front of trees

    Joe Dostilio

    Mathematics Fellow

    Joe says, “Each year around this time I like to share some links to videos and classroom resources for Pi Day. The history is so fascinating. I have always found that learning about the history behind mathematics is a great way to spark interest. Recently, my 11-year-old son and I have been watching a documentary on Albert Einstein and he has gotten super interested in the theory of relativity and the development of nuclear energy. Now, I think we will take some time to delve into the history of Pi and do some activities for calculating and using Pi.”

     

    Celebrate Pi Day!

    Here are some great resources for Pi Day 2024, including
    • the history of Pi,
    • a deeper look at the different ways mathematicians have calculated Pi over the years,
    • some activities the National Council of Teachers of Mathematics (NCTM)has pulled together,
    • some interesting facts and more activities curated by NASA, and
    • a video of then 10-year-old Charley Thomas reciting Pi to 258 places!

    The Hype and Hope of Generative AI in Education

    Feb 20, 2024 | English Language Arts Teaching and Learning, Instructional Leadership, Mathematics Teaching and Learning

    By Michael Telek

    Throughout history, when new technologies emerge, educators are often the early adopters. If it can increase learning in a meaningful way, that technology will find a home in the classroom.

    Xerox introduced its photocopier in 1959, allowing for mass
    production of materials. Electronic handheld calculators allowed us to say goodbye to slide rules in the 1970s. The internet has grown from a rarity in classrooms; to a necessity. In 1994, just three percent of public-school classrooms had internet access.  In 2021, 45 percent of schools reported having an electronic device (laptop or tablet) available for every student.
    Xerox machine

    The xerographic process, which was invented by Chester Carlson in 1938 and developed and commercialized by the Xerox Corporation. Credit: Xerox

     

    For school leaders and teachers around the globe, ChatGPT and generative artificial intelligence (AI) is a new frontier. OpenAI released their generative pre-trained language model ChatGPT 3.5, free to the public in November 2022. It is designed to understand context, generate coherent and contextually relevant responses, and engage in interactive dialogues. The model is fed a diverse range of internet text, allowing it to source solutions and grasp the intricacies of language, syntax, and context. Users can interact with ChatGPT by providing prompts or queries, and the model responds swiftly and accordingly

    That’s what makes ChatGPT different from the AI you may already be familiar with. The chatbots helping you online or over the phone. Alexa and Siri understanding your commands. AI algorithms picking the perfect content for your social media timeline. Same for streaming services suggesting your next show. ChatGPT uses algorithms that processes natural language inputs and predicts the next word based on what it’s already seen. Within two months of its global launch, ChatGPT had more than 100 million active daily users. Industry experts have dubbed it one of the fastest growing apps of all time. To give that claim a little more credence, investment firm UBS tracks these kinds of metrics. They said it took TikTok about 9 months to add 100 million users and Instagram 2.5 years to reach that feat. That success sent a seismic shift across Silicon Valley. Microsoft allegedly gave OpenAI a $10 billion investment to expand their Generative AI work (OpenAI is also behind DALL-E 2, another AI program that can generate art from mere lines of text).  Google’s stock dropped nearly 8 percent, wiping out more than $100 billion in value, when they botched the announcement of Bard, their answer to ChatGPT. The message from Wall Street was simple. Generative AI is the future, and the future is now.

    AI teacher

    An AI generated picture for “classroom” via Adobe Firefly.

    Billed as the next great, transformative tech, generative AI has spent the year generating a lot of buzz, both good and bad. It promises the automating of mundane tasks and increased productivity in the workplace. However, there are plenty of examples of the model making up answers. Legal experts are questioning if this is just a plagiarism machine. And of course, bad actors are always lurking.

    Is technology neutral?

    “People think that technology is neutral. Right? Like people think that oh, technology, it’s a computer. It’s math. It in no way can discriminate. But it’s really just amplifying and recreating existing discrimination that exists in our life,” said Dr. Angela Stewart, an assistant professor at the University of Pittsburgh’s School of Computing and Information and a research scientist at the University’s Learning Research and Development Center(LRDC) where she explores the intersection of the learning sciences, artificial intelligence, and human-computer interaction. Much of that work centers around equity and creating culturally responsive technology. Dr. Stewart has been studying AI for nearly a decade now, but in recent years has been focusing on how people learn about AI. How they think about its usage. How people create AI and technology systems through learning experiences.

    “Equity has always been something that has been a particular passion of mine. As a Black woman in computing, there’s certainly not a lot of us, so I often think about how that influences my experience every day in computing as well as the kinds of things that get developed,” said Dr. Stewart. “I think that one of the many reasons why these biases exist is because of the limited perspectives of people creating these systems.”

    During an episode that aired in March 2021, Jimmy Fallon was joined by Addison Rae, a white woman, to perform some of the most popular dances on Tik Tok. After a backlash, Fallon acknowledged that the original TikTok creators deserved recognition. Credit: The Tonight Show/YouTube

     

    Like the people behind the technology, AI programs are far from perfect. Biases across different technology have shown themselves to be problematic on many levels.  Speech recognition software that cannot understand non-American accents. Health care algorithms creating errors because they are only trained on a specific group of people or certain stages of a disease. Facial recognition software used by law enforcement that can racial discriminate. Tik Tok’s algorithm required tweaks after proving to hold bias. The app would promote white content creators over people of color who often times were the originator of a popular trend. The views are given to the white creators, and with that, the brand endorsements and money that comes from a viral hit. 

    Of course, having diverse voices in the room when technology is made, when the ideas are being initiated, are important. But if the leaders and decision-makers are not listening to those voices, or valuing those voices, then the problems will persist. “We can’t just rely on the most marginalized people to fix all of these societal oppressions,” said Dr. Stewart. “In this space, it is important for people who have power, let’s say researchers who have power and are creating new AI systems. Administrators, principals, and school board leaders use that power to advocate for learners. To advocate for parents. To advocate for equitable and inclusive uses of these technologies. Part of that starts with awareness, understanding what is happening, what is going on. Then the second part is actively taking steps to combat [bias].”

    “Machine learning demonstrates that we can actually make a machine learn and perform tasks, but with that comes a lot of implications,” said Dr. Cassandra Kelley, a researcher and lecturer specializing in AI and emerging technologies and a member of the crew at the Center for Integrative Research in Computing and Learning Sciences (CIRCLS) a partner of the University of Pittsburgh that supports the LRDC in their mission. Dr. Kelley agrees with the assessment from Dr. Stewart and says a lot of the conversation at CIRCLS is about the person behind the machine. “For instance, there can be biases learned from data that lead AI to generate and spread misinformation. Furthermore, how AI is used can lead to unethical consequences—especially if we rely on AI to make decisions for us.”

    Both Dr. Kelley and Dr. Stewart stressed their concern over user’s data. What’s happening to it? Where is it going? How long will it stay there? Is it being used responsibly? Machine learning requires lots and lots of data for the AI system to make predictions of what a human would do. However, if that data is being used responsibly and respectfully, it could make positive impact in the classroom. “[AI systems] could be a thought partner that you can utilize to brainstorm with, whether it’s thinking through problems or exploring creative ways to do something,” said Dr. Kelley. “From a productivity perspective, teachers might use AI to initially gather ideas or resources for lesson plans and assist with personalizing assessments for students’ varying needs. Essentially, AI could be leveraged to alleviate some of this workload and help to create with the teacher.

    Data could also be used to provide feedback not just to students, but teachers as well. Dr. Stewart is part of a team designing ClassInSight, an app for teachers that uses language-based AI techniques to visualize student and teacher discourse patterns in the classroom.

    Shock and AWE (Automated Writing Evaluation) 

    Without writing, it is hard to advance through school, let alone life. That’s why we have federal and state standards for English language arts that includes writing. Engaging students in the revision process of analytic text-based writing is often lacking during elementary years.  

    Teachers report feeling underprepared to teach writing, and the time needed to assess student writing is burdensome. When students do write, they rarely receive substantive feedback and rarely engage in cycles of revision that require them to apply feedback to strengthen their work.

    “We’re addressing what I’m thinking is a bit of a crisis, which is most students in most schools are only getting one-shot assessments. They’re doing one piece of writing one day and they’re not returning to it,” said Rip Correnti, a professor at the LRDC. “They’re never engaging in revision. Part of our design is to build in the revision as an essential part of writing.”

    Correnti is on the team that developed eRevise, an Automated Essay Scoring (AES) and Automated Writing Evaluation (AWE) system, for improving fifth to seventh-grade students’ skill in using text evidence. eRevise uses machine learning natural language processing (NLP) techniques to predict evidence-use scores in students’ writing.

    Architecture of eRevise

    The score provided by eRevise is based on features of evidence use on a typical grading. Using a rubric-based approach to ensure that the features of “good text-evidence use” (e.g., number of pieces of evidence provided, specificity of evidence) are well represented by the scoring algorithm. The system was trained on more than 1,500 previously collected and manually scored essays by humans.

    “We pretty easily can highlight surface level errors, spelling, grammar, whatever, but we’re talking more about trying to help students along with aspects of argumentation that are substantive and important,” said Elaine Wang, a policy researcher at RAND.

    eRevise Student Interface with example feedback messages.

    “We are motivated to make sure that kids are having opportunities to engage with a rich task. Students don’t often have a lot of opportunities to practice their argument writing. We were inspired by those two challenges in teaching practice,” said LRDC Associate Director and Senior Scientist Lindsay Clare Matsumura.

    Moving forward, the group wants to not only provide feedback through eRevise, but also directions for students on how to improve their revision skill, traditionally a difficult writing skill to learn. That’s where generative AI could play a role. As part of a new grant, they’re working to identify different revision patterns to bolster student revision strategies.

    Finding Our Way in This New Frontier 

    When ChatGPT was released, the first move of many was to block the technology until it could be better studied and understood. New York Public Schools banned students’ use of the software. Four months later, the nation’s largest school district rescinded that ruling after holding learning sessions with industry experts.

    Seckinger High School in Gwinnett County, Georgia went in the opposite direction. They embraced AI, making it the first artificial intelligence-themed school in the country. About 1,500 students received a college preparatory curriculum that was taught through the lens of artificial intelligence this school year.

    After caution and trepidation, AI is becoming more common in the classroom. A recent survey from Education Week found that a third of teachers asked are using artificial intelligence-driven tools. While most admit to using them “a little”, generative AI is creating lesson plans, building rubrics, and even composing emails to parents or writing letters of recommendation.

    “My advice is to think about the nuance. It’s not all good, and it’s also not all bad,” said Dr. Stewart. “In particular, key into the ways that ChatGPT and other kinds of generative AI systems might be supportive of learning.”

    We are in an era where misinformation runs rampant, now more than ever, Dr. Stewart says it will be important for students to critically think about sources and where they come from. Determine what’s real and true information versus what is not.

    It’s something governments of all shapes are sizes are currently wrestling with. At the federal level, there’s a bi-partisan effort in the House to combat “deepfakes” by creating baseline protection against AI abuse and uphold Americans’ First Amendment rights online.

    Since 2019, 17 states have enacted 29 bills focused on regulating the design, development and use of artificial intelligence, according to the Council of State Governments. North Carolina is the latest state to release AI guidance for schools. Educators in the Tar Heel State have been supplied parameters around AI’s use.

    “I really urge educators and administrators to think about it as a tool. When can the tool be used, and how, rather than thinking about it so strictly from this plagiarism perspective,” said Dr. Stewart.

    Dr. Kelley shares similar a sentiment. She echoed Dr. Stewart’s point in the nuances of the new technology and importance of having discussions surrounding the positives and perils with not just teachers and students, but also families.

    With generative AI changing the game, educators and administrators will have to adapt. There will need to be new standards surrounding academic integrity and further questions asked around how we teach with and about AI. Digital literacy and digital citizenship should be at the forefront of our focus.

    Another adjustment Dr. Kelley could see happening revolves around the dynamics of how we assess and gauge learning. She says in this new world, teachers will have to consider different types of evaluations for students. With generative AI’s ability to spit out writing prompts and pass multiple choice exams, it will be imperative that teachers incorporate further opportunities for collaboration and project-based work.

    “We have to consider different approaches and prepare students for a new future,” said Dr. Kelley. “Students need to have a clear understanding of how these technologies are created and intended to work, as well as their imperfections. They need to recognize that AI is developed by people and with that can come biases and other ethical concerns. Additionally, they will need guidance on interacting with such technologies, which should include learning about how issues such as cyberbullying and deepfakes can be further amplified if these technologies are not used appropriately.

    Reference:

    Zheng, Z., & Zhang, G. (2019). Practical research of pre-service teachers’ TPACK development based on design-based. China Educational Technology, 389, 86-94.