ChatGPT for Teens rated an “unacceptable risk,” students drafting AI policy for 10,000 districts, and two Utah takes on AI
Key Takeaways
- Safety settings alone won’t protect students. Common Sense Media’s Youth AI Safety Institute ran 4,000+ prompts through ChatGPT for Teens and rated it an “unacceptable risk,” its highest warning.
- Students are writing AI policy that districts are adopting. Two Washington seniors surveyed their community, then helped draft the Students First Act, now in front of more than 10,000 districts.
- A policy is only the first step. Utah gave every school access to Gemini, and two districts went in opposite directions, which shows what changes when guidelines come paired with teaching.
Researchers flag OpenAI’s ‘ChatGPT for Teens’ as an “unacceptable risk” for students
Weeks after OpenAI promised stronger teen protections, researchers tested them and gave the product their highest warning.
The Gist
In August, OpenAI announced ChatGPT for Teens, a version with safety settings for users ages 13 to 17 that promised stronger protections, including alerts to parents and caregivers when sensitive topics come up. Weeks later, Common Sense Media’s Youth AI Safety Institute, a group that evaluates how AI products affect young people, tested those protections and rated the product an “unacceptable risk” for students, the highest warning on its scale. That gap is the reason schools can’t lean on safety features alone, and why building students’ own judgment around AI matters now.
What to Know
Researchers at the Youth AI Safety Institute ran more than 4,000 prompts through ChatGPT, testing it both before and after OpenAI rolled out its new teen safety features, to see whether the changes actually protected young users. Some safeguards held up: the chatbot repeatedly refused to engage in inappropriate conversations and limited teens’ exposure to harmful content. Two newer features fell short. The first was parental controls: ChatGPT did not consistently alert caregivers linked to their teen’s account when a conversation turned to self-harm or eating disorders. The second was Study Mode, a setting meant to walk students through problems step by step instead of handing over answers; researchers found students could get around it and receive full solutions using a new “Show me the answer” button.
Tom Siegel, who directs the Youth AI Safety Institute and previously led trust and safety at Google, told KQED his team expected far bigger improvements given how OpenAI had promoted the update. OpenAI pushed back, saying some testing happened before its parental controls fully launched and that the assessment does not reflect how its teen safeguards actually work.
TSI’s Take
AI companies are working to make their platforms safer, but that doesn’t mean students can rely on them fully. They still need the judgment to protect themselves when a guardrail fails, and that means teaching them to question what AI tells them, spot its bias, and know when to step away. Educators can build those skills by:
- Pressure-testing AI against real people: Have students ask a chatbot and a trusted adult the same question, then compare what each one got right and what it missed. It builds the habit of checking AI instead of trusting it outright.
- Naming the moment AI turns into a companion: Help students notice when they’re using a chatbot for comfort or connection, and have them list real people they can turn to in those moments.
- Giving every student a privacy plan for AI: Decide together what they’ll never type into a chatbot, and what they’ll do if a conversation turns to something heavy like self-harm.
Give students real practice at telling solid AI advice from the kind they shouldn’t trust with the #WinAtSocial Lesson, Advice From Real People vs. AI Chatbots, where they put a chatbot’s guidance head-to-head with the people in their lives.
Two students helped draft an AI policy now reaching 10,000 districts
Two Washington seniors surveyed their classmates and teachers, then carried what they heard into a model AI policy for K-12 schools.
The Gist
When two Washington high school seniors looked at how their schools were handling AI, they started by asking classmates and teachers what a fair policy should actually do. That research traveled with them to Day of AI’s Youth AI Festival in Boston, where student representatives from all 50 states drafted The Students First Act, a nine-page model AI policy for K-12 schools. It tackles plagiarism, bullying, fairness, and teacher autonomy with ideas pulled straight from real classrooms, and the Student Senate passed it 84-14. The bill has since gone to more than 10,000 districts as recommended policy. For schools that are building out their AI guidelines, this serves as a reminder that the strongest AI policies are built with students from the start.
What to Know
Sophia Fenn and Kainoa Coit are high school seniors in Washington’s Peninsula School District who were chosen to represent their state at the Youth AI Festival. In the weeks before they traveled to Boston, they surveyed students, teachers, and families in their community about how AI was showing up in school. They wanted the policy they helped write to reflect the people who sit in classrooms every day, not just what adults assumed students needed. What they heard was nuanced: Fenn found that students welcome AI and still want room for their own ideas, while Coit set out to capture the full range of experiences, from students who leaned on AI heavily to those who avoided it.
The research these students gathered turned into concrete rules in The Students First Act. One provision asks every teacher to spell out in their syllabus when AI is allowed, limited, or banned for each assignment, and to keep those rules the same across all sections of the same course so students aren’t guessing from class to class. Another gives students the right to appeal when they’re wrongly accused of using AI: instead of a school relying on an AI detection tool, which can flag a student’s original work by mistake, two teachers or administrators review the assignment and make the call. Both ideas came directly from problems students see in their own classrooms, which is what happens when schools give them a real seat at the table.
TSI’s Take
When students help write the policies that affect them, they land on practical, fair solutions. Fenn, Coit, and their peers from every state focused on fairness, teacher autonomy, and student accountability because they know what classrooms are really like. Educators can give students that same ownership by:
- Inviting students onto AI policy teams: Save seats for students when your school drafts or revisits its AI guidelines so the expectations reflect how students use these tools day to day.
- Gathering student input first: Like Fenn and Coit did with their peers, survey students on where AI helps their learning and where it gets in the way before setting expectations.
- Putting AI expectations in writing: Share clear, assignment-level guidance on when AI is permitted, limited, or off the table so students know where they stand in every class.
Empower students to lead and shape your school’s AI policy by involving them from the start. Our School Leader’s Guide to Creating Clear AI Policies With Student Voice and Buy-In guides school teams on how to bring real student voice into AI expectations.
Two Utah districts, two takes on AI, and what schools can learn
Every Utah public school got access to Gemini. One district blocked it, another paired it with teaching, and the contrast is instructive.
The Gist
This school year, Utah’s education department gave every public school access to Google Gemini, a free AI chatbot, and let each district decide how students could use it. A few weeks in, KSL reports that a student’s access now depends entirely on which district they attend. Provo City School District took the cautious route, blocking Gemini and other AI chatbots on student Chromebooks while it finishes writing its AI policy. Canyons School District took a different approach, turning Gemini on for teachers, staff, and high schoolers, while teaching them how to use it in healthy ways. That difference matters for educators everywhere, because students will use AI whether or not their school allows it, and the ones being taught how to use it well are better prepared for what’s ahead.
What to Know
Canyons built its approach to AI around a strategy it calls a stoplight protocol, which tells students exactly how much AI they can use on any given assignment. Red means no AI at all. Yellow means students can use AI only in the specific way the teacher spells out. Green means AI is fully approved. Using AI to write an entire essay or finish homework stays off-limits, so students lean on it to brainstorm ideas, get feedback on a draft, or work through a concept they find confusing.
The point is to keep students doing the thinking. In one science class, students run their lab analysis through an AI model for feedback, then explain which of its suggestions they accepted or rejected and why, since AI output is not always correct. Teachers use AI too, to make lessons more relevant and to support multilingual students and those who need accommodations, and they do it without entering any information that identifies a student.
Canyons didn’t arrive here overnight. The district started this work in 2023, publishing an AI guidebook that fall and training teachers on how AI works and how to use it safely. Statewide rules are still catching up, as the Utah State Board of Education continues to develop its own AI guidance.
TSI’s Take
Writing an AI policy is only the first step in helping students use these tools responsibly. Canyons shows that guidelines work best when they come paired with education, teaching students to recognize when AI supports their learning and when it gets in the way. Schools can build that judgment in students by:
- Labeling AI use assignment by assignment: A simple system like Canyons’ stoplight protocol or the three-tier approach many of our partner schools use (no AI, AI for brainstorming and support, or full AI use) gives students a clear answer before they ever open a chatbot.
- Asking students to explain their AI choices: Have students note which AI suggestions they kept or tossed and why, which builds the habit of checking AI output.
- Talking through helpful and hurtful uses: Discuss real examples of AI supporting learning, like brainstorming, alongside examples where it takes over the thinking a class is meant to build.
Give students hands-on practice with the same judgment Canyons is teaching, through #WinAtSocial AI and Media Literacy Lessons. Students work through real situations where AI either helps or hurts their learning, practice checking its output, and build the habit of thinking for themselves first, so your policy becomes something students actually live out.
#WinAtSocial gives K-12 educators ready-to-teach lessons on AI literacy, checking AI output, and protecting student privacy, built around the apps and headlines students see every day. Request a demo to see how it works in your school.