🔐 Data Privacy for AI in Schools

Explore the data privacy policies of ChatGPT, Gemini, and Copilot, and discover best practices to safeguard student data privacy.

and

Mar 05, 2024

As the use of AI technologies rapidly expands in schools, the issue of AI data privacy has become a critical concern. AI systems rely on vast amounts of data to train their algorithms, and in an educational context, this data often includes sensitive personal information about students, such as academic records, behavioral data, and more. In this issue, we will discuss some of the data privacy policies among popular AI chatbots like ChatGPT and offer tips to promote best practices for data privacy in schools.

Discussions surrounding such concerns are crucial, and our team at AI x Education is committed to addressing them through our upcoming webinar series. Which topics are you most interested in seeing covered in our webinars?

Here is an overview of today’s newsletter:

Data privacy policies and concerns among commonly used chatbots like ChatGPT, Gemini, and Copilot
Tips to protect student data and privacy in the age of AI
Student usage case of ChatGPT to learn a new language
Latest research on generating math word problems that cannot be answered by LLMs

🚀 Practical AI Usage and Policies

Dr. Torrey Trust, Professor of Learning Technology at the University of Massachusetts Amherst, offers a thorough resource on AI & Ethics, covering privacy and student data, bias, misinformation, hallucinations, human labor, digital divide, intellectual property rights, and environmental costs of AI. Below, we share some of these insights from her presentation: GenAI & Ethics: Investigating ChatGPT, Gemini, & Copilot (Torrey Trust, 2024). Here are the breakdowns of Data & Privacy for the most commonly used Generative AI chatbots, along with some tips for educators:

OpenAI

OpenAI requires ChatGPT users to be 13 years or older

The use of ChatGPT by individuals under 13 years old would violate the Children’s Online Privacy Protection Act (COPPA), since OpenAI collects a LOT of user data. This includes the user’s time zone, country, dates and times of access, type of device you’re using, and computer connection! Here’s an example of the type of data it might collect from a user: https://webkay.robinlinus.com/

Likewise, if you ask a student to use ChatGPT to revise a college admissions essay that includes information about a trauma they experienced, OpenAI collects and keeps that data!

Use of ChatGPT by 13-18 year olds requires parental permission

Check out this article by EdWeek, Permission Slips to Use ChatGPT? Some Schools Say They’re Necessary, to learn more about the topic of parental permission for ChatGPT usage.

ChatGPT users can opt out of having their data used to train models

Quite simply, OpenAI uses your data to improve their products. You can opt out of having your data used to improve the way they train their model!

Google Gemini

Gemini requires Gemini users to be 13 years or older

Google Workspace for Education account holders under 18 years old will not be able to use Gemini. Students older than 13 would need to use their personal Gmail account to access Gemini.

Gemini stores your data activity for up to 18 months

Google collects a LOT of user data, including user conversations with the chatbot, usage data, location data, and user feedback. If you are 18 years or older, Google stores your activity (e.g., any “conversations” you have with Gemini) for up to 18 months. They also collect your location data, IP address, and home/work address. Google collects any information you input as data, so if you write a prompt including any personally identifiable information about your students, it keeps that data; and is a possible FERPA violation. Likewise, if you ask a student to use Gemini to revise a college admissions essay that includes information about a trauma they experienced, Google collects and keeps that data!

Microsoft Copilot

Microsoft requires Copilot users to be 13 years or older

Microsoft has data retention and deletion policies for data minimization

Copilot in Bing has data retention and deletion policies for you to better control your data.

Microsoft has ownership of the prompts you input

Any prompts that you input into Copilot or anything you create with Copilot are immediately owned by Microsoft. They can use your prompts and creations however they see fit. So, if your students come up with an amazing prompt that turns Copilot into a tutor for your class, Microsoft will own that prompt and could use or distribute it.

AI Data Privacy Summary

Don’t put any sensitive information in these tools (e.g., asking ChatGPT to write an email to a student about their grade - this is FERPA violation).
Don’t let students put any sensitive or identifying information into any of these tools.
ChatGPT requires parental permission for 13-18 year old users, Gemini and Copilot do not.
ChatGPT and Gemini can give away any data collected to “affiliates,” including, if requested, to federal authorities. (How Law Enforcement and Intelligence Agencies Are Buying Your Data from Brokers)
Microsoft & Google have more data privacy protections for users (thank you GDPR!)
Google tracks user location, OpenAI collects IP addresses, and Microsoft CoPilot doesn’t seem to collect any location data.
Any information input into these tools (e.g., any prompts they write) is data that can be used by the companies that made the tools.
Sharing your login with students so they can access ChatGPT is a violation of OpenAI’s terms of use (“You may not share your account credentials or make your account available to anyone else and are responsible for all activities that occur under your account.”)
Teach students about the privacy policies and terms of use of these tools (they may not know that their location and what they type into a prompt could be collected and stored).

📣 Student Voices and Use Cases

This week, we had a chance to speak with Emerald, a junior at Northwestern University studying Oboe Performance with a minor in Computer Science. In the following, we present select highlights from these conversations, which have been slightly edited for enhanced clarity and precision:

Q: In what ways have your teachers integrated AI tools into the classroom, and could you offer a specific example of how you interacted with these tools?

None of my teachers have integrated AI in my coursework so far. In fact, my musicology class has very much discouraged the use of ChatGPT when writing essays because we are expected to do the research on our own. For me, I have used AI for studying on my own instead. When I was learning Chinese, I would practice my Chinese by inputting sentences into ChatGPT and having it correct my grammar and vocabulary. What’s great about ChatGPT is that it can break down my sentences and explain why certain parts have incorrect grammar, and it can define specific words in my sentences.

Q: What do you wish your teachers knew about how students interact with AI?

I think it’s important for teachers to understand that most students know they shouldn’t be using AI to completely make up the work for them. Teachers should give students the benefit of the doubt that we are in fact being genuine about the use of these AI tools, not to plagiarize but as a tool to help with learning. AI tools like ChatGPT can help explain concepts in a way that makes the most sense to each student and can make learning more effective if used properly. I think that teachers can help by providing ways on how we can use AI most effectively for their classes, like what type of prompts are most helpful. This may require more planning and experimenting on the teacher’s side, but it can provide useful guidance for students when using AI tools.

Q: What role do you think AI should play in the assessment and grading process?

Sometimes it can be helpful to use AI to grade certain assignments, but I don’t think that AI should grade papers. Everyone has different writing styles, and there isn’t a clear right or wrong way to word sentences or write an essay. However, since the AI was trained on a certain set of data, it may just think that that particular style of writing is correct and punish students whose writing style varies from the data it’s been trained on. For instances, some sentences that are short does not necessarily indicate bad writing versus some sentences that are really long also doesn’t mean that it’s bad writing. Having 5 commas instead of 2 doesn’t necessarily mean anything to a computer, but to a human it does mean a lot. I don’t think there has been enough AI linguistic research done to give objective grades on written papers at the current moment.

📝 Latest Research in AI + Education

Duke University

LLM-Resistant Math Word Problem Generation via Adversarial Attacks ↗️

This research introduces a novel approach to generating math word problems (MWPs) that large language models (LLMs) are unable to solve, addressing the challenge of assessing students' true problem-solving abilities in an educational landscape increasingly dominated by advanced AI tools. By utilizing adversarial attacks that alter the numeric values in MWPs while preserving their structure, difficulty, and educational value, the researchers aim to create assignments that defy solutions by LLMs, thereby ensuring fairer educational evaluations. Through experiments with various LLMs, the method demonstrated a significant reduction in the models' problem-solving capabilities, highlighting shared mathematical vulnerabilities and proposing a cost-effective strategy for attacking high-cost models. This work not only contributes to the development of more robust educational tools but also enhances understanding of LLM limitations and the ethical implications of their use in education.

Xie, R., Huang, C., Wang, J., & Dhingra, B. (2024). LLM-Resistant Math Word Problem Generation via Adversarial Attacks. [Preprint]. arXiv:2402.17916.

Cornell University, Pennsylvania State University

A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education ↗️

The research explores the development and use of a no-code chatbot design tool for K-12 teachers, leveraging large language models (LLMs) to assist in educating adolescents about cyberbullying through role-playing scenarios. It highlights the potential of chatbots to provide personalized, interactive learning experiences at scale, addressing the challenge of teaching bystander intervention skills amid a widespread teacher shortage and the increasing prevalence of cyberbullying. The study involves 13 middle school teachers who used the tool to design chatbots, revealing their enthusiastic reception and innovative approach to utilizing chatbots as educational aids. Teachers saw themselves as "playwrights," crafting scenarios that allow students to safely explore both positive and negative responses to cyberbullying situations, thereby fostering a deeper understanding and practice of prosocial behaviors. The study identifies opportunities for leveraging LLMs to support teachers in scriptwriting, suggesting the potential for collaborative chatbot design involving teachers and students to enhance the relevance and effectiveness of cyberbullying education.

Hedderich, M. A., Bazarova, N. N., Zou, W., Shim, R., Ma, X., & Yang, Q. (2024). A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education. [Preprint]. arXiv:2402.17456.

📰 In the News

EdWeek

Schools are Taking Too Long to Craft AI Policy. Why That’s a Problem ↗️

Key takeaways:

Over a year since ChatGPT highlighted AI's impact on education, most school districts lack clear policies on AI use, struggling with issues like plagiarism and data privacy.
A survey by the EdWeek Research Center found that 79% of educators say their districts have no clear AI usage policies, and 56% expect AI tool use to increase in their districts.
Conflicting practices within schools, where AI tools might be freely used in one class and banned in another, highlight the lack of unified policy, creating confusion and inequity.
Rapid AI development challenges policy-making, with many educators feeling unable to address AI due to other priorities. However, there's a push for "living, breathing" AI policies that can adapt over time.
Despite challenges, there's recognition of AI's importance in education, with efforts to provide guidance and create AI policy frameworks to ensure students gain the necessary skills and navigate AI's ethical use responsibly.

Stanford Report

How Technology is Reinventing Education ↗️