ALP partnered with Virtual Virginia and the Virginia Association of School Superintendents (VASS) to develop beginner, intermediate, and advanced-level AI literacy courses for teachers across Virginia. We surveyed Virginia education leaders to see what topics they wanted to see in the courses, and found that student data privacy was overwhelmingly the number one topic. This blog is the first in a series highlighting content that was created for the AI literacy courses. Continue reading to learn how AI complicates data privacy and how educators can responsibly use tools while keeping students safe.
What is PII?
By now, we’ve learned that you can’t put personally identifiable information (PII) into AI, but the understanding of what is personally identifiable is murky, and the consequences are even murkier. The NY Times reported in 2006 that a woman was identified solely based on her internet search history. Twenty years later, it has become common for most of us to provide websites and apps with personal data regularly. If internet search history data is identifiable, what else constitutes PII?Personally identifiable information is often associated with obvious markers such as names, email addresses, or student ID numbers. In practice, PII extends far beyond these identifiers. For example, an AI prompt that says “a 4th‑grade student who receives speech services,” “a student who recently immigrated and is learning English,” or “a student who struggles with attention during independent work” may identify a real child within a school community even without a name when combined with all of a teacher’s queries. AI tools amplify this challenge because prompts often include contextual detail meant to improve output quality.
Where does my data go?
Take a look at the following examples of what happens to your data once it is shared with a system.
Scenario 1: “Just Enter Your Email to Get the Discount”
The setup:
You enter your email address on a company’s website to get a coupon, download a guide, or sign up for updates.
What you think is happening:
- The company stores your email
- You might get occasional marketing emails
- You can unsubscribe later
What often actually happens:
- Your email is stored in the company’s customer database.
- That database is connected to marketing automation platforms (e.g., Mailchimp, HubSpot) and analytics tools that track clicks, opens, location, and device type.
- Your email is hashed (converted into a coded identifier) and matched against advertising networks, social media platforms, and data brokers that already have profiles linked to that email.
- Your profile may now include: inferred age range, interests, purchasing likelihood, and location patterns.
- That profile can be sold to third-party data brokers, shared with “partners” listed vaguely in the privacy policy, or used to target ads across other sites and apps.
Key Insight:
Your email becomes a persistent identifier, not just a contact method. Even if the company never “steals” your data, it can legally circulate through an entire ecosystem and be linked to other information about you from across the web.
Scenario 2: Phone Number + Breach = Long-Term Exposure
The setup:
You enter your phone number into a form for delivery updates, appointment reminders, or account verification.
What you think is happening:
- The number is used only for that service
- It’s temporary and limited
What happens if there is a breach?
- The company’s database is hacked or improperly secured.
- Phone numbers are extracted – often alongside names, email addresses, and partial addresses.
- That data appears on dark web marketplaces and in private data trading forums.
- Other actors buy the data and combine it with previous breaches, public records, and social media profiles.
- Your phone number is now linked to scam attempts, robocalls, impersonation attempts, and account takeover attempts.
Key Insight:
Data breaches don’t just expose one system. They feed a larger data economy where fragments are recombined.
Maintaining FERPA Compliance with AI
Knowing that personal data exposure carries long-term consequences requires educators to be responsible stewards of student information. FERPA is a federal law that protects the privacy of student education records. At its core, FERPA gives families and eligible students rights regarding access to and disclosure of information directly related to a student and maintained by an educational agency or institution. In AI contexts, FERPA considerations arise when educators input student-related information into tools that are not covered by district agreements or when data is shared beyond approved systems. Even well-intentioned uses—such as pasting a student response into an AI tool to generate feedback—can raise compliance concerns if the tool stores, processes, or reuses that information outside district control.
School districts vet tools to ensure alignment with FERPA, state guidance, and district values. It also promotes equity by ensuring that all students interact with the same protected systems rather than a patchwork of individual tools. The following table raises awareness of what is or is not happening with student data when a teacher uses a free instance of a tool.
Using a free tool may feel faster or more flexible, but it often transfers privacy responsibility from the division to the individual educator—sometimes without the educator realizing it.
If you are interested in equipping your staff with knowledge about the impacts of AI on student data privacy, diving deeper into this information, and exploring other data privacy topics like anonymizing data, setting classroom norms that keep students’ data safe, and communicating to families about AI and student data, we would love to hear from you. Let ALP help your district equip educators with AI literacy through relevant, competency-based, professional learning courses.
AI Transparency Statement: AI was used to draft parts of this blog with provided research and context. AI-generated text was edited by the author.
By: Rachel Fruin and Shirin Mathew
References
- Barbaro, Michael, and Tom Zeller Jr. “A Face Is Exposed for AOL Searcher No. 4417749.” The New York Times, 9 Aug. 2006, https://www.nytimes.com/2006/08/09/technology/a-face-is-exposed-for-aol-searcher-no-4417749.html.
- U.S. Department of Education. “FERPA.” Protecting Student Privacy, Student Privacy Policy Office, https://studentprivacy.ed.gov/ferpa.
