Meta is changing the way it trains its AI chatbots to prioritize teen safety. This decision follows an investigative report that highlighted the company’s lack of AI safeguards for minors. The company will now train its chatbots to avoid engaging with teenage users on topics related to self-harm, suicide, disordered eating, and potentially inappropriate romantic conversations. Meta describes these as interim changes, with plans to release more robust and long-lasting safety updates for minors in the future.
A spokesperson, Stephanie Otway, acknowledged that the company’s chatbots were previously able to discuss these topics in ways Meta had deemed appropriate. The company now recognizes this was a mistake. Otway stated that as the community grows and technology evolves, Meta is continually learning how young people interact with these tools and is strengthening its protections accordingly. The company is adding more guardrails as an extra precaution, including training its AIs not to engage with teens on these sensitive topics but to guide them to expert resources instead. For now, teen access will also be limited to a select group of AI characters. These updates are already in progress, and Meta will continue to adapt its approach to help ensure teens have safe and age-appropriate experiences with AI.
Beyond the training updates, the company will limit teen access to certain AI characters that could hold inappropriate conversations. Some user-made AI characters available on Instagram and Facebook include sexualized chatbots with names like “Step Mom” and “Russian Girl.” Instead, teen users will only have access to AI characters that promote education and creativity.
These policy changes come just two weeks after a Reuters investigation uncovered an internal Meta policy document that appeared to permit the company’s chatbots to engage in sexual conversations with underage users. The document included example passages of acceptable responses, such as “Your youthful form is a work of art,” and “Every inch of you is a masterpiece – a treasure I cherish deeply.” Other examples showed how the AI tools were to respond to requests for violent imagery or sexual imagery of public figures.
Meta says the document was inconsistent with its broader policies and has since been changed. However, the report sparked sustained controversy over potential child safety risks. Shortly after the report was released, Senator Josh Hawley launched an official probe into the company’s AI policies. Additionally, a coalition of 44 state attorneys general wrote to a group of AI companies, including Meta, emphasizing the importance of child safety and specifically citing the Reuters report. The letter stated they were uniformly revolted by this apparent disregard for children’s emotional well-being and alarmed that AI Assistants were engaging in conduct that appears to be prohibited by their respective criminal laws.
Otway declined to comment on how many of Meta’s AI chatbot users are minors and would not say whether the company expects its AI user base to decline as a result of these decisions.