Character.ai Expands Safety Measures With AI and Partnerships

By Blockchain News | Created at 2026-09-03 18:23:52 | Updated at 2026-09-03 18:58:58 48 minutes ago

Character.ai Expands Safety Measures With AI and Partnerships

Character.ai is doubling down on its commitment to user safety with a series of updates aimed at tackling some of the most pressing challenges in AI-driven entertainment. The company announced its latest measures on September 3, emphasizing advancements in self-harm detection, moderation transparency, and controls for under-18 users.

Last year, Character.ai made headlines by removing open-ended chat capabilities for users under 18—a controversial move the company still views as one of its most significant safety decisions. Now, it’s building on that foundation with new tools and partnerships designed to address both user protection and community feedback.

AI-Driven Self-Harm Detection

One of the most notable updates is the refinement of self-harm detection systems. Character.ai's safeguards now analyze entire conversations for gradual signals of distress, rather than reacting to isolated messages. This approach aims to provide struggling users with pathways to real-world help, rather than abrupt interruptions.

To strengthen this initiative, Character.ai has partnered with organizations like Koko and ThroughLine. Koko offers free, self-guided emotional support tools, while ThroughLine connects users to localized crisis resources across the globe. These collaborations reflect a broader trend in tech, where companies are integrating mental health expertise into their platforms to proactively mitigate risks.

Moderation Transparency and User Controls

Character.ai is also addressing community concerns over content moderation. Creators will now receive notifications when their content is moderated, along with explanations for the decisions. An appeals process has been introduced, allowing creators to challenge moderation actions.

Additionally, the platform is giving users more control over their experience. New features allow users to block others, including their content and interactions, effectively creating personalized filters for a safer online environment.

Safety for Under-18 Users

The company continues to refine its safeguards for users under 18, deploying proprietary age-assurance technology to ensure accurate age verification. For parents, Character.ai has enhanced its Parental Insights tool through a partnership with k-ID, a leader in age and identity assurance. This tool provides weekly updates on user activity, offering greater visibility into how teens interact with the platform.

Global Partnerships to Bolster Safety

To tackle broader online safety risks, Character.ai has joined forces with organizations like ConnectSafely, Internet Watch Foundation, and StopNCII. These partnerships bring expertise in areas such as preventing child exploitation, non-consensual image abuse, and fostering digital wellness. By aligning with these global initiatives, the company is extending its safety net beyond its own systems.

Industry Context: Safety as a Competitive Advantage

Character.ai’s safety-focused updates reflect a broader trend in the tech and crypto sectors, where proactive risk prevention and transparency are becoming key differentiators. Earlier this year, Uber highlighted "Stand for Safety" as a core value in its 2026 governance report, while CompScience launched an AI-powered Safe Work Plan platform to mitigate workplace risks in April.

For Character.ai, these updates may not only protect users but also position the platform as a leader in responsible AI innovation. As the industry grapples with the challenges of rapid technological change, safety initiatives like these could become essential for earning user trust and maintaining regulatory compliance.

While AI continues to evolve, so do the risks. Character.ai’s ongoing investments in safety, transparency, and partnerships signal its intent to shape the future of AI entertainment responsibly.

Image source: Shutterstock