Roblox Open-Sources AI Safety Tools to Improve Online Safety

If you’ve ever wondered what actually stops a stranger from sliding into your kid’s Roblox chat and asking for their phone number, the answer is a set of AI models running quietly in the background — and Roblox just handed three of them to the rest of the internet for free. On August 19, 2026, Roblox open-sourced updated versions of its PII Classifier, its early-warning system Sentinel, and its voice safety classifier through the Robust Open Online Safety Tools (ROOST) community, along with a new benchmark dataset. Here’s what each tool actually does, how much better they’ve gotten, and why a platform under constant safety scrutiny is giving its detection tech away.

By the LemonLama Editorial Team · Fact-checked against Roblox’s official newsroom post and ROOST reporting · Published August 21, 2026

What did Roblox just open source?

Roblox released updated versions of three AI safety models it already runs on its own platform, plus a brand-new evaluation dataset. Naren Koneru, Roblox’s VP of Trust and Safety Engineering, announced the release as a contribution to the ROOST Model Community, a shared library of open-source safety tools that other platforms can download, test, and adapt for their own moderation systems.

The three tools, in plain terms

• PII Classifier v2.0 — catches attempts to share or request personal information, or to move a conversation off-platform.
• Roblox Sentinel v2 — flags early warning signs of potential grooming or child endangerment, before a conversation turns explicit.
• Voice safety classifier v3 — listens for rule-breaking speech in real-time voice chat and issues warnings or temporary suspensions.

How does the PII Classifier stop kids from being moved off Roblox?

A lot of online grooming doesn’t start with an explicit message — it starts with someone trying to get a child to switch to a less-monitored app. The Roblox PII Classifier is trained to detect players’ attempts to share or request personally identifiable information or move players to other platforms where protections may not be as robust, including efforts to bypass detection through misspellings, coded words, or obscure references to other platforms. Instead of just scanning for phone numbers or usernames, it reads the surrounding conversation for intent.

Roblox shared a simplified example of the kind of bypass attempt the model is trained to catch:

Neither message contains a raw link on its own, but read together they spell out a Discord invite. Version 2.0 is trained on conversational context rather than isolated messages, and on LLM-generated synthetic examples of complex bypass attempts and rare multilingual cases, which is how the supported language count scaled from 17 to 189. Roblox says this work raised the model’s F1 score from 63.41 to 90.52 — a meaningful jump in a metric that balances how many real violations get caught against how many false alarms get raised.

The same instinct that makes a PII Classifier useful — spotting when personal details are being handed over somewhere they shouldn’t be — applies just as much to your own accounts and passwords. If you’re managing logins for a household or a team, our guide on choosing the best password manager for your team in 2026 covers how to keep credentials out of the wrong hands in the first place.

Roblox also released a companion evaluation dataset, the PII Classifier Benchmark, built from synthetic multiuser chats that simulate phonetic bypasses, character substitution, coded language, and information deliberately split across multiple messages — plus “hard negative” examples of benign chats that merely look suspicious. The goal is to give other companies a realistic yardstick, since most existing PII datasets focus narrowly on named-entity extraction, which only covers a small slice of what actually shows up in real platform chats.

What does Roblox Sentinel actually detect?

Sentinel is the tool built specifically for the scariest scenario: an adult grooming a child through conversation that looks harmless on the surface. Built on contrastive learning, Sentinel is trained on patterns from both benign and eventually harmful conversations, then compares new conversations against those patterns to flag subtle warning signs for human review before they can escalate. Questions as ordinary-sounding as “how old are you” or “where do you live” can be red flags in context — just not in isolation, which is exactly why pattern-based detection matters more than keyword blocking here.

Human reviewers still make the final call — Sentinel lets them prioritize the chats most likely to need attention, so they can act on violating accounts and report cases to the appropriate authorities faster. Roblox first rolled Sentinel out in 2025, and in the first half of that year it says the system helped the company submit roughly 1,200 reports of potential child exploitation attempts to the National Center for Missing and Exploited Children, according to Roblox’s original Sentinel announcement.

The version released this week, Sentinel v2, mainly improves how the model scores and combines different risk signals. It expands from two combining functions to six, each suited to a different type of data, and adds new ways to evaluate which function fits a given use case — including explanations for why a conversation scored the way it did. Roblox says a configuration sweep across 324 setups that used to take nearly an hour now finishes in under three minutes, and the model’s ROC-AUC — a standard measure of how well it ranks risky conversations — rose from 0.894 with default settings to 0.996 with the best configuration.

How does the voice safety classifier work in real time?

Text chat isn’t the only place moderation has to happen — Roblox also runs live voice chat through an AI classifier. When a player breaks a rule, they get a notification explaining which policy they violated. Repeat violations can lead to voice chat being temporarily suspended for up to five minutes, with harsher consequences possible for larger violations or when other players file reports.

Version 3 of the classifier, released earlier in 2026, is the one Roblox is now contributing to ROOST. It moderates speech across 30 languages and eight violation categories, achieving 61% recall at a strict 1% false-positive rate across all supported languages, and adds built-in language detection. Getting there wasn’t free — the underlying model grew from 94.6 million to 320 million parameters, and Roblox used model distillation to keep it fast enough to run in real time.

Why is Roblox open-sourcing these safety tools now?

This isn’t Roblox’s first open-source safety release — it’s the latest step in a strategy the company started in 2024. Roblox first shared its voice safety classifier as an open-source model in 2024, describing it as part of a broader intent to share advances whenever they could increase safety and civility not just on Roblox but across the industry. In early 2025, Roblox became a founding partner of ROOST — Robust Open Online Safety Tools — alongside Google, OpenAI, Discord, and others, a nonprofit built to develop and distribute free, scalable safety tools for organizations that often lack the resources to build them from scratch.

There’s also real-world pressure behind the timing. Roblox has faced a wave of lawsuits alleging the platform failed to adequately protect children from predators, including a widely reported case in Iowa involving a 13-year-old girl. Sharing detection technology publicly is partly a safety strategy and partly a way for Roblox to demonstrate it’s serious about the problem — though it’s worth noting, as Roblox itself acknowledges, that releasing a tool doesn’t guarantee other platforms will actually adopt and properly tune it.

Roblox isn’t the only company folding AI into its safety and creative pipeline — if you’re curious how generative AI tools work more broadly, our explainer on what Leonardo AI is and how it’s used is a good starting point for the wider AI-in-gaming landscape these safety classifiers sit inside.

Frequently asked questions