The images travel the way shameful things have always travelled — in private. A Google co-authored study of internet users in the United States, Australia and the United Kingdom found this July that non-consensual AI-generated explicit content spreads largely through texts, direct messages and private clouds, where it is rarely detected or reported. The study’s conclusion put the burden where the wound begins: the “most critical intervention point” is the source, the tools that fabricate the pictures.
Which makes the following awkward for the companies that build those tools. As Ashley Belanger reported in Ars Technica, red-teaming in recent months suggests that as flagship image generators from Google and OpenAI grow more capable, they may be getting worse at refusing explicit output. The alarm comes from White Circle, a public benefit corporation backed by senior leaders and researchers from Anthropic, DeepMind, Hugging Face and OpenAI, founded after its CEO, Denis Shilov, posted one of the most famous universal jailbreaks of leading models in October 2024. Shilov finds the latest results “horrifying” enough to publish.
The attack that iterates
In a little-noticed preprint published last month, White Circle’s researchers went after the guardrails of DALL·E 3, Google’s Nano Banana 2 and GPT-Image-2. Using what they describe as “just off-the-shelf tools, available online for anyone to download,” plus ordinary API calls, they bypassed the safety filters and produced banned explicit content about 13 percent of the time — a rate they verified by reviewing outputs and call “the highest of any method tested under the same standard.” “Critically, none of these requests required custom-built or frontier AI models,” the company warned. No expertise is needed to repeat the attacks. (Midjourney went untested; it offers no public API suitable for controlled benchmarking.)
The method has a name — Red-teaming via Iterative Strategy Evolution, or RISE — and a simple architecture of paired adversaries. An attacking model, a Qwen downloaded from Hugging Face, generates jailbreaks; a judge model, Google’s Gemini 3.8 Flash, checks whether the result crossed the line. Instead of testing prompts one at a time, RISE hunts for patterns that work across many prompts and scenarios, yielding jailbreaks that are reusable and transferable between leading models. “You don’t need to have frontier models to break defenses of a frontier model, which is very interesting,” Shilov said.
Then the follow-up finding, which cuts deeper. After OpenAI and Google shipped their newest models — ChatGPT Images 2.5 and Nano Banana 2.1 — White Circle ran the same method against them and found “both are significantly more breakable than their previous versions.” Neither company answered Ars Technica’s questions, on the first findings or the second. White Circle’s working hypothesis: “either the newer models are more sophisticated and capable of generating a wider range of content, or that safety has been deprioritized in favor of capability.”
The testers admit their craft is young. Research engineer Dmitrii Kharlapenko told Ars that red-teaming image generators is “very poorly calibrated,” with “literally zero papers” establishing norms for the field — one reason, at a moment when labs race to release ever more eye-catching models, that the area remains “completely plagued with reporting issues.”
The watchdog that would not sign
Shilov’s remedy has the architecture of arms control: mandatory external safety testing, plus third-party monitoring “much like the system used to prevent nuclear threats,” run by a coalition that could detect AI threats in real time without bias. He notes the obstacle — nearly everyone doing interesting safety work ends up contracted to the frontier labs themselves. White Circle once joined Anthropic’s bug bounty program and found it “very constrained in its purpose”; work like the RISE paper, the company says, certainly would not have qualified. When OpenAI and Anthropic argued on Tuesday that outside evaluators should be bound by restrictive non-disclosure agreements to guard trade secrets, Reuters reported, White Circle answered that it would not sign anything that could block public release of critical findings. “They should be investing much more into safety measures and security measures around their models,” Shilov said. “And for us, it seems that they’re not actually doing this.”
White Circle plans to open-source its entire attack pipeline, so the model makers can measure the hole themselves — a welcome change of posture, given how recently researchers found major providers serving the sites that trade in deepfake abuse. The findings will be presented in November at NeurIPS 2026, which may put Google and OpenAI under a colder light than any they have answered so far. The company says it has shared its results with some frontier labs through “mutual contacts.”