An Australian AI pornography website has been barred from advertising on the world’s largest adult platform and targeted by Australia’s online safety regulator after the ABC revealed it could be breaching laws against child exploitation material.
Warning: This story includes references to child abuse material.
OurDream, which has been ranked as the biggest AI porn site on the internet and claims to have more than 20 million customers, offers a platform to “build [an] AI girlfriend from scratch” for uncensored interactions via text, pictures and video.
The site, which is run by the Sydney-based company Coreflow, says it prohibits any characters that appear to be under 18 years old.
This image, which has now been removed, appeared on the profile of a character on OurDream’s website. (ABC News)
But an ABC investigation found it featured some user-generated characters that appear as young as 14, according to an age-estimation tool used by law enforcement.
Experts told the ABC that this was child sexual abuse material, and Queensland police are now investigating the site.
Under Commonwealth law, child sexual abuse material is defined as material that is likely to offend a reasonable adult, and is a sexualised depiction of a person, or a representation of a person, who is or appears to be a child.
An age-estimation tool, Rigr, assessed this character’s age which appeared on the site to be 15.1, with a margin of error of 1.1 years. (ABC News)
The definition includes AI-generated images.
Pornhub, the world’s largest adult site, has now suspended all advertising from OurDream following the ABC’s reporting.
Aylo, Pornhub’s Canadian owner, said the International Centre for Missing and Exploited Children Australia (ICMEC) had “alerted us to concerns regarding certain content generated by the OurDream platform”.
“While the advertiser had undergone a first onboarding review by Aylo prior to their publication of any advertisements, the allegations were serious enough that Aylo determined the pausing of all advertisements, while it conducted a second, deeper review, was appropriate,” Aylo said.
The move is a setback for Coreflow, which a company insider not authorised to speak publicly told the ABC has spent $1 million advertising on Pornhub.
Coreflow did not respond to questions about its advertising spend or the ban.
Pornhub has suspended advertising from OurDream following the ABC’s reporting. (ABC News: Graphic by Jarrod Fankhauser)
ICMEC chief executive Colm Gannon said he reported the issue to Aylo.
“When we talked to [Aylo], the one question was, ‘Is this [an AI] depiction of real children or is it a depiction of anime?'” he said.
“And I went, ‘No, it’s a depiction of real children.’
“They went, ‘Gone’, like literally within seconds. It wasn’t even open to discussion.“
Code leaked to the ABC
Meanwhile, the ABC has obtained leaked copies of AI coding used by Coreflow to detect and screen underage characters on OurDream.
A company insider told the ABC the coding revealed systemic flaws in Coreflow’s safeguards against child exploitation material.
It shows the company uses a Chinese AI model, Qwen, to flag potentially illegal images generated by users.
However, the prompt given to the AI tool asks it to assess “how likely … any person depicted appears to be under 15 years old” rather than 18 years old.
Under Commonwealth law, sexual depictions of people who look under the age of 18 are illegal.
The AI was instructed to put images into categories that included:
- “probably adult (18+), but may be between 15-18 (some ambiguity)”
- “probably under 18, but is likely between 15-18 (ambiguous)”
Leaked coding suggests the site has used AI to focus on whether characters look under 15 years old, rather than 18. (Supplied)
Coreflow declined to answer questions about how it dealt with images in each category, or why the AI prompt was for 15 years, not 18.
The company insider, who spoke on condition of anonymity because they were not authorised to comment publicly, said it was likely the AI did not flag some “ambiguous” results.
This explained why the ABC found characters that appeared 14 and 15 years old, according to specialised age-estimation technology, they said
The company insider said setting the limit at 15 years was likely a technical ploy by the company to stop the AI rejecting “a significant amount of content intended to be 18”.
The company uses an under-18 filter when checking customer text prompts to build new characters.
When site users build new characters through text prompts, AI flags any phrase suggesting under 18 years. (Supplied)
However, it was possible to describe an underage character that passes this text filter, the company insider said.
“The problem comes after images are generated,” they said.
“It is actually a very hard and expensive technical problem to solve to generate AI porn that appears 18 but never appears any younger. It appears that Coreflow has taken shortcuts to grow quickly.”
A Coreflow spokesperson said the company “categorically rejects any suggestion that our platform condones or facilitates unlawful conduct”.
“OurDream does not publish content featuring characters that our systems determine are at risk of being under the age of 18,” they said.
The spokesperson said the site has “zero tolerance for requests to generate content that falls outside our strict safety standards”.
“OurDream is designed to operate in compliance with all applicable laws, and we continually review and strengthen our safeguards, including our moderation systems.”
Of the 17 AI-generated characters on OurDream that the ABC identified as looking underage, OurDream has removed 11 of them.
Age estimator used is ‘rubbish’, expert says
ICMEC’s Mr Gannon, who previously worked for Rigr AI, a company that created an age-estimation tool used by governments and law enforcement globally, said Coreflow’s technology was “not fit for purpose”.
The Qwen AI model, he said, was “not a ‘product-grade’ age-estimation tool” and had a median absolute error rating (MAE) of plus or minus six years.
Mr Gannon said this margin of error could mean a character that appears to be 15 years old would be wrongly judged by the technology to be up to 21 years old.
These were among the characters that appeared on the site an age-estimation tool judged to be under 18. (ABC News)
“It’s rubbish. There’s literally no point of why you’d actually plug that in,” he said.
“There’s no doubt that this is actually going to generate characters under 18.“
Mr Gannon said the leaked coding showed other serious flaws.
This included OurDream’s approach to monitoring user conversations with characters for any content suggesting the “sexual involvement of individuals under 18”.
The “key principles” for the AI detection were to “be lenient towards borderline content” and “if unsure, do NOT flag”.
Leaked code for OurDream tells AI to “be lenient towards borderline content” when checking chats for underage content. (Supplied)
Mr Gannon said not escalating this content to human moderators showed a company trying to “make sure the experience of the end user is of the utmost importance, as opposed to the trust and safety protocols and the ethical design of AI technology”.
“ What they’ve done is they’ve actually created the borderline content by the [AI] model they’re using and also the rules they’re using,” he said.
“This is a software that’s not safety by design; it’s the complete opposite.”
Coreflow said Qwen was just one of multiple detection layers and age-estimation tools it uses, and that it did not on its own determine moderation outcomes.
The company said it was now using a more advanced model.
Regulator reviewing OurDream
Following the ABC’s reporting into the material found on OurDream, the website is now on the radar of Australia’s online regulator, the eSafety Commissioner.
The agency said it contacted the site and “made OurDream aware of its obligations under the Online Safety Act’s codes and standards,” a spokeswoman said.
“These obligations apply in relation to both unlawful material such as child sexual exploitation material, as well as material that is age-restricted such as online pornography.
“eSafety is considering OurDream’s compliance with its obligations along with a wide range of other generative AI services.”
A menu of characters appears when you open OurDream. (Supplied)
Those obligations include “safety by design”, which “encourages technology companies to alter their design ethos from ‘moving fast and breaking things’ or ‘profit at all costs’ to ‘moving thoughtfully’, investing in risk mitigation at the front end”.
The spokeswoman said eSafety had the power to “receive complaints about and take action against synthetic forms of image-based abuse and child sexual exploitation material”.
“If a company fails to comply with online safety codes and standards, eSafety can pursue further regulatory action, including seeking civil penalties of up to $49.5 million,” she said.
“We take the generation of [child sexual abuse material], whether real or synthetic, very seriously and will take steps necessary to hold those companies accountable under Australian law.“
A spokesperson for Coreflow said the “safe and lawful use of our OurDream platform is Coreflow’s highest priority”.
“We employ a robust, multi-layered safety system that combines proprietary technology with best-in-class external AI content moderation tools, as well as human-in-the-loop for exception processes, to detect and prevent inappropriate and unlawful conduct on the OurDream platform,” they said.
