Instagram ‘is profiting from AI-generated child abuse images’: Charity lawyers target social media giant Meta for ‘not tackling’ a ‘new frontier of horror’ that sees paedophiles advertise websites selling vile deep-faked photos

Instagram is facing a legal challenge over claims it profits by allowing users to advertise AI-generated child sex abuse images.

It follows a police investigation which found that predators are using Instagram to promote websites selling the abuse material, which is available in just two clicks.

In what campaigners say is a ‘new frontier of horror’, paedophiles are using artificial intelligence software to generate thousands of sexualised images of children, which they brazenly market on Instagram, telling users to click on their sites to buy even more explicit material.

Law firm Schillings has launched a ‘groundbreaking legal challenge’ against social media giant Meta – which owns Instagram, Facebook and WhatsApp, on behalf of the 5Rights Foundation children’s charity.

It alleges that Instagram is ‘complicit in the exploitation of children online’ by hosting ‘content which puts both adult and child users at risk’. 

Paedophiles are using artificial intelligence software to generate thousands of sexualised images of children, which they brazenly market on Instagram, telling users to click on their sites to buy even more explicit material.

It cites a police dossier of evidence which ‘clearly shows that AI-generated sexualised images of children are widespread on Instagram, and that Instagram is connecting users with illegal content’.

Baroness Beeban Kidron, founder of 5Rights Foundation, which campaigns for a safe digital environment for children, vowed to pursue Meta through the courts if it does not act, warning: ‘It’s really easy to start on Meta platforms and end up in Hell two clicks later.’

It comes as a separate report today reveals that public reports of AI-generated abuse imagery across the open web have quadrupled in just six months, following a terrifying new trend in which offenders steal photographs of innocent children from social media and use software to ‘nudify’ victims, manipulating the imagery to fulfil their twisted fantasies.

The Internet Watch Foundation report warns that Britain is at a ‘frightening tipping point’ as AI tools are now able to generate images and videos so realistic that it is impossible to distinguish real from fake, making it difficult for officers to rescue victims.

Baroness Kidron said: ‘Seeing these sexualised images made me want to cry. It’s very young girls scantily clad in suggestive poses. 

‘AI child sexual abuse material is creating a new frontier of horror and Instagram is enticing people there and enabling users to access child sexual abuse material.

‘Children are being exploited and Meta is helping to grow the appetite for this material. 

‘If your business is to keep people clicking and you don’t give a monkey’s what they are clicking on, that is wilful blindness because the ultimate motive is making money.’

When undercover police officers started investigating in December, they found dozens of accounts with names such as ‘pervy kinks’ featuring AI-generated sexualised images of young children.

Alongside the partially naked pictures were links to pay-per-view websites and encrypted instant-messaging Telegram channels featuring footage of real children being raped.

During the six-month probe, Instagram’s algorithms recommended many similar accounts to officers, leading to fears that children stumbling on the sites may be directed into the clutches of child abuse gangs. 

Schillings said the openness of the material was ‘legitimising’ child abuse because paedophiles can network in plain sight without having to hide on the dark web. 

To understand how it works, Baroness Kidron permitted police to access photographs of herself, and they were able to show how AI created abuse images of her as an eight-year-old. ‘

It was horrendous to see these images, I can no longer look at pictures of myself as a child and not see these images,’ she said.

In January, officers flagged four offender profiles through Instagram’s in-app reporting function, but nothing happened.

Police then made an official information request regarding the accounts, but received no response from Meta. 

Schillings submitted a file of evidence last month, demanding an independent investigation. 

In a statement, Meta said that ‘all violating accounts’ had been removed earlier this year

Meta failed to respond until contacted by the Mail, when it claimed it never got the email. It has since removed all the accounts. 

Meta said: ‘All violating accounts were removed by us earlier this year. Whether it’s AI or an actual person, child exploitation of any kind is horrific and we have clear rules against it.

‘We regularly report apparent instances of this content to the National Centre for Missing and Exploited Children.

‘We also share leads with other companies to fight cross-platform abuse, and support law enforcement in its efforts to arrest and prosecute the criminals behind it.’