AI porn imagery sparks new problems for users and platforms

In a completely unsurprising evolution of events, AI image generators have started to be used in the creation of AI porn, illegal sexual images and deepfakes pornography.

Now, both the owners of these platforms and in some cases, law enforcement are fighting back.

In the most recent and milder instance, Midjourney, one of the most popular of these platforms is now reacting with a rather draconian step: The text-to-image generator is simply banning any words about the human reproductive system during image generation.

According to a recent report from the MIT Technology Review, this ban is just a temporary measure enacted by Midjourney to stop users from rendering shocking, gory, or explicitly sexual images.

Because of this ban, user word prompts like “placenta,” “fallopian tubes,” “mammary glands,” “sperm,” “uterine,” “urethra,” “cervix,” “hymen,” or “vulva” will trigger a block.

This block might be of just a word’s use in a specific instance, or in some cases of persistent users, it can trigger a ban of that user for a certain time.

According to David Holz, founder of Midjourney, in comments to MIT Technology Review, the ban is provisionally in place against the creation of violent and graphic sexual imagery until the independent research lab “improves things on the AI side.”

Holz also elaborates that his organization’s moderators track word use and the resulting images for the sake of modifying their prompt bans accordingly.

For the time being, it seems that using Midjourney for detailed non-sexual anatomical image generation is out of the question too.

Midjourney also has a community standards page where it lists other words that are banned too. Phrases and words that don’t involve reproductive organs can still be used though.

This case demonstrates what should have been obvious from the start: if you create an easy way to generate high-quality, realistic images with minimal effort, someone will use it to create porn.

That alone isn’t necessarily so terrible, even if some generally object to pornography for different reasons.

However, a deeper problem is when these same sexual image-rendering abilities go down much darker paths. This is likely what makes Midjourney worried enough to enact blanket word bans. It also brings us to our next two cases.

In one of these, pedophiles have started using AI image rendering programs to create sexual and sexualized images of the bodies of minors, and then editing the faces of real children (presumably culled from social media and other sites) onto them.

In a recent report in the Daily Mail, there has even been at least one arrest so far in a case of this.

The legal investigation happened in Spain, where a computer programmer was taken into custody in December of last year for creating child pornography with AI software.

According to the arresting investigators, the individual in question had a massive stash of images in which he’d used real photos of real minors to generate graphic sexual scenarios.

Though this has been one of the first arrests of its kind to date with this technology, it’s extremely likely that others are still doing the same thing undetected.

Finally, another recent case with lots of disturbing possibilities for future abuses involves a group of young Twitch live streamers discovering that photos of themselves had appeared in deepfakes porn websites.

The Twitch users found out that a number of websites were using AI-generated deepfakes to superimpose their faces onto the nude bodies of other people in sexually graphic videos and photos.

The quantity of these deepfakes pornographic videos and photos is also generally rising as AI becomes much more skilled at rendering images with minimal human effort.

According to one researcher, the number of pornographic deepfakes videos uploaded to one particular deepfakes streaming site rose from 1,897 pieces in 2018 to 13,000 by 2022.

These were receiving a total of more than 16 million views per month by the latter year. This will doubtlessly increase in 2023.

Midjourney’s problem with pornographic imagery is just the mildest expression of how AI and porn can combine in ugly ways.

The platform’s ban on words about reproductive organs may seem absurd at a glance, but the potential for bad publicity and moral quagmire via users creating blatantly illegal abuses with it is very real.

Highly Recommended

8 Tools for Photographers

Check out these 8 essential tools to help you succeed as a professional photographer.

Includes limited-time discounts.

Learn more here

You'll Also Like These:

Destin Sparks Nick Sparks How to Fix Common Lightroom Problems