August 21st, 2024

Google's 'Reimagine' tool helped us add wrecks, disasters, and corpses to photos

Google's "Reimagine" AI photo editing tool enables users to add elements to photos via text prompts, raising concerns about misinformation due to the ease of creating disturbing, unidentifiable images.

Read original articleLink Icon
Google's 'Reimagine' tool helped us add wrecks, disasters, and corpses to photos

Google's new AI photo editing tool, "Reimagine," included in the Pixel 9 series, allows users to add various elements to their photos using text prompts. This feature extends the capabilities of the previous Magic Editor, enabling the addition of realistic and sometimes disturbing imagery, such as car wrecks and corpses. During testing, users found it alarmingly easy to bypass the tool's safeguards to create unsettling images. Google acknowledged that while they have policies to prevent misuse, the effectiveness of these guardrails is limited, and the potential for abuse is significant. The AI-generated images lack clear identification markers, making it difficult to distinguish them from authentic photos. This raises concerns about the rapid advancement of photo manipulation technology outpacing the ability to detect and regulate misleading content. The ease of creating and sharing such images could lead to widespread misinformation, prompting a call for increased skepticism regarding the authenticity of online visuals.

- Google's "Reimagine" tool allows for the addition of realistic elements to photos using text prompts.

- Users can create disturbing imagery by creatively bypassing the tool's safeguards.

- The lack of clear identification for AI-generated images raises concerns about misinformation.

- The rapid advancement of photo editing technology outpaces detection and regulation efforts.

- Increased skepticism is advised when evaluating the authenticity of online images.

Link Icon 17 comments
By @rqtwteye - 3 months
This is starting to be silly. What's next?

- "Google Docs" allowed me to write death threat letters.

- My Brother printer allowed to print them.

- The postal service delivered them

- My Sony camera allowed me to take nude pictures of my neighbor through the bathroom window

We can't safeguard every tool. And I predict negative consequences will come from trying.

By @multimoon - 3 months
The tool is doing exactly what you asked it to do, and being surprised about that is silly.

Censorship is never a good thing.

By @flappyeagle - 3 months
This is the dumbest genre of article ever conceived. I can't begin to understand the mental confusion needed to motivate someone to write it.

What are they objecting to? Art? I can look at disturbing imagery by closing my eyes and imagining it. Let's ban my visual cortex.

Stuff like this gives journalists a bad name; it's selfish. It erodes trusts in the institution of the press for nothing more than a deadline and some clicks.

By @UtopiaPunk - 3 months
Given we live in a period of rampant mis-information and general media ill-literacy, it's difficult for me to imagine this tool being a net-positive for our societies. On the one hand, such tools can be used to generate false images, as the article demonstrates. On the other hand, the existence and widespread availability of such tools will bring much more doubt and skepticism for any photos that challenge one's beliefs or the status quo. Are you trying to show me photographic evidence to prove to me that something is true? Well, now I handwave it away as probably an AI generated image.

Maybe something will break, and the general population will become excellent at citing and verifying sources as a response to rampant fakes. However, given the generally sorry state of news and journalism, and seeing how many people on social media believe that AI slop is real, I'm skeptical.

By @paxys - 3 months
I know it's trying to do the opposite but this article comes off as a great ad for the feature. All those photos look great.
By @gotoeleven - 3 months
To be a bit of a polly anna, why is everyone so scandalized by AI tools that can be used to create bad things? Photoshop can, too. So can a paintbrush. No one would want to buy an electronic paintbrush that prevents you from painting particular images, so why is this so different? Just because it is easy and gives quality results?

We're basically already at the point where images and videos of unknown provenance can't be assumed to be real so how come people pay attention to journalists getting the vapors about scandalous things AI tools can do? Wouldnt everyone rather have a completely unlocked tool to do with as they will?

By @renewiltord - 3 months
One of the things we learned from previous business is that it’s better not to give journalists access to your things. If you can greyball them you should. It’s harder when your offering is a consumer SaaS app, but if you have bigger enterprise deals it’s rarely beneficial.

They are not very smart people, in general, but very good at optimizing for the thing that gets them views: ragebait.

In this case, there’s nothing to be done for it. Ideally, Google spins off image models to a separate company that doesn’t hurt the brand.

The rest of us will have this tool. But perhaps it’s too much for the normies.

By @sowbug - 3 months
As a person who recently discovered that aphantasia is a thing, and that I have it, I am troubled that most of you have the ability to create disturbing imagery in your minds.

I will be requesting the addition of safeguards for everyone's protection.

By @SirMaster - 3 months
The examples look like average photoshops we have been seeing for well over a decade at this point.
By @kernal - 3 months
>The new feature on the Pixel 9 series is way too good at creating disturbing imagery — and the safeguards in place are far too weak.

Yes, let's kneecap it because it's way too good. Safeguards just make users migrate to other services to generate what they want.

By @krunck - 3 months
And this is just a consumer product. Just think what a nation or corporation could do with a meager budget. News(from centralized sources) is dead. We can't trust images, audio, or video any more.
By @denysvitali - 3 months
If anything this honestly looks like a great ad for this feature.

I think this shouldn't be newsworthy - the tool is just doing what you asked. It's the same as complaining with $pencil_producer that their pencils allowed you to draw disturbing images.

I think it would be more "newsworthy" if it would produce racist outcomes (e.g: asking it to draw a criminal and the tool produces always the same minority / output), but we're also probably past that - we've already seen those news articles.

By @foota - 3 months
This sort of "worst intentions" stuff is FUD. Sure, you can leak some potentially embarrassing photos of someone with doctored drugs on the floor.

So what? I can Photoshop some powder into a picture too. It might look better, but not really that much. I think the media needs to accept that images are no longer trustworthy unless there's some chain of evidence tied to them.

I can say "John was on the floor with a bucket of cocaine", that doesn't make it true.

By @yieldcrv - 3 months
my new hobby has been making AI videos of TED Talk-esque "Creatives" passionately saying "AI has no soul and people will always be able to tell"

and watching all the luddites on social media agree with the genAI person

By @retrochameleon - 3 months
It's going to be nearly impossible to validate wartime journalism now, and it's can easily be weaponized for misinformation.
By @jrockway - 3 months
I'm trying to get upset about a tool that lets you photoshop a smoking trash can onto a sidewalk, and it's just not happening.

I feel like this is similar to all technical progress. Once, only dedicated wizards could do something. Then, they are outraged when the general public can do it.

Misinformation sucks, but restricting access to photo tools is not the solution. Better education is. It's the solution to pretty much all problems. (And even then, people aren't as dumb as you may think. Trump is heavily using AI photos to claim that people are endorsing him, and I don't think anyone thinks that Taylor Swift is actually cosplaying Uncle Sam and endorsing Trump.)