December 2024 | This Month in Generative AI: Reimagining Your Photos

A photograph created using Google Pixel 9 including a subject added through the mobile phone's AI editing tool. Photo credit: Hany Farid

News and trends shaping our understanding of generative AI technology and its applications.

For the past 25 years, my students and I have been developing techniques to detect all forms of digital manipulation. For almost as long, I've been asked the same question: "Will we get to a point where we won't be able to tell if something is real or fake?" My usual response is, "I don't know, but it hasn't happened yet."

However, when I started taking photos and playing with the latest AI-powered on-camera photo editing capabilities of Google's new Pixel 9, I started doubting my stock answer.

Below are two examples where, after taking a photo (left), I specified a region in the image and typed "add a monster" and "add a car damaged by branches." Although not perfect, the results are pretty impressive. What makes this type of manipulation so difficult to detect is that most of the content is natural, and so general-purpose AI-or-not models trained to distinguish natural from AI-generated images will struggle. In addition, because all the modifications are happening directly in the camera app, the resulting manipulated image doesn't have some of the usual tell-tale signs that may emerge when using traditional photo editing software like Adobe Photoshop or an AI-powered photo editing application.

After creating dozens of images and analyzing them with a range of my forensic tools, I noticed an interesting and persistent artifact that can be used to detect and isolate parts of the image that were digitally modified or added. Shown below is the result of this analysis on the monster image, where the blue-tinted region is automatically identified as having been modified after capture. (The shaded regions in the sky correspond to parts of the image that could not be analyzed because they are largely uniform in intensity.)

For now, I'm not going to publicize how I’m performing this analysis. Maybe this will give me a little more time to continue to say, "I don't know, but it hasn't happened yet."

I also discovered that Google attaches some metadata to AI-modified images, which is nice as long as the metadata doesn't get stripped out. This gives me the opportunity to repeat a request I have been making for 25 years to social media platforms: Please stop removing identifying metadata that would make it easier to understand the provenance of images and videos.

Google also recently announced the beta release of a watermarking toolkit, SynthID, which embeds into content an imperceptible signal that is designed to be resistant to simple modifications, allowing for downstream identification. It seems that these watermarks will be embedded into the full AI-generated images created by, for example, Imagen 3, but this feature – as I understand it – has not yet been rolled out.

The addition of a watermark and metadata are nicely aligned with the efforts of the C2PA to tag AI-generated or modified content at the point of creation.

As a side note, I was pleased to see that the Pixel 9 places some limits on the modifications that can be made. Although it allowed me to add damage to a car and building, it didn't allow me to add fire. I also wasn't able to remove clothing from a person (I tried on a photo of myself), a trend proliferating on social and messaging platforms.

I have no doubt that this same AI editing feature will soon roll out on all mobile devices. When it does, we are going to have to contend with a complex world that goes beyond just “AI or not?” and gets into more nuanced and complex questions: What parts of an image are manipulated, and how much manipulation is too much?

Author bio: Professor Hany Farid is a world-renowned expert in the field of misinformation, disinformation, and digital forensics. He joined the Content Authenticity Initiative (CAI) as an advisor in June 2023. The CAI is an Adobe-led community of media and tech companies, NGOs, academics, and others working to promote adoption of the open industry standard for content authenticity and provenance. Professor Farid teaches at the University of California, Berkeley, with a joint appointment in electrical engineering and computer sciences at the School of Information. He’s also a member of the Berkeley Artificial Intelligence Lab, Berkeley Institute for Data Science, Center for Innovation in Vision and Optics, Development Engineering Program, and Vision Science Program, and he’s a senior faculty advisor for the Center for Long-Term Cybersecurity. He is also a co-founder and Chief Science Officer at GetReal Labs, where he works to protect organizations worldwide from the threats posed by the malicious use of manipulated and synthetic information.

He received his undergraduate degree in computer science and applied mathematics from the University of Rochester in 1989, his M.S. in computer science from SUNY Albany, and his Ph.D. in computer science from the University of Pennsylvania in 1997. Following a two-year post-doctoral fellowship in brain and cognitive sciences at MIT, he joined the faculty at Dartmouth College in 1999 where he remained until 2019. Professor Farid is the recipient of an Alfred P. Sloan Fellowship and a John Simon Guggenheim Fellowship, and he’s a fellow of the National Academy of Inventors.