Blog

Part 5: Photo forensics from lighting shadows and reflections

by Hany Farid

In the past few posts, I have described representative examples of passive forensic techniques. This category of tools for distinguishing the real from the fake assumes no explicit knowledge of the content source or specialized recording equipment.

The benefit of these techniques is that they are applicable to a broad category of content. The drawback, however, is they sometimes require manual intervention. The shadow and reflection analysis, for example, requires identifying corresponding points on an object and shadow or reflection. While this is a relatively simple step for a human analyst, it is still out of reach of automation. As a result, these techniques cannot be applied to the torrent of daily online uploads.

Active forensic techniques, on the other hand, operate at the source of creation, embedding into or extracting from an identifying digital watermark or signature. There is a long history of marking content to prove authenticity, indicate ownership, and protect against counterfeiting. For example, Getty Images, a massive image archive, adds a visible watermark to all digital images in their catalog. This allows customers to freely browse images while protecting Getty’s assets.

An “invisible” or steganographic watermark can be added to a digital image by, for example, tweaking every 10th image pixel so that its color (typically a number in the range of 0 to 255) is even-valued. Because this adjustment is so minor, the watermark is imperceptible. And, because this periodic pattern is unlikely to occur naturally, it can be used to verify an image’s provenance.

The ideal watermark is one that is imperceptible and also resilient to basic manipulations like cropping or resizing. Although the above pixel manipulation is not resilient to, for example, image resizing, many robust watermarking strategies have been proposed that are resilient — though not impervious — to attempts to remove them.

The benefit of watermarks is that identifying information is directly associated with a piece of content before it is released into the world, making identification fast and easy. The drawback to this approach is that watermarks are vulnerable to attack, where an adversary can digitally remove the watermark while leaving the underlying content largely intact.

Therefore, in addition to embedding watermarks, a creator can extract an identifying fingerprint from the content and store it in a secure centralized ledger. (This fingerprint is sometimes called a perceptual hash — see article 2 in Further Reading below.) The provenance of a piece of content can then be determined by comparing the fingerprint of an image or video that is out in the world to the fingerprint stored in the ledger. Both watermarks and fingerprints can be made cryptographically secure, making it very difficult to forge.

This type of watermarking and fingerprinting is equally effective for actively tracking AI-generated and human-recorded content. The benefit of this approach is that it can efficiently scan the billions of online daily uploads. The drawback is that it requires specialized hardware or software to be used at the point of creation and recording. It also requires downstream platforms like YouTube, Twitter, and TikTok to respect the watermarks, and it requires browsers to display the accompanying content labels. Although not perfect, these combined technologies make it significantly harder to create a compelling fake and significantly easier to verify the integrity of real content.

Importantly, this active approach is agnostic as to the truthfulness or value of content. The goal of this approach is simply, but critically, to distinguish between human-recorded and AI-generated content.

A mockup of a social media feed that integrates Content Credentials viewing and interaction. Content Credentials activate the C2PA standard for content authenticity and provenance.

I started this series by talking about my initial journey some 25 years ago into the world of media authentication and forensics, and so I'll end with some thoughts of what may lie ahead.

A combination of many factors has contributed to a complex and at times chaotic online information ecosystem. With democratized access to powerful editing and creation software, the widespread deployment of generative AI is poised to significantly complicate this environment. Harms ranging from the creation of non-consensual sexual imagery to small- and large-scale fraud and disinformation have already emerged even in these early days of this AI revolution.

Legitimate concerns have also been raised regarding creators' rights and their very livelihoods. Here we can draw on some historical analogs. The introduction of photography did not — as some 19th-century artists feared — destroy art or artists. It disrupted some aspects like portrait paintings, but it also liberated artists from realism, eventually giving rise to Impressionism and the Modern Art movements.

Similarly, the digitization of music did not kill music or musicians, but it did fundamentally change the way we produce and listen to music, and it spawned new genres of music. This history suggests that generative AI does not necessarily have to lead to the demise of creators, but that it can, with proper care and thought, be a new enabling medium.

There is much to be excited about during these early days of the AI revolution. At the same time, there are legitimate and real concerns as to how these new technologies are being weaponized against individuals, societies, and democracies, and how they might disrupt large sections of the workforce. Advances in AI do not need to lead us into an unrecognizable dystopian future. But, technologists, corporate leaders, and regulators need to carefully consider how they develop and deploy technologies, and build in proper safeguards from the point of conception through rollout.

Further reading:

[1] H. Farid. Watermarking ChatGPT, DALL-E and Other Generative AIs Could Help Protect Against Fraud and Misinformation, The Conversation, March 27, 2023.

[2] H. Farid. An Overview of Perceptual Hashing. Journal of Online Trust and Safety, 1(1), 2021.

[3] Z. Epstein, et al. Art and the Science of Generative AI. Science, 380(6650):1110-1111, 2023.

More from this series:

Part 5: Photo forensics from lighting shadows and reflections

Author bio: Professor Hany Farid is a world-renowned expert in the field of misinformation, disinformation, and digital forensics. He joined the Content Authenticity Initiative (CAI) as an advisor in June 2023. The CAI is a community of media and tech companies, non-profits, academics, and others working to promote adoption of the open industry standard for content authenticity and provenance.

Professor Farid is the recipient of an Alfred P. Sloan Fellowship and a John Simon Guggenheim Fellowship, and he’s a fellow of the National Academy of Inventors.

Receive updates and community news. Consider joining the CAI as a member.

Stay connected on LinkedIn and Twitter.
Explore the CAI’s open-source tools, powering Content Credentials, verifiable details or digital “nutrition labels” about how content was created.

Guest User 9/14/23 Guest User 9/14/23

Photo forensics from lighting shadows and reflections

This is the fifth article in a six-part series.

More from this series:

by Hany Farid

Where there is light, there are shadows. The relationship between an object, its shadow, and the illuminating light source(s) is geometrically simple, and yet it’s deceptively difficult to get just right in a manipulated or synthesized image. In the image below, the bottle's cast shadow is clearly incongruous with the shape of the bottle. Such obvious errors in a shadow are easy to spot, but more subtle differences can be harder to detect.

CGI model credit to Jeremy Birn, Lighting and Rendering in Maya.

Below are two images in which the bottle and its cast shadow are slightly different. (The rest of the scene is identical). Can you tell which is consistent with the lighting in the rest of the scene?

The geometry of cast shadows is dictated by the 3D shape and location of an object and the illuminating light(s). This relationship is wonderfully simple: A point on an object, its corresponding shadow, and the light source responsible for the shadow all lie on a single line.

Because (in the absence of lens distortion) straight lines in the 3D scene are imaged to straight lines in the 2D image, this basic constraint holds in the image. Locate any point on a shadow and its corresponding point on the object, and draw a line through them. Repeat for as many clearly defined shadow and object points as possible, and for an authentic image, all the lines will intersect at one point — the location of the illuminating light.

Below are the results of this simple geometric analysis applied to the two images above, clearly revealing the fake.

From shadows to reflections

As a student taking the subway to work each morning, I would stare out the window and watch the tunnel walls whiz by superimposed atop reflections of my fellow passengers. On occasion someone’s reflection would catch my eye, and I would look back into the subway car to get a better view. But I would invariably not see them where I thought they would be given the position of their reflection on the window. As a budding image and vision scientist this really bothered me — how could it be that it was so hard to reason about something so simple as a reflection in a window?

Below are two images of the same basic scene. The reflection of the table and garbage can are slightly different; everything else is the same. Can you tell which is correct?

The geometry of reflections is fairly straightforward. Consider standing in front of a mirror and looking at your reflection. An imaginary straight line connects each point on your body with its reflection. These lines are perpendicular to the mirror’s surface and are parallel to one another. When photographed at an oblique angle, however, these imaginary lines will not remain parallel but will converge to a single point. This is the same reason why railroad tracks that are parallel in the world appear to converge in a photograph.

This geometry of reflections suggests a simple forensic technique for verifying the integrity of reflections. Locate any point on an object and its corresponding point on the reflection, and draw a line through them. Repeat for as many clearly defined object and reflection points as possible. As you do this, you will find that all the lines should intersect at one point.

Below are the results of this simple geometric analysis, which clearly reveals the second reflection to be the fake.

Notice that this geometric analysis is exactly the same as that used to analyze shadows. The reason is that in both cases we are exploiting basic principles of perspective projection that dictate the projection of straight lines.

In practice, there are some limitations to a manual application of this geometric analysis. We must take care to select appropriately matched points on the shadow/reflection and the object. We can best achieve this when the object has a distinct shape, like the corner of a cube or the tip of a cone. Depending on the scene geometry, the constraint lines may be nearly parallel, making the computation of their intersection vulnerable to slight error in selecting matched points. And, it may be necessary to remove any lens distortion in the image that causes straight lines to be imaged as curved lines that will then no longer intersect at a single point.

From Adobe Photoshop to generative AI

When we first developed these techniques, we were primarily concerned with images that were manipulated with Photoshopto, for example, digitally add a person or object and its shadow or reflection. Because our own visual system is not particularly sensitive to inconsistent shadows and reflections (see item 1 in the Further Reading section below) and because it is difficult to precisely match the 3D scene geometry within 2D photo editing software, this technique has proven quite effective at uncovering fakes.

Classic computer-generated imagery is produced by modeling 3D scene geometry, the surrounding illumination, and a virtual camera. As a result, rendered images accurately capture the geometry and physics of natural scenes, including shadows and reflections. In contrast, AI-generated imagery is produced by learning the statistical distribution of natural scenes from a large set of real images. Without an explicit 3D model of the world, it is natural to wonder how accurately synthesized content captures the geometry of shadows and reflections.

Although we are still early in the age of generative AI, today's AI-generated images seem to struggle to produce perspectively correct shadows and reflections. Below is a typical example, generated using OpenAI's DALL-E2, in which the shadows are inconsistent (yellow lines), the reflections are impossibly mismatched and missing, and the shadows in the reflection are oriented in exactly the wrong direction. (See item 2 in the Further Reading section below for a more detailed analysis.

Even in the absence of explicit 3D modeling of objects, surfaces, and lighting — as found in traditional CGI-rendering — AI-generated images exhibit many of the properties of natural scenes. At the same time, cast shadows and reflections in mirrored surfaces are not fully consistent with the expected perspective geometry of natural scenes.

The trend in generative AI has been that increasingly larger synthesis engines yield increasingly more realistic images. As such, it may just be a matter of time before generative AI will learn to create images with full-blown perspective consistency. Until that time, however, this geometric forensic analysis may prove useful.

Further reading:

1. S.J. Nightingale, K.A. Wade, H. Farid, and D.G. Watson. Can People Detect Errors in Shadows and Reflections? Attention, Perception, & Psychophysics, 81(8):2917-2943, 2019.

2. H. Farid. Perspective (In)consistency of Paint by Text. arXiv:2206.14617, 2022.

More from this series:

Author bio: Professor Hany Farid is a world-renowned expert in the field of misinformation, disinformation, and digital forensics. He joined the Content Authenticity Initiative (CAI) as an advisor in June 2023. The CAI is a community of media and tech companies, non-profits, academics, and others working to promote adoption of the open industry standard for content authenticity and provenance.

Professor Farid is the recipient of an Alfred P. Sloan Fellowship and a John Simon Guggenheim Fellowship, and he’s a fellow of the National Academy of Inventors.

Receive updates and community news. Consider joining the CAI as a member.
Stay connected on LinkedIn and Twitter.
Explore the CAI’s open-source tools, powering Content Credentials, verifiable details or digital “nutrition labels” about how content was created.

Guest User 9/6/23 Guest User 9/6/23

Photo forensics from lighting environments

In his latest piece, digital forensics expert and CAI advisor Hany Farid investigates how lighting environments can help distinguish real images from AI-generated ones. He dives into techniques used in computer-generated imagery (CGI) technology we've grown accustomed to through entertainment and gaming and today's AI systems producing photorealistic images.

This is the fourth article in a six-part series.

More from this series:

by Hany Farid

Classic computer-generated imagery (CGI) is produced by modeling 3D scene geometry, the surrounding illumination, and a virtual camera. As a result, rendered images accurately capture the geometry and physics of natural scenes. By contrast, AI-generated imagery is produced by learning the statistical distribution of natural scenes. Without an explicit 3D model of the world, we might expect that these images will not always accurately capture the 3D properties of natural scenes.

The lighting of a scene can be complex, because any number of lights can be placed in any number of positions, leading to different lighting environments. Under some fairly modest assumptions, however, it has been shown that a wide range of lighting can be modeled as a weighted sum of nine 3D lighting environments that each capture a variety of lighting patterns from different directions.

My team and I first wondered if lighting environments on AI-generated faces match those on photographic images. Next, we wondered if the lighting on multiple faces or objects in an image are the same, as would be expected in an authentic photo.

Analyzing lighting environments first requires fitting and aligning a 3D model to individual objects. Fortunately, for faces, fitting a 3D model to a single image is a problem that has largely been solved. (See article 1 in the Further Reading section.)

Below you’ll see a 3D model fitted to an AI-generated face. From this aligned 3D model, we can estimate the surrounding lighting environment, which reduces to nine numeric values corresponding to the weight on each of nine 3D lighting environments.

An AI-generated face and a fitted 3D model. (Credit: Hany Farid)

As you can see in the plot on the left below, there are some significant deviations between the nine lighting values for real faces (horizontal axis) and diffusion-generated faces (vertical axis). Interestingly, as you can see in the plot on the right, the lighting on GAN-generated faces (see previous post) are nearly indistinguishable from the lighting on real faces.

The relationship between lighting environments on real (horizontal axis) and AI-generated (vertical axis) faces. The plot on the left corresponds to diffusion-based faces, the plot on the right corresponds to GAN-generated faces. (Credit: Hany Farid)

While a comparison of lighting environments may expose a diffusion-based face (or object), it appears that this approach will not provide much insight for GAN-based faces. This is the reverse of our previously described technique, which was applicable to GAN-based faces but not diffusion-based faces, further highlighting the need for a wide variety of approaches.

The above analysis can be expanded to compare the lighting between two people or two objects situated side by side in an image where, in a naturally photographed scene, we expect the lighting to be the same. For this analysis, we find that diffusion-based faces and objects exhibit relatively large (but not always perceptually obvious) differences in lighting. (See article 3 in the “Further reading” section below for more details.)

The strength of this lighting analysis is that we have discovered a fairly fundamental limitation of today's AI-generated content: In the absence of explicit 3D models of the physical world, diffusion-based models struggle to create images that are internally (across multiple people/objects) and externally (as compared to photographic images) consistent. The weakness of this technique is that, unlike other approaches, it does not necessarily lend itself to a fully automatic and high-throughput analysis. The fitting of 3D models can be somewhat time intensive and require manual intervention.

Another strength of this technique is that even when made aware of the issue of lighting inconsistencies, it is not easy for an adversary to adjust the image. That is because the pattern of illumination is the result of an interaction of the 3D lighting and the 3D scene geometry, whereas most photo editing occurs in 2D. Also, as you will see in the next post, there are additional aspects to lighting that we can exploit to distinguish the real from the fake.

Further reading:

[1] Y. Feng, H. Feng, M.J. Black, and T. Bolkart. Learning an animatable detailed 3D face model from in-the-wild images. ACM Transactions on Graphics, Proc. SIGGRAPH, 40(4):88:1– 840 88:13, 2021.

[2] M. Boháček and H. Farid. A Geometric and Photometric Exploration of GAN and Diffusion Synthesized Faces. Workshop on Media Forensics at CVPR, 2023.

[3] H. Farid. Lighting (In)consistency of Paint by Text. arXiv:2207.13744, 2022.

More from this series:

Author bio: Professor Hany Farid is a world-renowned expert in the field of misinformation, disinformation, and digital forensics. He joined the Content Authenticity Initiative (CAI) as an advisor in June 2023. The CAI is a community of media and tech companies, non-profits, academics, and others working to promote adoption of the open industry standard for content authenticity and provenance.

Professor Farid is the recipient of an Alfred P. Sloan Fellowship and a John Simon Guggenheim Fellowship, and he’s a fellow of the National Academy of Inventors.

Receive updates and community news. Consider joining the CAI as a member.
Stay connected on LinkedIn and Twitter.
Explore the CAI’s open-source tools, powering Content Credentials, verifiable details or digital “nutrition labels” about how content was created.

Guest User 8/30/23 Guest User 8/30/23

Photo forensics for AI-generated faces

Is your new social media connection a real person or AI-generated? In the latest installment of Hany Farid's series, he describes a technique for detecting GAN-generated faces, such as those typically found in online profiles.

This is the third article in a six-part series.

More from this series:

by Hany Farid

As I discussed in my previous post, generative adversarial networks (GANs) can produce remarkably realistic images of people that are nearly indistinguishable from images of real people. Here I will describe a technique for detecting GAN-generated faces, such as those typically found in online profiles

Below you will see the average of 400 StyleGAN2 faces (left) and 400 real profile photos from people with whom I am connected on LinkedIn (right).

Because the real photos are so varied, the average profile photo is fairly nondescript. By contrast, the average GAN face is highly distinct with almost perfectly focused eyes. This is because the real faces used by the GAN discriminator are all aligned in the same way, resulting in all the AI-generated faces having the same alignment. It seems that this alignment was intentional as a way to improve generation of more consistently realistic faces.

In addition to having the same facial alignment, the StyleGAN faces also appear from the neck up. By contrast, real profile photos tend to show more of the upper body and shoulders. It is these within-class similarities and across-class differences that we seek to exploit in our technique for detecting GAN-generated faces.

The average of 400 GAN-generated faces (left) and 400 real profile photos (right). (Credit: Hany Farid)

Starting with 10,000 faces generated by each of the three versions of StyleGAN (1, 2, 3), we learn a low-dimensional representation of GAN-generated faces. This learning takes two forms: the classic, linear, principal components analysis (PCA) and a more modern auto-encoder.

Regardless of the underlying computational technique, this approach learns a constructive model that captures the particular properties of StyleGAN-generated faces. In the case of PCA, this model is a simple weighted sum of a small number of images learned from the training images.

Once the model is constructed, we can ask if any image can be accurately reconstructed with this model. The intuition here is that if the model captures properties that are specific to GAN faces, then GAN faces will be accurately reconstructed with the model but real photos (that do not share these properties) will not.

Below you will see the average reconstruction error for 400 GAN faces (left) and 400 real faces (right). They are displayed on the same intensity scale where, the brighter the pixel value, the larger the reconstruction error. Here you will see that the real faces have significantly higher reconstruction errors throughout, particularly around the eyes.

The average reconstruction error for 400 GAN-generated faces (left) and 400 real profile photos (right). (Credit: Hany Farid)

By simply computing this reconstruction error, we can build a fairly reliable classifier to determine whether a face was likely to have been generated by StyleGAN. We can make this classifier even more reliable by analyzing the eiGAN-face model weights needed for the full reconstruction.

Using this approach, we can detect more than 99% of GAN-generated faces, while only misclassifying 1% of photographic faces — those that happen to match the position and alignment of GAN faces.

Any forensic technique is, of course, vulnerable to attack. In this case, an adversary could crop, translate, and/or rotate a GAN-generated face in an attempt to circumvent our detection. We can, however, build this attack into our training by constructing our model not just on GAN faces, but also on GAN faces that have been subjected to random manipulations. (See the article in the Further Reading section below for more details on this attack and defense.)

No defense will ever be perfect, so the forensic identification cannot rely on just one technique. It must instead rely on a suite of techniques that examine different aspects of an image. Even if an adversary does not find a way to circumvent this particular detection, newer diffusion-based, text-to-image synthesis techniques (e.g., Dall-E, Midjourney, and Adobe Firefly) do not exhibit the same facial alignment as GAN-generated faces. As a result, they will fail to be detected by this specific technique. In my next post, I will describe a complementary forensic analysis that is more effective at detecting faces generated using diffusion-based techniques.

Further reading:

[1] S. Mundra, G.J.A. Porcile, S. Marvaniya, J.R. Verbus and H. Farid. Exposing GAN-Generated Profile Photos from Compact Embeddings. Workshop on Media Forensics at CVPR, 2023.

More from this series:

Author bio: Professor Hany Farid is a world-renowned expert in the field of misinformation, disinformation, and digital forensics. He joined the Content Authenticity Initiative (CAI) as an advisor in June 2023. The CAI is a community of media and tech companies, non-profits, academics, and others working to promote adoption of the open industry standard for content authenticity and provenance.

Professor Farid is the recipient of an Alfred P. Sloan Fellowship and a John Simon Guggenheim Fellowship, and he’s a fellow of the National Academy of Inventors.

Receive updates and community news. Consider joining the CAI as a member.
Stay connected on LinkedIn and Twitter.
Explore the CAI’s open-source tools, powering Content Credentials, verifiable details or digital “nutrition labels” about how content was created.

Guest User 8/22/23 Guest User 8/22/23

How realistic are AI-generated faces?

In the latest installment of this CAI series, Hany Farid unpacks what was once the most common computational technique for synthesizing images. Learn about telltale clues and new fast-improving technology making it more difficult to distinguish real faces from those synthetically made.

This is the second article in a six-part series.

More from this series:

Read part 1: From the darkroom to generative AI

by Hany Farid

Before we dive into talking about specific analyses that can help distinguish the real from the fake, it is worthwhile to first ask: Just how photorealistic are AI-generated images? That is, upon only a visual inspection, can the average person tell the difference between a real image and an AI-generated image? Because the human brain is so finely sensitive to images of people and faces, we will focus on the photorealism of faces, arguably the most challenging task for AI-generation.

Before diffusion-based, text-to-image synthesis techniques like DALL-E, Midjourney, and Adobe Firefly splashed onto the screen, generative adversarial networks (GANs) were the most common computational technique for synthesizing images. These systems are generative, because they are tasked with generating an image; adversarial, because they pit two separate components (the generator and the discriminator) against each other; and networks, because the computational machinery underlying the generator and discriminator are neural networks.

Although there are many complex and intricate details in these systems, StyleGAN (and GANs in general) follow a fairly straightforward structure. When tasked with creating a synthesized face, the generator starts by splatting down a random array of pixels, and then it feeds this first guess to the discriminator. If the discriminator, equipped with a large database of real faces, can distinguish the generated image from the real faces, the discriminator provides this feedback to the generator. The generator then updates its initial guess and feeds this update to the discriminator in a second round. This process continues with the generator and discriminator competing in an adversarial game until an equilibrium is reached when the generator produces an image that the discriminator cannot distinguish from real faces. Above you can see a representative set of eight GAN-generated faces.

In a series of recent perceptual studies, we examined the ability of trained and untrained observers to distinguish between real and synthesized faces. The synthetic faces were generated using StyleGAN2, ensuring diversity across gender, race, and apparent age. For each synthesized face, a corresponding real face was matched in terms of gender, age, race, and overall appearance.

In the first study, 315 paid online participants were shown — one at a time — 128 faces, half of which were real. The participants were asked to classify each as either real or synthetic. The average accuracy on this task was 48.2%, close to chance performance of 50%, with participants equally as likely to say that a real face was synthetic as vice versa.

In a second study, 219 new participants were initially provided with a brief training on examples of specific rendering artifacts that can be used to identify synthetic faces. Throughout the experiment, participants were also provided with trial-by-trial feedback informing them whether their response was correct. This training and feedback led to a slight improvement in average accuracy, from 48.2% to 59.0%.

While synthesized faces are highly realistic, there are some clues that can help distinguish them from real faces:

StyleGAN-synthesized faces have a common structure consisting of a mostly front-facing person from the neck up and with a mostly uniform or nondescript background.
Facial asymmetries are telltale signs of synthetic faces. They are often seen in mismatched earrings or glasses frames. (See the top-left image in the figure below.)
When viewing a face, we tend to focus most of our attention in a "Y" pattern, shifting our gaze between the eyes and mouth and often missing some glaring artifacts in the background that may contain physically implausible structures. (See the top-right image in the figure below.)
StyleGAN-synthesized faces enforce a facial alignment in the training and synthesis steps that results in the spacing between the eyes being the same and the eyes being horizontally aligned in the image. (See the bottom image in the figure below.)

Eight GAN-generated faces (middle); a magnified view of the fifth face (top left) revealing asymmetric earrings; a magnified view of the sixth face (top right) revealing a stranger synthesis artifact on the shoulder; and an average of all eight faces (bottom) revealing that the eyes in each face are aligned to the same position and interocular distance. (Credit: Hany Farid)

Having largely passed through the uncanny valley, GAN-generated faces are nearly indistinguishable from real faces. Although we have not carried out formal studies, my intuition is that the more recent diffusion-based, text-to-image generation yields faces that are slightly less photorealistic. As this technology improves, however, it will surely lead to images that also pass through the uncanny valley.

Having seen that the visual system may not always be reliable in distinguishing the real from the fake, in my next post I will describe our first technique that exploits the facial alignment of GAN-generated faces to distinguish them from real faces.

Further reading:

[1] S.J. Nightingale and H. Farid. AI-Synthesized Faces are Indistinguishable from Real Faces and More Trustworthy. Proceedings of the National Academy of Sciences, 119(8), 2022.

More from this series:

Read part 1: From the darkroom to generative AI

Author bio: Professor Hany Farid is a world-renowned expert in the field of misinformation, disinformation, and digital forensics. He joined the Content Authenticity Initiative (CAI) as an advisor in June 2023. The CAI is a community of media and tech companies, non-profits, academics, and others working to promote adoption of the open industry standard for content authenticity and provenance.

Professor Farid is the recipient of an Alfred P. Sloan Fellowship and a John Simon Guggenheim Fellowship, and he’s a fellow of the National Academy of Inventors.

Receive updates and community news. Consider joining the CAI as a member.
Stay connected on LinkedIn and Twitter.
Explore the CAI’s open-source tools, powering Content Credentials, verifiable details or digital “nutrition labels” about how content was created.

Guest User 8/15/23 Guest User 8/15/23

From the darkroom to generative AI

In this era of generative AI, how do we determine whether an image is human-created or machine-made? Read the first part in a series exploring techniques used for identifying synthetic media and how we regain trust in the visual record.

This is the first article in a six-part series.

More from this series:

Part 5: Photo forensics from lighting shadows and reflections

Part 6: Passive versus active photo forensics in the age of AI and social media

by Hany Farid

Stalin, Mao, Hitler, Mussolini, and many other dictators had photographs manipulated in an attempt to rewrite history. These men understood the power of photography: If they changed the visual record, they could potentially change history.

Soviet secret police official Nikolai Yezhov, pictured to the right of Joseph Stalin, was later removed from this photograph. (Credit: Fine Art Images/Heritage Images/Getty Images & AFP/GettyImages)

In the past, altering the historical record required cumbersome and time-consuming darkroom techniques. Starting in the early 2000s, powerful and low-cost digital technology began to make it easier to record and alter digital images. And today, sophisticated generative-AI software has fully democratized the ability to create compelling digital fakes.

Twenty-five years ago, I was waiting in line at the library when I noticed an enormous book in the return cart called The Federal Rules of Evidence. I thumbed through the book and came across Rule 1001 of Article X, which outlined the rules under which photographic evidence can be introduced in a court of law. The rules seemed straightforward until I read the definition of original:

An “original” of a writing or recording means the writing or recording itself or any counterpart intended to have the same effect by the person who executed or issued it. For electronically stored information, “original” means any printout — or other output readable by sight — if it accurately reflects the information.

I was struck that the definition of original included this vague statement: “... or other output readable by sight.”

At the time, digital cameras and digital editing software were still primitive by today’s standards, and generative AI was unimaginable. The trajectory of technology, however, was fairly clear, and it seemed to me that advances in the power and ubiquity of digital technology would eventually lead to complex issues around how we can trust digital media. As we enter the new age of generative AI, these issues have only become more salient.

Although it varies in form and creation, generative AI content (a.k.a. deepfakes) refers to images, audio, or video that has been automatically synthesized by an AI-based system. Deepfakes are the latest in a long line of techniques used to manipulate reality — from Stalin's darkroom to Photoshop to classic computer-generated renderings. However, their introduction poses new opportunities and risks now that everyone has access to what was historically the purview of a small number of sophisticated organizations.

Even in these early days of the AI revolution, we are seeing stunning advances in generative AI. The technology can create a realistic photo from a simple text prompt, clone a person's voice from a few minutes of an audio recording, and insert a person into a video to make them appear to be doing whatever the creator desires. We are also seeing real harms from this content in the form of non-consensual sexual imagery, small- to large-scale fraud, and disinformation campaigns.

A photographic image (left), and AI-generated images generated by StyleGAN (middle) and Stable Diffusion (right). (Credit: Hany Farid)

Building on our earlier research in digital media forensics techniques, over the past few years my research group and I have turned our attention to this new breed of digital fakery. All our authentication techniques work in the absence of digital watermarks or signatures. Instead, they model the path of light through the entire image-creation process and quantify physical, geometric, and statistical regularities in images that are disrupted by the creation of a fake.

In this series of posts, I will describe a collection of different techniques that we use to determine whether an image is real or fake. I will explain the underlying analyses — including analyzing shadows, reflections, vanishing points, specular reflections in the eye, and AI-generated faces — and the conditions under which the analyses are suitable.

While these and related techniques can be powerful, I need to emphasize their limitations.

First, it is important to understand that while the presence of evidence of manipulation can tell us something, the absence of traces of manipulation do not prove that an image is real. We cannot differentiate between a flawless fake and the real thing.

Second, nearly all forensic techniques have a limited shelf life because techniques to manipulate content are continually improving. So while these techniques are applicable today, we have to continually be aware of how generative AI is evolving.

Finally, these forensic techniques are only one part of a larger ecosystem needed to regain trust in the visual record. This includes efforts like the Content Authenticity Initiative (to which I contribute), some regulatory pressures, education, and the responsible deployment of AI technologies.

Further reading:
[1] H. Farid. Image Forensics. Annual Review of Vision Science, 5(1):549-573, 2019.

More from this series: