There are a variety of issues that arise when considering the combination of artificial intelligence and image manipulation. With the rise of AI, we are increasingly able to alter, tag, classify, and colorize images (just to name a few processes) with ease. While this may open new avenues for understanding and investigating the past, working through AI systems increase the risk of creating images, that while initially convincing, in fact are entirely incorrect. We will use examples from Carleton College’s student scrapbook archives, specifically this image of Cannon Falls.
In his article, “ChatGPT is a Blurry JPEG of the Web,” Ted Chiang provides important context for the ways in which artificial intelligence reformulates previously published information and AI’s ability, or lack thereof, to produce accurate information. He writes, “But, even if a large language model includes only the information we want, there’s still the matter of blurriness. There’s a type of blurriness that is acceptable, which is the restating of information in different words. Then there’s the blurriness of outright fabrication, which we consider unacceptable when we’re looking for facts. It’s not clear that it’s technically possible to retain the acceptable kind of blurriness while eliminating the unacceptable kind.” Considering Chiang’s argument in the context of image manipulation, Chiang raises an important idea. Fundamentally, AI tools are trained to create approximations. Thus, when manipulating historic images, AI is creating a product which might look true, but due to the inherent “blurriness” in its modeling may entirely misconstrue what the image originally captured. Take the original and colorized images of Cannon Falls below created in DeepAI’s Deoldify Tool as an example. While the colors look generally convincing in the colorized image, these are simply approximations from large training datasets. There is no way to make sure the colorization is correct. In turn, these training datasets will fold in a variety of biases that are not distinguishable in the final form, but will affect our perception of the past.
This is particularly important as large image datasets are used in research. Our tagging of images with AI can be regularly incorrect. A good example comes in my segmented image created in the Distant Viewer Explorer displayed below. While it is largely correct, there is no house in the picture, despite what the model says. This error is easy to identify in a single image, but issues like this would be much harder to track across a larger dataset. Yet, without scrutiny, this will lead to incorrect analyses as the data itself is flawed from the start.
Sonja Drimmer, offers a sharper critique of image colorization in her article “How AI is Hijacking Art History.” Speaking of AI colorization, she writes, “But this effort to ‘bring events back to life’ routinely mistakes representations for reality. Adding color does not show things as they were but recreates what is already a recreation – a photograph – in our own image, now with computer science’s seal of approval.” As Drimmer underscores, when colorizing images we also miss the larger contextual nature of these pieces as representations. In other words, in their original form they are already approximations of a moment in time. Colorizing them holds the potential to get further from, as opposed to closer to the past.
Great post, Jesse! I brought up some of the same points about how AI can be misleading, especially without careful consideration of its outputs. I think you bring this up in your post, but I wonder what this means for large scale image analyses—for instance, what if someone puts a bunch of images into a program for analysis and processes them immediately. What kinds of troubles might that arise?