My biggest issue with using AI to add color to black and white photographs is that it can almost only lead us astray. There is no guarantee that the AI is correct about the colors, and while it may get close, we’ll never actually know the truth. Assuming that we do, depending on the photograph, could potentially alter our understanding of history.
In Ted Chiang’s, ChatGPT is a Blurry JPEG of the Web, he asks us to “[t]hink of ChatGPT as a blurry jpeg of all the text on the Web”, and I’m inclined to go along with his logic. It certainly explains why I saw so many people asking AI how many “r”s are in strawberry and having the response be two, or why AI “””art””” often has people with too many fingers.
But understanding it this way doesn’t make me feel any better about it. In fact, it makes me feel worse. I always knew that AI just scraped whatever it could get its hands on for data and mashed it together until it resembled something that could pass as original if you don’t look too closely. What I didn’t think about is the idea of all of its information being compressed and some being inherently lost. On the smallest scale, we never notice. But AI is doing this so much that there must be giant gaps in its learning that are filled by hallucinations.
These gaps are what worry me, and why I don’t think we should be relying on AI to predict anything, especially not something like colors in a photograph that are so specific and so easy to get wrong. I also believe that if we have the information to approximate the colors, there’s no need for AI to do it for us. At that point, either recoloring it ourselves by hand or simply writing a caption with what we believe are the correct colors would be enough. As Ted Chiang says, “So just how much use is a blurry JPEG, when you still have the original?”
Hello Sabrina,
I really enjoyed your writing. I also believe that the use of AI in colorizing images may be problematic. On top of that, I think that people should implement more policies to have better management of AI in general. In this case, the government would be responsible for this. It is hard to imagine how AI could develop and evolve without specific policies and guidelines to limit its use. One of the worries I have is that people might be taking advantage of AI to harm other people, which is not what I want to see. Again, AI policies are really important, and we all have to be fully aware of its power.
This is a good way of phrasing AI’s current limitations, I appreciate your analysis of AI generated and manipulated art as being scraped and mashed together. It is off-putting to think that art manipulation uses similar processes to that of generation; even if manipulation does not quite use the same scraping techniques, the process of colorization requires the AI to be trained on some human-made material. I am curious about your choice in naming the filling of these gaps as ‘hallucinations.’ The term is certainly evocative of what AI places there, but is that filled gap not the product of AI scraping and reusing data? The choice of writing the colors down and asking the viewer to imagine them seems like a stretch at first; but I would agree with the sentiment as one’s imagination could produce a more vivid image.
What a well thought out, concise, and brilliantly phrased post, Sabrina! I could not agree more in terms of AI image colorization being irredeemably treacherous. I also wondered as to why on Earth we would ever settle for an ambiguous AI colorized photo, when we already have the contextual information to do it ourselves. I was unable to find a way to integrate that great Ted Chiang quote into my post, so I am glad you were able to do it!
Sabrina, your classification results show something that I haven’t seen in any other week 4 lab. On the original, the model’s top guess was only 6.64%, but after colorization it said “balance beam” at 48.53%. The color made the AI about seven times more confident without making it any more correct, which is Chiang’s blurry JPEG problem playing out in your own photo.