Once again, it seems that an image that challenges our notions of visual perception is making the rounds on some social networks. This time it is an image from Dr. Akiyoshi Kitaoka, a Professor of Psychology at Ritsumeikan University in Japan, who specializes in creating some highly effective demonstrations that allow us to see ourselves “see.”
This time it is a photograph of strawberries that appear to be veiled behind a digital curtain of cyan:
What may come as a surprise to you is that there is indeed no “red” within this image.
So why do we perceive red within this picture? Generally speaking—color constancy.
Popularized on social media via the 2015 viral blue dress photograph, color constancy can be defined as the perception of seemingly invariant properties of a surface’s spectral reflectance despite changes in illumination and viewing conditions. For example, roses look red, violets look blue, and these appearances do not seem to change color with changes in illumination.
One of the contributing mechanisms to color constancy is what is known as chromatic adaptation. Chromatic adaptation influences the perception of a particular color within the context of its surroundings. In Wyszecki & Stiles’ COLOR SCIENCE: Concepts and Methods, Quantitative Data and Formulae, chromatic adaptation is defined as a change in the visual response to a color stimulus that is caused by (1) previous exposure to a conditioning stimulus (such as a luminous colored light or intensely colored surface) or (2) simultaneous presentation of the color stimulus against a surround or background of a different color.
Such mechanisms of perceptual adaptation are often associated with or explained by low-level visual processes (e.g., adaptation and opponency processes near the level of the receptor.) However, it is important to acknowledge that visual perception (or our perception of color) does not occur at the level of the photoreceptor any more than our perception of “touch” occurs at the level of the mechanoreceptor. To this point, one of the questions that popped up on social media in connection to the sharing of this demonstration was “Is that partly to do with people knowing that strawberries are very red?”
The responses seemed to downplay such higher-level processing contributions while steering attention back to low-level, data-driven mechanisms.
But are such responses correct in downplaying these contributions?
In an article regarding Dr. Kitaoka’s impressive new demonstration, neuroscientist Bevil Conway (National Eye Institute) suggested (after explaining the familiar components of color constancy of course) that there is more at work. “Conway said this illusion is also helped out by the fact that we recognize the objects as strawberries, which we very strongly associate with the color red, so our brain is already wired to be looking for those pigments.” – Kaleigh Rogers, Motherboard
It is true that modern research has shown that color constancy is not a simple mechanism to understand. For example, while substantial chromatic adaptation takes place in the retina, additional contributory processes have been shown to take place as far as V4 and into the temporal lobe (inferior temporal cortex.) FMRI experiments in humans have shown that instantaneous color constancy involves strong activation in both the fusiform color area and V1, along with significant activity in V2 and V3 (Barbur & Spang, 2008). The importance of V4 for color constancy was tested in lesion studies by Wild et al. (1985) and Walsh, Carden, Butler, and Kulikowski (1993).
Now, I should make it clear that activity that far along the ventral stream, (into the inferior temporal cortex), is indeed in or near the vicinity of object-recognition territory. But let’s put that aside and consider a more significant aspect of this issue that may better address the above question—the inverse problem. (For those not familiar with the concept, the inverse problem of vision is the idea that information at the level of the retina (proximal stimulus) is ultimately an ambiguous conflation of distal stimulus variables (illumination, reflectance, absorption, and transmission).)
The idea that the visual system is somehow able to separate wavelength information from surface reflectance can indeed be viewed as another consideration of the inverse problem. Therefore it should stand to reason that, given the inability of the visual system to have direct access to the properties of the world, the visual system links the frequency of occurrence of biologically determined stimuli to useful perceptual and behavioral responses without the need for recovering real-world properties. Fortunately, there exists a great deal of evidence for this interpretation of biological vision.
For example, if we were to look at another one of Dr. Kitaoka’s fascinating demonstrations we can see that that the left eye of each figure appears to hold color while the right appears achromatic (without color). Such demonstrations are common examples of color constancy, as well as chromatic adaptation which again, are often explained by low-level operations of adaptation and opponency. But could there be more at work here?
One issue worth consideration here is apparent disparities in “effect magnitude” between the less complex graphics of color swatches below the figures and the figures themselves. Do the colors perceived in the small, central gray squares appear identical in chroma when compared to the perceived colors observed in the eyes of the figures? How about in the lower section where the comparison is made even closer by keeping the context somewhat ambiguous but moving the spatial relationships closer to the original figure configurations?
Now, does this mean that we are linking the available stimulus to familiar female figures that indeed have cyan, yellow or red eyes? LOL—of course not. What I am stating is that we (both individuals and the species) have found behavioral success in the past by perceiving an achromatic stimulus in such contexts as a particular color. Furthermore, there may also be a push from our past association of “color” with our concept of the eye.
Here are our strawberries again with an abstracted version to see if a less familiar context holds the same magnitude:
While some hypotheses may hold more predictive power than others here, we can indeed be certain that current research reveals that there is much more in play in addition to the significant contributions from low-level adaptation and opponency.
Hope this offers some food for thought….
Happy Painting!
A good way to help perceive the achromatic grey of the cyan tinted strawberries is to squint when looking at the picture. Squinting reduces the light level and switches our visual perception toward luminance/value.
I just discovered your blog. I’d be interested in what you consider a good primer set of references that I or anyone else should read to bring us up to date with the current state of perceptual research.
Thanks for the fascinating discussion.
Thank you for your feedback Abbott—as well as the great question. Here are a few texts that should provide you with a good amount of information on your quest to better understand some current ideas about visual perception:
One of my favorite textbooks for a comprehensive look into the interdisciplinary world of modern vision science is Stephen E. Palmer’s Vision Science: Photons to Phenomenology. While the book is almost 20 years old now (published in 1999), it provides a wonderfully robust view into many of the major topics related to vision, from early neural processing of image structure in the retina to high-level visual attention, memory, imagery, and awareness. The presentation throughout is theoretically sophisticated yet requires minimal knowledge of mathematics. There is also an extensive glossary, as well as appendices on psychophysical methods, connectionist modeling, and color technology. The book will serve not only as a comprehensive textbook on vision, but also as a valuable reference for researchers in cognitive science, psychology, neuroscience, computer science, optometry, and philosophy. Unfortunately, this book is rather expensive fetching about $50-$100.
I would also recommend Why We See What We Do Redux: A Wholly Empirical Theory of Vision (Second Edition) by Dale Purves and Beau R. Lotto. The First Edition of this provocative book reviewed a broad range of evidence leading to the conclusion that the visual system does not reveal the physical world by an analysis of retinal images and their representation by the visual system. Rather, what we see is based on the history of the species and the individual as a means of contending with the inherent uncertainty of light stimuli. It follows that visual perceptions are reflexive manifestations of past behavioral success rather than the result of a logical processing of present stimuli.
These ideas were met with considerable skepticism. To quote from the preface of this new edition:
“Although the ideas and evidence about the genesis of what we see in the First Edition were appreciated in some quarters, the reception in others was distinctly cool. Given the opinion of some critics that the wholly empirical concept of vision we proposed was either unbelievable or incomprehensible, we felt duty bound to try again. Our objective was, and remains, to present a different and seemingly inevitable framework for understanding perception and its underlying neural mechanisms. We hope this new edition will encourage more readers to consider this concept of vision and its implications for interpreting, modeling, and ultimately understanding the structure and function of the human visual system.”
As you might tell from my writings, I indeed subscribe to the empirical ranking theory of vision (the idea of vision presented in Why We See What We Do Redux: A Wholly Empirical Theory of Vision). If you enjoy this you might like to take a look at Beau Lotto’s recent book, Deviate. I really enjoyed it.
Another interesting read is Visual Intelligence: How We Create What We See by Donald Hoffman.
I should mention that I recently ordered A Natural History of Seeing: The Art and Science of Vision Hardcover by Simon Ings. It was published in 2008 and looked pretty good. I’ll let you know how it is.
If any of the above texts seem a tad too dense you might enjoy starting with something like Margaret Livingstone’s Vision and Art, The Biology of Seeing. This book by Harvard neurobiologist Margaret Livingstone explores the inner workings of vision, demonstrating that how we see art depends ultimately on the cells in our eyes and our brains. In Vision and Art,Livingstone explains how great painters fool the brain: why Mona Lisa s smile seems so mysterious, Monet s Poppy Field appears to sway in the breeze, Mondrian s Broadway Boogie Woogie blinks like the lights of Times Square, and Warhol s Electric Chair pulses with current. Drawing on history and her own cutting- edge discoveries, Livingstone offers intriguing insights, from explanations of common optical illusions, to speculations on the correlation of learning disabilities with artistic skill. By skillfully bridging the space between science and art, Vision and Art will both arm artists and designers with new techniques that they can use in their own craft, and thrill any reader with an interest in the biology of human vision.
I hope these are helpful! I’ll let you know if I think of any others.
Anthony,
Thanks for the thoughtful and thorough reply. A little background of my interest in the subject beyond just my personal art is probably in order. From the late 90’s to 2007 I was heavily involved in building animation curricula and teaching in the Seattle area. Ultimately I became the Assoc. Dean at one of the colleges. My work culminated with the creation of a BFA program that included a first semester freshman course in Vision Biology. I used Livingstone’s book as the textbook for the course and the Palmer book as well as drilling into doctoral abstracts to prep my lectures. Glad to hear that these core ideas are still germaine. (The course also covered core study skills because we were finding that too many of our art students arrived at our doorstep thinking of themselves as poor students. We found exposing them to things memorization research helped.)
I got burned out on the academic politics and stepped away from collegiate academia for a decade. I’ve been slowly heading back to the field of art education and wanted to see what I needed to do to catch up with the basic knowledge of perception and processing.
I’m interested to read the Purves and Lotto material. My best friend and I spend a great deal of thought and consideration discussing the Visual Language and specifically the role of the three systems the mind uses to mine contrast relationship relative to contrast thresholds. Finite systems such as luminance, satuation and opacity compare fixed fields of data with definite caps. Then there’s the functionally infinite scales of size and time. And finally there is the case of rotational information such as hue and musical key. Each system has prestty straight forward thresholds of suppression and anplification. Seeng blades of grass pales in survival importance relative being able to see the tiger in the grass. The Perkinje shift helps survive going to the water in low light.
We always worked to help our students understand that visual data is amplified or suppressed multiple times before it reaches the higher processing centers of our brain. We don’t make decisions based on the raw data because our sensory systems aren’t passive.
Thank you again for taking the time to provide a thorough answer.
Abbott
Thank you for the background info Abbott! It sounds like your students were very fortunate to have access to your efforts. Please let me know how you like the Purves/Lotto material. Some are quite resistant to the empirical ranking theory of vision that Purves and Lotto promote as it really decimates the the intuitive concept that our visual system has some magical aspect of verdicality. It’s like telling some people that the world isn’t flat. What’s important to keep in mind though is how the theory is supported by other avenues of research. For example, Donald D. Hoffman, a professor of cognitive science at the University of California, Irvine has spent the past three decades studying perception, artificial intelligence, evolutionary game theory, and the brain. In a 2016 article with Quanta Magazine, Dr. Hoffman states, “The classic argument is that those of our ancestors who saw more accurately had a competitive advantage over those who saw less accurately and thus were more likely to pass on their genes that coded for those more accurate perceptions, so after thousands of generations we can be quite confident that we’re the offspring of those who saw accurately, and so we see accurately. That sounds very plausible. But I think it is utterly false. It misunderstands the fundamental fact about evolution, which is that it’s about fitness functions — mathematical functions that describe how well a given strategy achieves the goals of survival and reproduction. The mathematical physicist Chetan Prakash proved a theorem that I devised that says: According to evolution by natural selection, an organism that sees reality as it is will never be more fit than an organism of equal complexity that sees none of reality but is just tuned to fitness. Never.” Furthermore, in regards to the hundreds of thousands of computer simulations run by Dr. Hoffman and his research team, he states, “Some of the [virtual] organisms see all of the reality, others see just part of the reality, and some see none of the reality, only fitness. Who wins? Well, I hate to break it to you, but perception of reality goes extinct. In almost every simulation, organisms that see none of reality but are just tuned to fitness drive to extinction all the organisms that perceive reality as it is. So the bottom line is, evolution does not favor veridical, or accurate perceptions. Those perceptions of reality go extinct.”
BTW–I picked up the Simon Ing book and am making my way through it now. So far so good–In fact, I have to say that it is really interesting in the way it is written as the author presents fairly technical concepts in a writing style that would seem more common to a novelist. I’ll share more when I finish it~~~ 😀