Jason Gold

Humans are remarkably adept at recognizing objects across a wide range of views. A notable exception to this general rule is that turning a face upside down makes it particularly difficult to recognize 1, 2, 3. This striking effect has prompted speculation that inversion qualitatively changes the way faces are processed. Researchers commonly assume that configural cues strongly influence the recognition of upright, but not inverted, faces 3, 4, 5. Indeed, the assumption is so well accepted that the inversion effect itself has been taken as a hallmark of qualitative processing differences [6]. Here, we took a novel approach to understand the inversion effect. We used response classification 7, 8, 9, 10 to obtain a direct view of the perceptual strategies underlying face discrimination and to determine whether orientation effects can be explained by differential contributions of nonlinear processes. Inversion significantly …

Perceptual discrimination improves with practice. This ‘perceptual learning’is often specific to the stimuli presented during training 1, 2, 3, 4, 5, indicating that practice may alter the response characteristics of cortical sensory neurons 6, 7. Although much is known about how learning modifies cortical circuits 8, it remains unclear how these changes relate to behaviour. Different theories assume that practice improves discrimination by enhancing the signal 1, 9, 10, diminishing internal noise 11, 12 or both 13. Here, to distinguish among these alternatives, we fashioned sets of faces and textures whose signal strength could be varied, and we trained observers to identify these patterns embedded in noise. Performance increased by up to 400% across several sessions over several days. Comparisons of human performance to that of an ideal discriminator showed that learning increased the efficiency with which …

To better understand how the visual system makes use of information across spatial scales when identifying different kinds of complex patterns, we measured human and ideal contrast identification thresholds to estimate identification efficiency for 1- and 2-octave wide band-pass filtered letters and faces embedded in 2-D dynamic Gaussian noise. Varying stimulus center frequency from 1 to 70 c/object had different effects on letter and face identification efficiency. In the 2-octave conditions, identification efficiencies decreased by 0.25–0.5 log units for letters and 0.5–1.2 log units for faces as center frequency increased from 6.2 to 49.5 c/object, but only letters were identifiable at center frequencies below 6.2 c/object. In the 1-octave conditions, letter identification efficiencies increased by about 0.5 log units as center frequency increased from 1.1 to 2.2 c/object, and were nearly constant from 2.2 to 35 c/object. Letters …

The visual system is constantly faced with the problem of identifying partially occluded objects from incomplete images cast on the retinae. Phenomenologically, the visual system seems to fill in missing information by interpolating illusory and occluded contours at points of occlusion, so that we perceive complete objects. Previous behavioural 1, 2, 3, 4, 5, 6, 7 and physiological 8, 9, 10, 11, 12 studies suggest that the visual system treats illusory and occluded contours like luminance-defined contours in many respects. None of these studies has, however, directly shown that illusory and occluded contours are actually used to perform perceptual tasks. Here, we use a response-classification technique 13, 14, 15, 16, 17, 18, 19, 20 to answer this question directly. This technique provides pictorial representations — ‘classification images’ — that show which parts of a stimulus observers use to make perceptual decisions …

Performance in perceptual tasks often improves with practice. This effect is known as ‘perceptual learning,’ and it has been the source of a great deal of interest and debate over the course of the last century. Here, we consider the effects of perceptual learning within the context of signal detection theory. According to signal detection theory, the improvements that take place with perceptual learning can be due to increases in internal signal strength or decreases in internal noise. We used a combination of psychophysical techniques (external noise masking and double‐pass response consistency) that involve corrupting stimuli with externally added noise to discriminate between the effects of changes in signal and noise as observers learned to identify sets of unfamiliar visual patterns. Although practice reduced thresholds by as much as a factor of 14, internal noise remained virtually fixed throughout training …

Much recent research has aimed to establish whether visual working memory (WM) is better characterized by a limited number of discrete all-or-none slots or by a continuous sharing of memory resources. To date, however, researchers have not considered the response-time (RT) predictions of discrete-slots versus shared-resources models. To complement the past research in this field, we formalize a family of mixed-state, discrete-slots models for explaining choice and RTs in tasks of visual WM change detection. In the tasks under investigation, a small set of visual items is presented, followed by a test item in 1 of the studied positions for which a change judgment must be made. According to the models, if the studied item in that position is retained in 1 of the discrete slots, then a memory-based evidence-accumulation process determines the choice and the RT; if the studied item in that position is missing, then a …

When you see a person’s face, how do you go about combining his or her facial features to make a decision about who that person is? Most current theories of face perception assert that the ability to recognize a human face is not simply the result of an independent analysis of individual features, but instead involves a holistic coding of the relationships among features. This coding is thought to enhance people’s ability to recognize a face beyond what would be expected if each feature were shown in isolation. In the study reported here, we explicitly tested this idea by comparing human performance on facial-feature integration with that of an optimal Bayesian integrator. Contrary to the predictions of most current notions of face perception, our findings showed that human observers integrate facial features in a manner that is no better than would be predicted by their ability to use each individual feature when shown …

The present experiment was conducted in order to examine the role of cue—target discriminability on early occurring attentional cuing effects and late occurring inhibition of return (IOR). The experiment used a single target stimulus in conjunction with three different cue stimuli. The cues were the same as the target, different in color, shape, and luminance to the target, or did not spatially overlap with the target. At shorter stimulus onset asynchronies (SOAs; 100 and 200 msec), attentional cuing effects were only found with the nonoverlapping cues. However, at longer SOAs (400 and 800 msec), approximately equal IOR effects were found with all three types of cues. The results indicated that the physical characteristics of the cues and targets affected the pattern of reaction times at the shorter SOAs but not at the longer SOAs. The conclusion is that the biphasic pattern of early facilitation and late inhibition …

The bubbles method is a recently developed variant of reverse correlation methods that have been used in psychophysics and physiology. We show mathematically that for the broad and important class of noisy linear observers, the bubbles method recovers much less information about how observers process stimuli than reverse correlation does. We also show experimentally that the unusual type of noise used in the bubbles method can drastically change human observers’ strategies in psychophysical tasks, which reduces the value of the information that is obtained from a bubbles experiment. We conclude that reverse correlation is generally preferable to the bubbles method in its present form, but we also give suggestions as to how the bubbles method could be modified to avoid the problems we discuss.

After observers see an object or pattern, their visual memory of what they have seen decays slowly over time. Nearly all current theories of vision assume that decay of short-term memory occurs because visual representations are progressively and randomly corrupted as time passes. We tested this assumption using psychophysical noise-masking methods, and we found that visual memory decays in a completely deterministic fashion. This surprising finding challenges current ideas about visual memory and sets a goal for future memory research: to characterize the deterministic “forgetting function” that describes how memories decay over time.

A critical issue in object recognition research is how the parts of an object are analyzed by the visual system and combined into a perceptual whole. However, most of the previous research has examined how changes to object parts influence recognition of the whole, rather than recognition of the parts themselves. This is particularly true of the research on face recognition, and especially with questions related to the neural substrates. Here, we investigated patterns of BOLD fMRI brain activation with internal face parts (features) presented singly and in different combinations. A preference for single features over combinations was found in the occipital face area (OFA) as well as a preference for the two-eyes combination stimulus over other combination stimulus types. The fusiform face area (FFA) and lateral occipital cortex (LO) showed no preferences among the single feature and combination stimulus types. The …

Abstract Unlike frozen snapshots of facial expressions that we often see in photographs, natural facial expressions are dynamic events that unfold in a particular fashion over time. But how important are the temporal properties of expressions for our ability to reliably extract information about a person's emotional state? We addressed this question experimentally by gauging human performance in recognizing facial expressions with varying temporal properties relative to that of a statistically optimal (“ideal”) observer. We found that people recognized emotions just as efficiently when viewing them as naturally evolving dynamic events, temporally reversed events, temporally randomized events, or single images frozen in time. Our results suggest that the dynamic properties of human facial movements may play a surprisingly small role in people's ability to infer the emotional states of others from their facial expressions.

Humans can readily perceive biological motion from point-light (PL) animations, which create an image of a moving human figure by tracing the trajectories of a small number of light points affixed to a moving human body. We have applied ideal observer analysis to a standard biological motion discrimination task involving either full-figure or PL displays. Contrary to current dogma, we find that PL animations can be rich in potential stimulus information but that human observers are remarkably inefficient at exploiting this information. Although our findings do not discount the utility of PL animation, they do provide a realistic measure of the computational challenge posed by biological motion perception.

A constant problem faced by the visual system is the identification of partly occluded objects within the visual scene. Recent experiments have demonstrated that the visual system engages in a process of visual completion, where the hidden parts of objects are filled into the visual representation. Recent experiments have also suggested that there may be a time course to this completion process. Here, we examined the spatiotemporal properties of visual completion by having observers classify figures defined by either luminance-defined or illusory contours and then correlating their decisions with externally added spatiotemporal visual noise. This “response classification” technique allowed us to derive a spatiotemporal correlation map (a “classification movie”) that revealed the locations used by observers at each point in space and time during the stimulus presentation. We found that observers gradually became more influenced by noise at locations corresponding to illusory contours across the first 175 ms of stimulus presentation. Our results are consistent with the idea that there is a time course to the completion process on the order of∼ 175 ms.

Zhang and Luck (Psychological Science, 20, 423–428, 2009) found that perceptual memories are lost over time via sudden death rather than gradual decay. However, they acknowledged that participants may have instead lost memory for the locations of objects. We required observers to recall only a single object. Although the paradigm eliminated the need to maintain object–location bindings, the possibility that observers would use verbal labels increased. To measure the precision of verbal labeling, we included explicit verbal-labeling and label-matching trials. We applied a model that measured the contributions of sudden death, gradual decay, and verbal labeling to recall. Our model-based evidence pointed to sudden death as the primary vehicle by which perceptual memories were lost. Crucially, however, the sudden-death hypothesis was favored only when the verbal-labeling component was …