key: cord-0322546-nb7r5e79
authors: Rideaux, Reuben; West, Rebecca K.; Wallis, Thomas S. A.; Bex, Peter J.; Mattingley, Jason B.; Harrison, William J.
title: Spatial structure, phase, and the contrast of natural images
date: 2021-11-02
journal: bioRxiv
DOI: 10.1101/2021.06.16.448761
sha: 94f9a13af4af83275083f03dc508c051edb92c18
doc_id: 322546
cord_uid: nb7r5e79

The sensitivity of the human visual system is thought to be shaped by environmental statistics. A major endeavour in vision science, therefore, is to uncover the image statistics that predict perceptual and cognitive function. When searching for targets in natural images, for example, it has recently been proposed that target detection is inversely related to the spatial similarity of the target to its local background. We tested this hypothesis by measuring observers’ sensitivity to targets that were blended with natural image backgrounds. Targets were designed to have a spatial structure that was either similar or dissimilar to the background. Contrary to masking from similarity, we found that observers were most sensitive to targets that were most similar to their backgrounds. We hypothesised that a coincidence of phase-alignment between target and background results in a local contrast signal that facilitates detection when target-background similarity is high. We confirmed this prediction in a second experiment. Indeed, we show that, by solely manipulating the phase of a target relative to its background, the target can be rendered easily visible or undetectable. Our study thus reveals that, in addition to its structural similarity, the phase of the target relative to the background must be considered when predicting detection sensitivity in natural images.

The human visual system is tasked with parsing the complexity of natural environments 19 into a coherent representation of behaviourally relevant information. These operations have 20 been shaped by various selective pressures over evolutionary and developmental 21 timescales. Therefore, the perceptual computations that guide cognition and behaviour 22

ultimately serve to extract functional information from rich and complex naturalistic 23 environments (Carandini et & Olshausen, 2001) . For example, a common task is to find a pre-defined 25 target object in a complex or cluttered visual environment. The vast majority of our 26 knowledge of the visual system, however, has been derived from experiments using 27 relatively sparse stimulus displays that are not representative of our typical visual diets. The 28 aim of the present study was to investigate how natural image structure influences target 29 detection. We tested how detection is influenced by the spatial structure, phase, and 30

contrast of natural image backgrounds to determine the features that best predict detection 31 sensitivity. 32

Luminance contrast plays a critical role in most visual tasks. The human visual system 33 is tuned to detect contrast across a range of spatial and temporal frequencies. Neurons in 34 primary visual cortex (V1) are classically understood as processing local regions of oriented 35

contrast that can define the borders of objects (Hubel & Wiesel, 1959) . Such properties of 36 individual neurons govern phenomenal perception and are thought to be shaped by the 37 statistics of natural environments (Barlow, 1961 (Barlow, , 1972 . The encoding of contrast within the 38 visual system is most commonly studied with oriented grating stimuli, such as Gabor 39

wavelets. Grating stimuli are conveniently characterised by a simple set of parameters: 40 orientation, contrast, position, and spatial frequency. From a computational perspective, 41 "Gabor wavelet analyses" allow the decomposition of any image into mathematically 42 tractable component features. Such analyses are relatively simple and are common in many 43 computer vision applications. Early theory suggested that analogous decomposition 44 processes occur in the visual system (Campbell & Robson, 1968 ). However, more recent 45 studies suggest that individual visual neurons encode complex higher-order statistical 46 information that is not necessarily predicted by Gabor parameters (e.g. Cadena et al., 2019) . 47

One common approach to investigate contrast sensitivity in natural conditions is to 48 have observers detect contrast-defined targets embedded in digital photographs or movies. 49 Relative to sensitivity as typically quantified with a uniform background, spatio-temporal 50 contrast sensitivity is diminished when viewing dynamic movies, particularly for lower 51 frequencies (Bex et al., 2009 ). Furthermore, during free viewing of natural movies, the large-52 scale retinal changes caused by saccadic eye movements also diminish sensitivity likely 53 due to forward and backward masking (Dorr & Bex, 2013; Wallis et al., 2015) . In general, 54

such studies have revealed that the sensitivity of the visual system does indeed depend on 55 naturalistic context (Bex & Makous, 2002; Geisler, 2008) . 56

Researchers have further sought to understand the statistical regularities of natural 57 scenes that impact the detectability of targets. For example, various image structures, such 58 as the density of edges within close proximity to the target, negatively impact detection 59 sensitivity (Bex et al., 2009 ; see also Wallis et al., 2015) . Indeed, the discriminability of visual 60 objects can be predicted from the spatial proximity of surrounding visual clutter (Balas et The present study 76 The aim of the present study was to test observers' sensitivity to targets presented on 77 natural image backgrounds. Importantly, we designed the test stimuli a priori such that 78 targets approximated the appearance of, and were aligned with, the local structure of a 79 natural image background, or differed from the local structure. We therefore distinguish 80 target-background alignment from target-background similarity in terms of the stimulus 81 generation procedure (alignment) versus an image statistic (similarity). As shown in Figure  82 1, we automated the placement of targets within natural backgrounds according to oriented 83 contrast energy at different image regions. We created two conditions, one in which targets 84

were aligned with their backgrounds and one in which targets were misaligned with their 85 backgrounds. In contrast to this stimulus generation procedure, target-background similarity 86 is a measure of the correlation between a target and a background without a target. While 87 target-background similarity ranges from 0 -1 for all stimuli, our stimulus generation method 88 results in higher similarity scores for aligned targets than misaligned targets (on average). 89

Based on previous studies showing a negative impact of increasing target-background 90 similarity on detection (Bex et al., 2009; Sebastian et al., 2017) , we expected to find worse 91 detection sensitivity when targets were aligned with the background -and were therefore 92 highly similar -than when they were misaligned relative to the background -and were 93 therefore relatively dissimilar. To anticipate our results, however, we found the opposite, 94

instead revealing that the influence of target-background similarity on detection sensitivity 95 depended almost entirely on the relative phase of the target. 96 

Participants. We used a single-subjects design in which we measured observers' 115 perceptual performance with high precision and treat each observer as a replication (Smith 116 & Little, 2018 the distractor was a natural image with no target filters. The filters were designed to be 142 similar or dissimilar from the underlying natural image structure. We generated target stimuli 143 by blending a source image of a natural object with derivative of Gaussian wavelets 144 (henceforth: filters). The blending process followed four steps: 1) find the dominant 145 orientation of each pixel in a source image, 2) find the relative contrast of each pixel in the 146 image, 3) draw some number of filters at the highest contrast image regions, and 4) combine 147

the filters with a source image. We expand on these steps below. 148

First, we used a steerable filter approach to determine the dominant orientation at 149 every pixel in a given source image (Freeman & Adelson, 1991) . Filters were directional first 150 derivative of Gaussians oriented at 0° and 90°: 151 7 +,-# = 7 . & 168 ! indexes the x-y coordinates of the contrast maxima, and 7 +,-# is the orientation at 169 this location. We then steered a filter at this location as follows: 170

Here, S is the resulting filter stimulus in the range -1 to 1. Note the additional subscript 172 of the filters that indicates that the filters were centred on the location of the contrast maxima, 173 ! . This is trivially achieved by centring the x-y coordinates in Equations 1 and 2 on the 174 coordinates of ! . 175 windowed in a circular aperture with a diameter matching the width of the source image (i.e. 186 2°) and a raised cosine edge, transitioning to zero contrast in 6 pixels. To constrain the filters 187 generated by Equation 9 to appear within the windowed portion of the stimulus, the same 188 aperture was applied to the contrast map, C, prior to generating the stimulus. Any values 189 lower or higher than 0 or 1, respectively, in 345, were clipped. 190

For trials in which multiple filters were combined with a source image, we used an 191 iterative procedure to draw n local maxima from the contrast image. Following the argmax 192 operation in Equation 7, we updated the contrast map to minimise the contrast at the 193 maxima: 194

Where 3 is the contrast map for the n-th filter, and ( ! , ) is a two-dimensional 196

Gaussian with a peak of 1 centred on the location of the maxima ! , and a standard deviation 197 of . is the standard deviation of the basis filters, while is a scaling factor that 198 determines the spatial extent of change in the contrast map. The effect of this adjustment is 199 the creation of a new local maxima at a different location than in the previous iteration. The 200 greater the value of , the greater the spatial spread of filters. The first filter location is always 201 the image region with highest contrast. After accounting for the effect of , subsequent filters 202 are placed in regions of diminishing contrast. In trials in which multiple filters were present, 203 backgrounds were randomly selected as described above. 204 Image selection. The 26,107 images in the THINGS database are grouped into 1,854 205

concepts (e.g. "dog", "cup", "brush" etc), such that there are at least 12 unique, high quality 206 images for each concept (Hebart et al., 2019) . In each testing session (500 trials), we 207 selected 1000 source images from unique concepts such that no two images were drawn 208 from the same concept. The target background was thus always drawn from a different 209 concept than the distractor image. However, it was necessary that some concepts were 210 repeated across testing sessions, and it was also possible that some individual images were 211 also repeated across sessions (but never within sessions). 212

On each trial, we selected two images from the set of 500: one image for the target 213 background, and a second image was the distractor. On half the trials, the target filters were 214 generated from the target background and were therefore aligned with the background, 215

while on the other half of trials they were generated from the distractor image -but blended 216

with the target background -and were therefore misaligned relative to the target 217 background. Target filters were generated from the distractor background on misaligned 218 trials, as opposed to an unused image, so that the filters and their source image were 219 presented on every trial, but we doubt this decision was important to our results. We chose 220 to present two different background images on each trial, rather than, for example, 221

presenting two of the same background images, because we did not want observers to 222 attempt to simply spot the difference between two similar images. Instead, observers had to 223 perform a more natural task of searching unfamiliar and unique backgrounds for targets. 224 Procedure. A typical trial sequence is shown in Figure 2 . Each trial began with a small 225 red fixation spot in the centre of the display, followed by outlines of the upcoming stimulus 226

locations. Natural image backgrounds were followed by a blank of 500ms, after which time 227

the observer reported which of the two patches contained the target filter(s) using the 228 keyboard. Following the observer's response, the image patch with target filters was re-229 displayed for an additional 500ms, outlined in green or red depending on whether the 230 observer's response was correct or incorrect, respectively. Feedback was provided to 231 facilitate observers reaching a stable level of performance. No breaks were programmed 232 but could be taken by withholding a response. Each session included ten repeats of each 233 trial type, all presented in random order, giving 500 trials per session and taking 234

approximately 15 minutes when no breaks were taken. 235 

Where is the sum of weighted linear predictors and !: is the absence or presence 257 of the signal (i.e. 0 or 1, respectively) on the i-th trial. By fitting such a probit model, estimates 258 of the predictor weights : and ! are identical to d' and c, respectively, as calculated in 259

Equation 13 and Equation 14. Whereas these equations fully specify sensitivity and bias in 260 a single interval present/absent judgement task, some small modifications are needed to 261 quantify sensitivity in a two-alternative forced choice task (2AFC) as in our experiment. First, 262 !: denotes whether the target filter(s) appeared in the left or right spatial interval, defined 263 as -.5 or .5, respectively. Similarly, observers' reports (i.e. "target appeared in the left or right 264

interval"), were defined as -1 and 1, respectively. Finally, in a 2AFC, observers have two 265 opportunities to detect the target -once per spatial interval -and so raw d' will be greater 266 than in a single-interval detection design. Therefore, sensitivity (but not bias) must be scaled 267 by

Importantly, we can extend Equation 15 to quantify sensitivity to any number of other 270 predictors, F : 271

[=] = ! + : !:

Consider, for example, the influence of filter amplitude ( ) on an observer's sensitivity: 273

Equation 19

Note that filter amplitude is entered into the model as an interaction with target location 275 because the model's predicted outcome is a spatial report; target amplitude alone can only 276 predict a change in bias. In preliminary model fits, we found that such bias was not 277 significantly different from zero, and thus included only interactive terms to facilitate 278 interpretability of the standard bias term, ! . We selected other model predictors according 279

to the model that produced the lowest Akaike information criterion (AIC; see below). 280

Finally, we implemented this model as a multilevel GLM (GLMM) to partially pool 281 coefficient estimates across observers (Gelman & Hill, 2007 

Here, F,H is the offset for each predictor and observer , relative to the parameter's 288 mean , contingent on the parameters' estimated population variance * . The partial pooling 289 of observers' data in a GLMM results in more extreme values being pulled toward the 290 population mean estimate. Note that in our experiment, however, such pooling is relatively 291 minor due to the large number of trials, and therefore high precision, of each observer's 292 estimated performance, as well as the relatively small number of observers. Because 293 images were drawn randomly from trial to trial from a pool of tens of thousands of images, 294

we did not expect many, if any, repeats of each image. We therefore did not model the 295 background images as a random effect, but we note that such a design could be chosen in 296 future to estimate the variance associated with each tested background. 297

We entered into the model the factors target amplitude, number of filters, and target-298

background alignment, which, as noted above, were each entered as an interaction with the 299 spatial interval of the target. In hindsight, our inclusion of the condition in which target 300 amplitude was 0 was unnecessary. For all such trials, therefore, we set all predictors to have 301 a value of 0 so they were omitted from model calculations. The model fit was improved by 302 including nonlinear terms by raising amplitude and number of Gabors to the exponents 0.5 303 and 2, respectively. We further tested all combinations of interactions, but none improved 304 the model fit as assessed by the Akaike information criterion. 305

Post-hoc analysis of interaction between the number of filters and filter 306 amplitude. We modelled the joint influence of number of filters and filter amplitude on 307

proportion correct as a two-dimensional surface (see Figure 5B ). The surface is defined as: 308

Where z is the surface of estimated proportion correct, f<α J , α K E is a cumulative 310 probability function relating target amplitude to accuracy according to a threshold and 311 variance, α J and α $ , respectively, and f<n J , n K E is a cumulative probability function relating 312 the number of filters to accuracy according to a threshold and variance, n J and n $ , 313

respectively. Here, ⊙ refers to the element wise product of cumulative distributions. This 314 function can be thought of as a two-dimensional psychometric function, with separable 315 means and standard deviations. The input parameters into the cumulative functions were 316 free parameters, fit by minimising the summed squared error between the average 317 proportion correct and z using Matlab's fminsearch( background to include only the same regions as the locations of target filters. This was 329 achieved by first computing a contrast map in which two-dimensional Gaussians were 330 positioned at each target location. Gaussians had the same standard deviation as the target 331 filters and had their peaks normalised to one. We then computed the elementwise product 332 of the source image and this contrast map, which produced the background image entered 333

into Equation 23. Note that our method to generate stimuli and target-background similarity 334 both depend on oriented contrast within the frequency band of the target. Variations in 335 structural similarity for the aligned and misaligned targets are shown in Figure 6 . 336

We tested observers' ability to detect target filters that were blended with natural image 338 backgrounds. Targets were designed such that they were either aligned or misaligned with 339 the structure of the background. We tested detection of 1, 2, 4, 8 and 16 target filters and 340 across a range of target amplitudes. We first describe our results in terms of raw accuracy, 341

and then report the results of our modelling analysis in which we quantified observers' 342 sensitivity in terms of d'. 343

The influence of target amplitude, number of filters, and target-background alignment 344

The proportion of correct responses as a function of each factor is shown in Figure 3 . 345 The amplitude of the target most clearly impacts accuracy, such that accuracy increases 346 approximately linearly with (log) target amplitude ( Figure 3A ). Although the relationship 347 between proportion correct and the number of filters is less consistent ( Figure 3B) , there is 348 a general increase in accuracy with increasing number of target filters. As described below, 349 however, the relationship between number of filters and sensitivity was not significant. We 350 also found a highly consistent effect of target-background alignment across observers 351 ( Figure 3C ). Contrary to our expectations based on recent reports of similarity masking 352 (Sebastian et al., 2017) , however, we found better performance when target filters were 353 aligned with their backgrounds than when they were misaligned with their backgrounds. This 354 difference in performance is significant at the group level (t(2) = 5.862, p = 0.028, d = 3.38), 355

but we more formally quantify the relationship between these factors in a GLMM below. 356

Importantly, given the high measurement precision of these data (1000 observations per 357 data point shown in Figure 3C ), we can treat each observer as an independent replication 358 of the effect, regardless of any specific inferential statistic (Smith & Little, 2018) . 

We estimated detection sensitivity as a function of the experimental factors with a 366 GLMM. Modelled sensitivity is shown in Figure 4 in the same format as Figure 3 . Target  367 amplitude and target-background alignment were both significant contributors to the model. 368

Importantly, as shown in Figure 4C , sensitivity was greater when target filters were aligned 369 with the background than when they were misaligned. The number of filters did not predict 370 sensitivity, which is consistent with the relatively noisy relationship between accuracy and 371 the number of filters as shown in Figure 3B . We manipulated the number of filters because we expected to find an improvement in 379 performance with increasing filter number. The lack of a main effect of filter number in the 380 results above, therefore, was unexpected. Although not critical to our primary interest in the 381 influence of target-background alignment, we tested whether the number of filters interacted 382 with target amplitude using a more direct test than the full GLMM above. Data points in 383 Figure 5A show mean proportion correct responses, marginalised to show the influence of 384 the number of filters for each target amplitude. At the two highest target amplitudes tested 385 (top two lines), there is indeed an effect of the number of filters: as the number of filters 386

increases, so too does observers' accuracy. The same data points are shown in Figure 5B  387 arranged as a surface that maps proportion correct as conditional on the combination of 388

conditions. We interpolated these points as a two-dimensional surface function that 389 quantifies the interaction between number of filters and amplitude (see Methods). The warm 390 colours clustering in the top right corner reveal that increasing the number of filters in the 391 targets has the strongest effect at higher target amplitudes. 392 

The influence of target-background similarity 399

The perceptual performance described above reveals that observers were better able 400

to detect targets on natural image backgrounds when the targets were aligned with the 401 underlying spatial structure of the background than when the targets were misaligned with 402 the background. These results are contrary to our expectations based on the data of 403

Sebastian et al (2017) who found that sensitivity is negatively correlated with the structural 404 similarity between target and background, a metric that scales from zero (no similarity) to 405 one (perfect similarity). We therefore next tested whether observers' performance was 406

instead positively correlated with target-background similarity using the same analysis of 407 similarity as in this previous study (Equation 23 ). 408 Shown in Figure 6A are the histograms of similarity for all trials across observers, 409

which, by design, can be separated according to the filter alignment relative to the 410 background. The dashed vertical line shows the median similarity of all trials, regardless of 411 target-background alignment. By isolating trials from each condition according to whether 412 they fall above or below this arbitrary cut-off, we can test whether accuracy depends more 413 on similarity or target-background alignment. Figure 6B shows the proportion of correct 414 target detections for aligned and misaligned targets that were most and least similar to the 415 background, respectively. As per the main analyses above, accuracy was greater for aligned 416 Proportion correct than misaligned trials. The more diagnostic analysis is shown in Figure 6C , in which the 417 proportion of correct target detections are shown for aligned trials that were less similar than 418 the included misaligned trials (i.e. we limit the analyses to the tails of the similarity 419 distributions). We again find that accuracy was higher for the aligned condition than the 420 misaligned condition, despite the aligned targets having lower similarity with the background 421 than the misaligned trials. Therefore, target-background alignment predicts performance 422 much more strongly than target-background similarity. 423 424

In a final analysis of perceptual performance, we attempted to replicate the findings of 435 Sebastian et al (2017) using the condition in our experiment that is most analogous to the 436 one in theirs, namely, the misaligned condition. In both this previous study and the 437 misaligned condition of the present study, the blending of target filters and their backgrounds 438 did not depend on any structural alignment. Instead, any incidental alignment can be 439 quantified in terms of structural similarity. We therefore tested whether we found an inverse 440 relationship between target-background similarity and detection accuracy for the misaligned 441 condition. Figure 7 shows proportion correct for trials in similarity bins (bin width = 0.1) for 442 each observer. We modelled these data with logistic regression, with random intercept and 443 slopes grouped by observer (i.e. a logistic GLMM). Fits are shown as solid lines in Figure 7 . 444

There is a clear negative relationship between structural similarity and accuracy, with a 445 mean slope of -0.96 (population standard deviation = 0.75). Similarity was not a significant 446 predictor in the model (p = 0.13), but the trends are nonetheless consistent across observers 447

and also with the data of Sebastian et al. Indeed, Bex and Makous (2002) speculated that this dependence of local contrast on phase 473 alignment explains a loss of sensitivity to phase-scrambled natural images. We tested this 474 hypothesis directly in Experiment 2. Note, however, that, while phase-mismatched target 475 filters reduce local contrast, they may not be less visible. See, for example, the 476 demonstrations in Figure 8B . While the phase-mismatched filter has a lower local contrast 477 than the phase-matched filter, the phase-mismatched filter is conspicuous. Indeed, within 478 each half of these images, the absolute change of luminance is the same, regardless of filter 479 phase. It therefore remains an open question as to how this manipulation will affect 480 observers' sensitivity. 481 

The results presented above reveal that the alignment of target filters with the spatial 496 structure of the background determines detection sensitivity at least somewhat 497

independently of target-background similarity. In Experiment 2 we tested our hypothesis that 498 aligned targets are easier to detect because their amplitude is additive with the background 499 amplitude, increasing local contrast. We therefore compared detection sensitivity to target 500 filters that were either aligned or misaligned, and were either phase-matched or phase-501 mismatched with the original source image. 502

All methods were identical to those of the preceding experiment, with the following 504 changes. This experiment was carried out in our testing lab on a Display++ monitor 505 (Cambridge Research Systems) with 14bit luminance precision (i.e. our local lockdown had 506 lifted). The experimental design was a 2 (alignment: aligned versus misaligned) x 2 (phase: 507 phase-matched versus phase-mismatched) factorial design (see Figure 8 for example 508 stimuli). All target filters had an amplitude of 0.15, which, based on the data shown in Figure  509 4, we expected to yield a mean d' of approximately 1. In all trials, there were four filters. 510

Importantly, on half the trials, target filters were blended with the background as per 511

Experiment 1, whereas in the other half of the trials the phase of the filters were reversed 512 before blending. Note that, in the misaligned condition, the phase of the filters relative to the 513 background is somewhat arbitrary, so in the analysis we average across these trials. Each 514 In Experiment 2, we included conditions that provided an opportunity to replicate our 518 findings from Experiment 1. As demonstrated in the top left panel of Figure 8 , we included 519 a condition in which target filters were both aligned and phase-matched to the natural 520 background structure, as per the aligned condition of Experiment 1. We first compare 521 observers' accuracy in this condition with the accuracy in the misaligned condition. The 522 results are shown as connected points in Figure 9A , and reveal better performance in the 523 (phase-matched) aligned condition than the misaligned condition for all observers. We 524 therefore replicate the results from Experiment 1 under strict laboratory conditions. Also 525 shown in Figure 9A are the results from the phase-mismatched condition, in which target 526 filters were aligned with their backgrounds, but had their phase inverted. Importantly, phase-527 matched and phase-mismatched targets were well equated on similarity (phase-matched 528 and phase-mismatched average similarities were .7 and 0.71, respectively). In the phase-529 mismatched condition, however, all observers were close to chance level, revealing they 530

were unable to detect these targets (mean accuracy = 49%; RR = 45%, RW = 50%, WH = 531 51%). 532 

We again tested whether there was an inverse relationship between accuracy and 541 similarity in the condition most closely matching the condition tested by Sebastian et al 542 (2017), i.e. the misaligned condition (see Figure 7) . We again used a logistic GLMM, and 543 similarity was binned in 0.1 steps. In contrast to the fitted model in Experiment 1, however, 544

we found a non-significant positive relationship between similarity and proportion correct 545 (slope = 0.33, population standard deviation = 0.28, p = 0.596). Fits to observers' data are 546

shown in Figure 9B -D. 547

Discussion 548

The aim of Experiment 2 was to test the prediction that targets aligned with their natural 549 image backgrounds are easier to identify than targets that are misaligned with their 550 backgrounds (i.e. the results of Experiment 1) due to a difference in local contrast. The 551 differences in local contrast across these conditions results from contrast additivity in the 552 aligned condition when filters are phase-matched with their background. We tested this 553 the filters were aligned with the background structure. Inverting the phase of target filters 555 aligned with their background has a subtractive effect on local contrast, and, as we 556 expected, rendered observers incapable of detecting the targets (Figure 9 ). The results of 557 Experiment 2 thus support the notion that target-background similarity is not a useful metric 558 of the detectability of targets per se, whereas the interaction between relative phase and 559 structural alignment is critical. 560

We next aimed to develop a model that captures the key results reported for 562 Experiment 1 and Experiment 2. We hoped to account for the finding that aligned targets 563 are more accurately detected than misaligned targets, and that this effect of alignment 564 depends on the relative phase of target and background. These effects suggest that 565

observers are tuning to local changes in the images caused by the additivity of filter and 566

background luminance. The model is therefore based on simple luminance and contrast 567 detection mechanisms like those involved in the generation of our stimuli (Figure 1 ). On 568 each trial, the model detects difference in various image statistics across the target and 569 distractor images, and then generates a response based on these differences. Simulated 570 responses were determined by fitting the model output to observers' responses in 571

Experiment 1. We then tested whether the fitted model reproduced the key Experiment 2 572 results. 573

On each trial, the model was given the target image and the distractor image. The 574 maximum contrast of each image was found by taking the maximum value of contrast maps 575

as computed in Equations 1 -6. The maximum luminance extreme of each image was the 576 maximum absolute deviation of each image from mid-grey, capturing both local minima and 577 maxima in images across trials. For each trial and each metric, we computed a ratio between 578 left and right images: 579 values indicate greater extremes in the right image. We weighted these metrics by fitting 584 them to observers' responses (i.e. left or right spatial interval) using logistic regression. We 585 then analysed the model predictions as per the behavioural analyses. We built different 586 models that included 1) just the absolute luminance peak of stimuli, 2) just the contrast 587 energy maxima of stimuli, or 3) both. 588

The best model was one that detects the absolute luminance peak and the maximum 589 contrast energy within the target and distractor images. As shown in Figure 10 , this model 590

reproduces the qualitative patterns of performance observed in Experiment 1 (compare the 591 model data in Figure 10 with the empirical data in Figure 3) . Importantly, each image statistic 592

significantly contributes to the model (p's < 0.001), and including both parameters provided 593 a better fit than including either parameter alone based on formal model comparison (chi-594 square test compared with the next best model: * (Δ = 4) = 114.3, < 0.001). Note that 595 the generative model responses are fit to the observer data based solely on the image-596 computable features, not on the labels of the experimental conditions (e.g. the data are not 597 fit to aligned vs misaligned conditions) -yet the model reproduces the same patterns of data 598 across conditions as observers. Adding the model's predicted response to the signal 599 detection model also improved estimates of observers' d' ( * (Δ = 6) = 163.1, < 0.001). 600

These model results are consistent with the notion that adding filters to the image causes 601 local peaks in absolute luminance and contrast that can differentiate the target image from 602 the distractor image (Bex & Makous, 2002) . As described next, however, this model cannot 603 account for the findings in Experiment 2. 604 605 

We fit the model to observers' responses from Experiment 1, as shown above, and 608

tested whether the fitted model could predict observers' responses to Experiment 2. The 609

critical test is whether the model falls to chance when filters were spatially aligned but phase-610 inverted relative to their backgrounds (i.e. the phase-mismatched condition). As shown in 611 Figure 11 , however, model performance was well below chance in this condition. This below-612 chance performance occurs because phase-inversion reduces the luminance and contrast 613 peaks of the target image to below those of the distractor image, resulting in the model 614

reporting the distractor as the target more often than not. The model again reproduces the 615 effect of alignment, but overestimates the size of the effect. The overestimation may have 616 resulted from the changes we made between experiments, including using different displays 617 and filter amplitudes. These differences are less relevant than the model's gross 618 misestimation of the phase-mismatched condition as described below. 619 

The relatively poor fit to the phase-mismatched condition of Experiment 2 is 623

informative. Observers' data cannot be explained fully by assuming that they adopted a 624 simple rule in which they detected luminance and contrast peaks. For phase-matched trials, 625 therefore, the superior sensitivity to aligned filters over misaligned filters cannot be solely 626 accounted for by tuning to local peaks. We attempted to improve the fit using other image 627 pocket" filter-rectify-filter model (Harrison & Bex, 2016; Landy, 2013) , but none improved the 629 fits of the model. Rather than taking the maximum contrast or luminance extreme, we also 630 tried using k-maxima (up to k = 1000), which also did not improve the fits. Observers' 631 performance, therefore, escapes a relatively straightforward low-level explanation. 632

The aim of the present study was to test observers' ability to detect contrast-defined 634 targets that have been blended with natural image backgrounds. We designed the target 635 filters so that their orientations were either aligned or misaligned with the local background 636 structure. Based on the recent report that detection sensitivity is inversely related to target-637 background similarity (Sebastian et al., 2017) , we expected to find worse performance in 638 the aligned condition relative to the misaligned condition based on the notion that aligned 639 targets would have higher target-background similarity than misaligned targets. Across two 640 experiments, however, we found superior detection of targets that were designed to be 641 aligned and phased-matched to the structure of the background compared with targets that 642

were designed to be misaligned with their backgrounds and were thus lower in similarity. As 643 noted below, our goal was not to replicate the study by Sebastian et al, but instead to test 644 the role of target-background similarity in target detection using a novel approach. 645

Our experiments show that observers' sensitivity does not linearly scale with similarity 646 in all cases. In Experiment 1 we found a positive relationship between similarity and 647 sensitivity: observers were most sensitive to targets in which target filters were aligned with, 648

and most similar to, background structure ( Figure 3C and Figure 4C ). We replicated this 649 finding in Experiment 2 ( Figure 9A ). When we limited our analyses to only trials in which 650 target filters were misaligned with the background structure, we found mixed results across 651 the two experiments: a negative relationship between detection accuracy and target-652 background similarity in Experiment 1 (Figure 7) , but a positive relationship in Experiment 2 653

( Figure 9C -D). The cause of this difference in results across experiments is not clear, but 654

we note that we did not design either experiment to specifically measure the relationship 655 between similarity and sensitivity in this way, and neither model was significant. Regardless, 656

there was no clear evidence of an inverse relationship as we expected. 657

The limitation of a specific structural similarity metric as a predictor of sensitivity in our 658 study is most apparent in our Experiment 2 results. By matching or inverting the phase of 659 filters aligned with the background structure, we produced target-background images that 660

were equivalent in similarity but were different in their detectability (Figure 8 and Figure 9A ). 661

When phase was inverted relative to the background, observers' performance was at 662 chance level. The phase-mismatched condition therefore removed the information 663 observers depended on to perform the task (i.e. contrast to the target image and observers' performance ( Figure 3B and Figure 4B ). We expected to 675 find such a relationship based on the simple principle that there is an additional opportunity 676

to detect a target for each filter added (Macmillan & Creelman, 2004 ). On closer inspection 677 of the proportion correct data presented in Figure 3B , only a single data point in each of RW 678 and WH's data are inconsistent with the expected trend for all observers. The lack of a 679 statistically robust finding, therefore, may be due to our limited number of observers. A clear 680 main effect of the number of filters was possibly also obscured by an interaction with target 681 amplitude as shown in Figure 5 . It is interesting that our generative model also produced a 682 somewhat noisy relationship between accuracy and the number of filters. It is possible that 683 observers outperformed our model in the phase-mismatched condition of Experiment 2 684 because they integrated information over multiple locations rather than using the maximum 685 in each image. strategy. This insight is obviously limited in its usefulness, because a template-matching 722 strategy is equivalent to the computations performed in our model. 723

Our results are consistent with those of Neri (2011), who investigated the influence of 724 target phase relative to the structure of a natural image background and how these effects 725 differ when the stimulus is inverted. Neri found that, when the background is upright, 726 observers tune to feature detectors that are in-phase with a natural edge. When the 727 background is inverted, observers' tuning is less phase-aligned with the natural edge. These 728 results suggest that upright scenes produce a bias in visual processing that steers 729 observers' filtering toward the local phase of natural structures. These results are consistent 730 with our own finding that observers are most sensitive to filters that are spatially aligned, 731

and in-phase, with the background. 732

The extent to which the design of visual targets determines detectability in natural 733 backgrounds is thus clearly an important consideration. As shown in a demonstration by 734 Sebastian et al, it is incontrovertible that there are some targets for which phase is 735 unimportant for visibility. We reproduce such a demonstration in the top row of Figure 12 . 736 We question, however, the relevance of a similarity metric in explaining the visibility of the 737 target in this demonstration, considering that similar demonstrations can be produced in 738 which target-background similarity is greater, and yet the target is easily visible (bottom row, 739 Figure 12 ). The importance of target-background phase (in)variance likely depends on 740 multiple factors, such as target design, as well as differences in the background in the region 741 of the target (i.e. 'partial masking', see Sebastian et al., 2020) . In addition to testing target 742 visibility in different backgrounds based on contrast, luminance, and similarity (Sebastian et  743 al bottom right panel (0.15 bottom right vs 0.12 in the top right) . 754 We used images from the THINGS database as naturalistic backgrounds. The 755

THINGS database is a recently released database with over 26,000 images from 1854 756 categories (Hebart et al., 2019) . Relatively little has been reported about the basic statistical 757

properties of the images in this database, and so it is possible that they may deviate from 758 what one may expect from typical natural images 1 . In Figure 13 we show the mean image 759 spectra as a function of spatial frequency and orientation. This analysis shows that THINGS 760

images have, on average, the same basic image spectra as typically found in natural 761

images: contrast energy decreases with increasing spatial frequency ( Figure 13A ), and 762

there is an over representation of cardinal orientations ( Figure 13B ). It is therefore unlikely 763 that something in particular about the distribution of contrast energy in the THINGS images 764 played a key role in our results. However, it is likely that showed that edge detection in visual noise can be greatly influenced by the allocation of 786 visual attention. Taken together, these findings suggest that the detectability of targets in 787 the present study may have depended on the specific objects in background images, and 788 how combinations of low-level and high-level factors guided observers' visual attention. 789

However, we did not design our experiments to examine such possible differences across 790 object images. The availability of repositories such as the THINGS database makes such 791 questions possible to address in future studies. 792

In summary, we tested observers' ability to detect targets in natural image 793 backgrounds. Observers were most sensitive to targets when they were aligned and phase-794 matched with their backgrounds. Inverting the phase of aligned targets reduced observers' 795 detection performance to chance. To best model the image factors that predict human 796 sensitivity to contrast defined targets in natural backgrounds, therefore, the phase of the 797 target relative to the background must be considered. 798 2 This function has the form: = ( − * | (2 ) | T ) * < * < (2 )E + + 1E, where E is contrast energy, is orientation in radians, and , , , and are free parameters. See Harrison (2021) for a more detailed analysis of the THINGS images. 

JBM was supported by 809 the Australian Research Council (ARC) Centre of Excellence for Integrative Brain Function 810 (ARC Centre Grant CE140100007). PJB was supported by a National Institute of

A summary-statistic representation in 816 peripheral vision explains visual crowding

Possible Principles Underlying the Transformations of Sensory 819

Sensory Communication

Single Units and Sensation: A Neuron Doctrine for Perceptual 822 Psychology? Perception

Fitting Linear Mixed-Effects Models 824 Using lme4

The effects of contrast, spatial scale, and orientation 827 on foveal and peripheral phase discrimination

Spatial frequency, phase, and the contrast of natural 829 images

Contrast sensitivity in natural scenes 832 depends on edge as well as spatial frequency structure

The Psychophysics Toolbox

Deep convolutional models improve predictions of macaque V1 837 responses to natural images

Application of Fourier analysis to the visibility of 840 gratings

Do we know what the early visual system does? Journal of 843

Peri-Saccadic Natural Vision

Relations between the statistics of natural images and the response 848 properties of cortical cells

The design and use of steerable filters

Visual Perception and the Statistical Properties of Natural Scenes

Data Analysis Using Regression and Multilevel/Hierarchical 857 Models

Crowding changes appearance

Crowding follows the binding of relative 861 position and orientation

Luminance and contrast of images in the THINGS database

Integrating retinotopic features in spatiotopic 865 coordinates

A unifying model of orientation crowding in peripheral 868 vision

Reply to Pachai et al

Visual crowding is a combination of an increase of 873 positional uncertainty, source confusion, and featural averaging

Voluntary control of illusory contour formation. 876 Attention, Perception, and Psychophysics

Perceived contrast in complex images

THINGS: A database of 1,854 object concepts and more than 26,000 882 naturalistic object images

Receptive fields of single neurones in the cat's striate 885 cortex

What's 887 new in psychtoolbox-3

Texture analysis and perception

Positional uncertainty in peripheral and 891 amblyopic vision

Detection theory: A user's guide

Global Properties of Natural Scenes Shape Local Properties of Human Edge 895

Dynamic engagement of human motion detectors across space-time 897 coordinates

Object segmentation controls image reconstruction from natural scenes

How close are we to understanding v1? Neural 902 Computation

The human visual system is 904 optimised for processing the spatial information in natural visual images

The VideoToolbox software for visual psychophysics: Transforming 907 numbers into movies

Rethinking the role of top-down attention 909 in vision: Effects attributable to a lossy representation in peripheral vision

Constrained sampling experiments 912 reveal principles of detection in natural scenes

Local reliability weighting explains 916 identification of partially masked objects in natural images

Natural image statistics and neural 920 representation

Small is beautiful: In defense of the small-N design

Prior object-knowledge sharpens 925 properties of early visual feature-detectors

Image correlates of crowding in natural scenes

Sensitivity to gaze-contingent contrast 930 increments in naturalistic movies: An exploratory report and model comparison

Image content is more important than Bouma's Law for scene metamers