Ecker, Alexander S.

[["dc.contributor.author","Michaelis, Claudio"],["dc.contributor.author","Bethge, Matthias"],["dc.contributor.author","Ecker, Alexander S."],["dc.date.accessioned","2020-03-20T09:11:18Z"],["dc.date.available","2020-03-20T09:11:18Z"],["dc.date.issued","2018"],["dc.description.abstract","We tackle the problem of one-shot segmentation: finding and segmenting a previously unseen object in a cluttered scene based on a single instruction example. We propose a novel dataset, which we call $\\textit{cluttered Omniglot}$. Using a baseline architecture combining a Siamese embedding for detection with a U-net for segmentation we show that increasing levels of clutter make the task progressively harder. Using oracle models with access to various amounts of ground-truth information, we evaluate different aspects of the problem and show that in this kind of visual search task, detection and segmentation are two intertwined problems, the solution to each of which helps solving the other. We therefore introduce $\\textit{MaskNet}$, an improved model that attends to multiple candidate locations, generates segmentation proposals to mask out background clutter and selects among the segmented objects. Our findings suggest that such image recognition models based on an iterative refinement of object detection and foreground segmentation may provide a way to deal with highly cluttered scenes."],["dc.identifier.arxiv","1803.09597v2"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63370"],["dc.language.iso","en"],["dc.title","One-Shot Segmentation in Clutter"],["dc.type","preprint"],["dc.type.internalPublication","unknown"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.firstpage","3730"],["dc.bibliographiccitation.lastpage","3738"],["dc.contributor.author","Gatys, Leon A."],["dc.contributor.author","Ecker, Alexander S."],["dc.contributor.author","Bethge, Matthias"],["dc.contributor.author","Hertzmann, Aaron"],["dc.contributor.author","Shechtman, Eli"],["dc.date.accessioned","2020-03-31T14:43:41Z"],["dc.date.available","2020-03-31T14:43:41Z"],["dc.date.issued","2017"],["dc.description.abstract","Neural Style Transfer has shown very exciting results enabling new forms of image manipulation. Here we extend the existing method to introduce control over spatial location, colour information and across spatial scale. We demonstrate how this enhances the method by allowing high-resolution controlled stylisation and helps to alleviate common failure cases such as applying ground textures to sky regions. Furthermore, by decomposing style into these perceptual factors we enable the combination of style information from multiple sources to generate new, perceptually appealing styles from existing ones. We also describe how these methods can be used to more efficiently produce large size, high-quality stylisation. Finally we show how the introduced control measures can be applied in recent methods for Fast Neural Style Transfer."],["dc.identifier.doi","10.1109/CVPR.2017.397"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63421"],["dc.language.iso","en"],["dc.relation.conference","2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"],["dc.relation.eventend","2017-07-26"],["dc.relation.eventlocation","Honolulu, HI, USA"],["dc.relation.eventstart","2017-07-21"],["dc.relation.isbn","978-1-5386-0457-1"],["dc.relation.ispartof","2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"],["dc.relation.issn","1063-6919"],["dc.title","Controlling Perceptual Factors in Neural Style Transfer"],["dc.type","conference_paper"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.artnumber","5"],["dc.bibliographiccitation.firstpage","1"],["dc.bibliographiccitation.issue","12"],["dc.bibliographiccitation.journal","Journal of Vision"],["dc.bibliographiccitation.lastpage","29"],["dc.bibliographiccitation.volume","17"],["dc.contributor.author","Wallis, Thomas S. A."],["dc.contributor.author","Funke, Christina M."],["dc.contributor.author","Ecker, Alexander S."],["dc.contributor.author","Gatys, Leon A."],["dc.contributor.author","Wichmann, Felix A."],["dc.contributor.author","Bethge, Matthias"],["dc.date.accessioned","2020-03-31T14:26:39Z"],["dc.date.available","2020-03-31T14:26:39Z"],["dc.date.issued","2017"],["dc.description.abstract","Our visual environment is full of texture-\"stuff\" like cloth, bark, or gravel as distinct from \"things\" like dresses, trees, or paths-and humans are adept at perceiving subtle variations in material properties. To investigate image features important for texture perception, we psychophysically compare a recent parametric model of texture appearance (convolutional neural network [CNN] model) that uses the features encoded by a deep CNN (VGG-19) with two other models: the venerable Portilla and Simoncelli model and an extension of the CNN model in which the power spectrum is additionally matched. Observers discriminated model-generated textures from original natural textures in a spatial three-alternative oddity paradigm under two viewing conditions: when test patches were briefly presented to the near-periphery (\"parafoveal\") and when observers were able to make eye movements to all three patches (\"inspection\"). Under parafoveal viewing, observers were unable to discriminate 10 of 12 original images from CNN model images, and remarkably, the simpler Portilla and Simoncelli model performed slightly better than the CNN model (11 textures). Under foveal inspection, matching CNN features captured appearance substantially better than the Portilla and Simoncelli model (nine compared to four textures), and including the power spectrum improved appearance matching for two of the three remaining textures. None of the models we test here could produce indiscriminable images for one of the 12 textures under the inspection condition. While deep CNN (VGG-19) features can often be used to synthesize textures that humans cannot discriminate from natural textures, there is currently no uniformly best model for all textures and viewing conditions."],["dc.identifier.doi","10.1167/17.12.5"],["dc.identifier.pmid","28983571"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63417"],["dc.language.iso","en"],["dc.relation.eissn","1534-7362"],["dc.relation.issn","1534-7362"],["dc.title","A parametric texture model based on deep convolutional features closely matches texture appearance for humans"],["dc.type","journal_article"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.artnumber","062707"],["dc.bibliographiccitation.issue","6"],["dc.bibliographiccitation.journal","Physical Review. E"],["dc.bibliographiccitation.volume","91"],["dc.contributor.author","Gatys, Leon A."],["dc.contributor.author","Ecker, Alexander S."],["dc.contributor.author","Tchumatchenko, Tatjana"],["dc.contributor.author","Bethge, Matthias"],["dc.date.accessioned","2020-04-01T11:34:52Z"],["dc.date.available","2020-04-01T11:34:52Z"],["dc.date.issued","2015"],["dc.description.abstract","Synaptic unreliability is one of the major sources of biophysical noise in the brain. In the context of neural information processing, it is a central question how neural systems can afford this unreliability. Here we examine how synaptic noise affects signal transmission in cortical circuits, where excitation and inhibition are thought to be tightly balanced. Surprisingly, we find that in this balanced state synaptic response variability actually facilitates information transmission, rather than impairing it. In particular, the transmission of fast-varying signals benefits from synaptic noise, as it instantaneously increases the amount of information shared between presynaptic signal and postsynaptic current. Furthermore we show that the beneficial effect of noise is based on a very general mechanism which contrary to stochastic resonance does not reach an optimum at a finite noise level."],["dc.identifier.doi","10.1103/PhysRevE.91.062707"],["dc.identifier.pmid","26172736"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63430"],["dc.language.iso","en"],["dc.relation.eissn","1550-2376"],["dc.relation.issn","1539-3755"],["dc.title","Synaptic unreliability facilitates information transmission in balanced cortical populations"],["dc.type","journal_article"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.artnumber","e1006897"],["dc.bibliographiccitation.issue","4"],["dc.bibliographiccitation.journal","PLoS Computational Biology"],["dc.bibliographiccitation.volume","15"],["dc.contributor.author","Cadena, Santiago A."],["dc.contributor.author","Denfield, George H."],["dc.contributor.author","Walker, Edgar Y."],["dc.contributor.author","Gatys, Leon A."],["dc.contributor.author","Tolias, Andreas S."],["dc.contributor.author","Bethge, Matthias"],["dc.contributor.author","Ecker, Alexander S."],["dc.date.accessioned","2020-03-18T10:51:24Z"],["dc.date.available","2020-03-18T10:51:24Z"],["dc.date.issued","2019"],["dc.description.abstract","Despite great efforts over several decades, our best models of primary visual cortex (V1) still predict spiking activity quite poorly when probed with natural stimuli, highlighting our limited understanding of the nonlinear computations in V1. Recently, two approaches based on deep learning have emerged for modeling these nonlinear computations: transfer learning from artificial neural networks trained on object recognition and data-driven convolutional neural network models trained end-to-end on large populations of neurons. Here, we test the ability of both approaches to predict spiking activity in response to natural images in V1 of awake monkeys. We found that the transfer learning approach performed similarly well to the data-driven approach and both outperformed classical linear-nonlinear and wavelet-based feature representations that build on existing theories of V1. Notably, transfer learning using a pre-trained feature space required substantially less experimental time to achieve the same performance. In conclusion, multi-layer convolutional neural networks (CNNs) set the new state of the art for predicting neural responses to natural images in primate V1 and deep features learned for object recognition are better explanations for V1 computation than all previous filter bank theories. This finding strengthens the necessity of V1 models that are multiple nonlinearities away from the image domain and it supports the idea of explaining early visual cortex based on high-level functional goals."],["dc.identifier.doi","10.1371/journal.pcbi.1006897"],["dc.identifier.pmid","31013278"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63344"],["dc.language.iso","en"],["dc.relation.eissn","1553-7358"],["dc.relation.issn","1553-7358"],["dc.title","Deep convolutional models improve predictions of macaque V1 responses to natural images"],["dc.type","journal_article"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.artnumber","2654"],["dc.bibliographiccitation.issue","1"],["dc.bibliographiccitation.journal","Nature Communications"],["dc.bibliographiccitation.volume","9"],["dc.contributor.author","Denfield, George H."],["dc.contributor.author","Ecker, Alexander S."],["dc.contributor.author","Shinn, Tori J."],["dc.contributor.author","Bethge, Matthias"],["dc.contributor.author","Tolias, Andreas S."],["dc.date.accessioned","2022-03-01T11:45:57Z"],["dc.date.available","2022-03-01T11:45:57Z"],["dc.date.issued","2018"],["dc.identifier.doi","10.1038/s41467-018-05123-6"],["dc.identifier.pii","5123"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/103508"],["dc.language.iso","en"],["dc.notes.intern","DOI-Import GROB-531"],["dc.relation.eissn","2041-1723"],["dc.rights.uri","https://creativecommons.org/licenses/by/4.0"],["dc.title","Attentional fluctuations induce shared variability in macaque primary visual cortex"],["dc.type","journal_article"],["dc.type.internalPublication","unknown"],["dspace.entity.type","Publication"]]

[["dc.contributor.author","Günthner, Max F."],["dc.contributor.author","Cadena, Santiago A."],["dc.contributor.author","Denfield, George H."],["dc.contributor.author","Walker, Edgar Y."],["dc.contributor.author","Gatys, Leon A."],["dc.contributor.author","Tolias, Andreas S."],["dc.contributor.author","Bethge, Matthias"],["dc.contributor.author","Ecker, Alexander S."],["dc.date.accessioned","2020-03-18T14:29:36Z"],["dc.date.available","2020-03-18T14:29:36Z"],["dc.date.issued","2019"],["dc.description.abstract","Divisive normalization (DN) has been suggested as a canonical computation implemented throughout the neocortex. In primary visual cortex (V1), DN was found to be crucial to explain nonlinear response properties of neurons when presented with superpositions of simple stimuli such as gratings. Based on such studies, it is currently assumed that neuronal responses to stimuli restricted to the neuron's classical receptive field (RF) are normalized by a non-specific pool of nearby neurons with similar RF locations. However, it is currently unknown how DN operates in V1 when processing natural inputs. Here, we investigated DN in monkey V1 under stimulation with natural images with an end-to-end trainable model that learns the pool of normalizing neurons and the magnitude of their contribution directly from the data. Taking advantage of our model's direct interpretable view of V1 computation, we found that oriented features were normalized preferentially by features with similar orientation preference rather than non-specifically. Our model's accuracy was competitive with state-of-the-art black-box models, suggesting that rectification, DN, and a combination of subunits resulting from DN are sufficient to account for V1 responses to localized stimuli. Thus, our work significantly advances our understanding of V1 function."],["dc.identifier.doi","10.32470/CCN.2019.1211-0"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63367"],["dc.language.iso","en"],["dc.notes.preprint","yes"],["dc.relation.conference","Conference on Cognitive Computational Neuroscience 2019"],["dc.relation.eventend","2019-09-16"],["dc.relation.eventlocation","Berlin, Germany"],["dc.relation.eventstart","2019-09-13"],["dc.relation.iserratumof","yes"],["dc.title","Learning Divisive Normalization in Primary Visual Cortex"],["dc.type","conference_paper"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.contributor.author","Gatys, Leon A."],["dc.contributor.author","Ecker, Alexander S."],["dc.contributor.author","Bethge, Matthias"],["dc.date.accessioned","2020-04-03T10:48:29Z"],["dc.date.available","2020-04-03T10:48:29Z"],["dc.date.issued","2016"],["dc.description.abstract","We introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition. Samples from the model are of high perceptual quality demonstrating the generative power of neural networks trained in a purely discriminative fashion. Within the model, textures are represented by the correlations between feature maps in several layers of the network. We show that across layers the texture representations increasingly capture the statistical properties of natural images while making object information more and more explicit. Extending this framework to texture transfer, we introduce A Neural Algorithm of Artistic Style that can separate and recombine the image content and style of natural images. The algorithm allows us to produce new artistic imagery that combines the content of an arbitrary photograph with the appearance of numerous well-known artworks, thus offering a path towards an algorithmic understanding of how humans create and perceive artistic imagery."],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63583"],["dc.language.iso","en"],["dc.title","Texture Modelling Using Convolutional Neural Networks"],["dc.type","preprint"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.artnumber","326"],["dc.bibliographiccitation.issue","12"],["dc.bibliographiccitation.journal","Journal of Vision"],["dc.bibliographiccitation.volume","16"],["dc.contributor.author","Gatys, Leon"],["dc.contributor.author","Ecker, Alexander"],["dc.contributor.author","Bethge, Matthias"],["dc.date.accessioned","2020-04-01T11:45:49Z"],["dc.date.available","2020-04-01T11:45:49Z"],["dc.date.issued","2016"],["dc.description.abstract","In fine art, especially painting, humans have mastered the skill to create unique visual experiences by composing a complex interplay between the content and style of an image. The algorithmic basis of this process is unknown and there exists no artificial system with similar capabilities. Recently, a class of biologically inspired vision models called Deep Neural Networks have demonstrated near-human performance in complex visual tasks such as object and face recognition. Here we introduce an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality. The system can separate and recombine the content and style of arbitrary images, providing a neural algorithm for the creation of artistic images. In light of recent studies using fMRI and electrophysiology that have shown striking similarities between performance-optimised artificial neural networks and biological vision, our work offers a path towards an algorithmic understanding of how humans create and perceive artistic imagery. The algorithm introduces a novel class of stimuli that could be used to test specific computational hypotheses about the perceptual processing of artistic style."],["dc.identifier.doi","10.1167/16.12.326"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63432"],["dc.relation.issn","1534-7362"],["dc.title","A Neural Algorithm of Artistic Style"],["dc.type","journal_article"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]

[["dc.bibliographiccitation.artnumber","230"],["dc.bibliographiccitation.issue","12"],["dc.bibliographiccitation.journal","Journal of Vision"],["dc.bibliographiccitation.volume","16"],["dc.contributor.author","Wallis, Thomas"],["dc.contributor.author","Ecker, Alexander"],["dc.contributor.author","Gatys, Leon"],["dc.contributor.author","Funke, Christina"],["dc.contributor.author","Wichmann, Felix"],["dc.contributor.author","Bethge, Matthias"],["dc.date.accessioned","2020-04-01T12:00:09Z"],["dc.date.available","2020-04-01T12:00:09Z"],["dc.date.issued","2016"],["dc.description.abstract","An important hypothesis that emerged from crowding research is that the perception of image structure in the periphery is texture-like. We investigate this hypothesis by measuring perceptual properties of a family of naturalistic textures generated using Deep Neural Networks (DNNs), a class of algorithms that can identify objects in images with near-human performance. DNNs function by stacking repeated convolutional operations in a layered feedforward hierarchy. Our group has recently shown how to generate shift-invariant textures that reproduce the statistical structure of natural images increasingly well, by matching the DNN representation at an increasing number of layers. Here, observers discriminated original photographic images from DNN-synthesised images in a spatial oddity paradigm. In this paradigm, low psychophysical performance means that the model is good at matching the appearance of the original scenes. For photographs of natural textures (a subset of the MIT VisTex dataset), discrimination performance decreased as the DNN representations were matched to higher convolutional layers. For photographs of natural scenes (containing inhomogeneous structure), discrimination performance was nearly perfect until the highest layers were matched, whereby performance declined (but never to chance). Performance was only weakly related to retinal eccentricity (from 1.5 to 10 degrees) and strongly depended on individual source images (some images were always hard, others always easy). Surprisingly, performance showed little relationship to size: within a layer-matching condition, images further from the fovea were somewhat harder to discriminate but this result was invariant to a three-fold change in image size (changed via up/down sampling). The DNN stimuli we examine here can match texture appearance but are not yet sufficient to match the peripheral appearance of inhomogeneous scenes. In the future, we can leverage the flexibility of DNN texture synthesis for testing different sets of summary statistics to further refine what information can be discarded without affecting appearance."],["dc.identifier.doi","10.1167/16.12.230"],["dc.identifier.uri","https://resolver.sub.uni-goettingen.de/purl?gro-2/63436"],["dc.language.iso","en"],["dc.relation.issn","1534-7362"],["dc.title","Seeking summary statistics that match peripheral visual appearance using naturalistic textures generated by Deep Neural Networks"],["dc.type","journal_article"],["dc.type.internalPublication","no"],["dspace.entity.type","Publication"]]