Raw Instinct: Trust Your Classifiers and Skip the Conversion

Super-resolution is a fundamental problem in computer vision which aims to overcome the spatial limitation of camera sensors. While significant progress has been made in single image super-resolution, most algorithms only perform well on synthetic data, which limits their applications in real scenarios. In this paper, we study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We focus on two issues of existing super-resolution algorithms: lack of realistic training data and insufficient utilization of visual information obtained from cameras. To address the first issue, we propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. For the second issue, we develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images. In addition, we propose a dense channel-attention block for better image restoration as well as a learning-based guided filter network for effective color correction. Our model is able to generalize to different cameras without deliberately training on images from specific camera types. Extensive experiments demonstrate that the proposed algorithm can recover fine details and clear structures, and achieve high-quality results for single image super-resolution in real scenes.

Local feature descriptors and detectors have been widely used in computer vision in the last years for solving object detection and recognition tasks. Research efforts have been focused on reducing the complexity of these descriptors and improving their accuracy. However, these descriptors have not been tested until now on raw image data. This paper presents a study on the use of two of the most known and used feature descriptors, SURF and SIFT, directly on raw CFA images acquired by a digital camera. We are interested in understanding if the number and quality of the keypoints obtained from a raw image are comparable to the ones obtained in the grayscale images, which are normally used by these transforms. The results that we present show that the number and positions of the keypoints obtained from grayscale images are similar to the ones obtained from CFA images and furthermore to the ones obtained from grayscale images that resulted directly from the interpolation of a CFA image.

713 A New EUS Imaging Diagnosis for Pancreatic Cancer: A Pilot Study of Differential Diagnosis Between Chronic Pancreatitis and Pancreatic Cancer Based on the Quantitative Analysis With New Technology Using Raw Image

Gender classification is one of the most challenging problems in computer vision. Facial gender detection of neonates and children is also known as a highly demanding issue for human observers. This study proposes a novel gender classification method using frontal facial images of people. The proposed approach employs principal component analysis (PCA) and fuzzy clustering technique, respectively, for feature extraction and classification steps. In other words, PCA is applied to extract the most appropriate features from images as well as reducing the dimensionality of data. The extracted features are then used to assign the new images to appropriate classes – male or female – based on fuzzy clustering. The computational time and accuracy of the proposed method are examined together and the prominence of the proposed approach compared to most of the other well‐known competing methods is proved, especially for younger faces. Experimental results indicate the considerable classification accuracies which have been acquired for FG‐Net, Stanford and FERET databases. Meanwhile, since the proposed algorithm is relatively straightforward, its computational time is reasonable and often less than the other state‐of‐the‐art gender classification methods.

To overcome the data movement bottleneck, near-sensor and in-sensor computing are becoming more and more popular. However, in the existing near-/in-sensor computing architectures for vision tasks, the effect of the image signal processing (ISP) pipeline, which is of great importance to the final vision performance [1], is always ignored. In this work, we propose a synthesized RAW image-based end-to-end computer vision paradigm, taking the effect of ISP pipeline into account. In the proposed approach, a generative adversarial network (GAN)-based tool is used to convert the fully processed color images to their corresponding RAW Bayer versions, generating the training data for end-to-end vision models. In the inference stage, RAW images from the sensor are directly fed to the end-to-end model, bypassing the entire ISP pipeline. Experimental results show that by training/tuning the CNN models using synthesized RAW images, it is possible to design an end-to-end (from RAW image to vision task) vision system that directly consumes RAW image data from the sensor with negligible vision performance degradation. By skipping the ISP pipeline, an image sensor can be directly integrated with the back-end vision processor without a complex image processor in the middle, making near-/in-sensor computing a practical approach.

Attribute hierarchy based multi-task learning for fine-grained image classification

Novel view synthesis is a classic problem in computer vision. It refers to the generation of previously unseen views of a scene from a set of sparse input images taken from different viewpoints. One example of novel view synthesis is the interpolation of views in between the two images of a stereo camera. Another classic problem in computer vision is video frame interpolation, which is important for video processing. It refers to the generation of video frames in between existing ones and is commonly used to increase the frame rate of a video or to match the frame rate to the refresh rate of the monitor that the video is being displayed on. Interestingly, off-the-shelf video frame interpolation can directly be employed to successfully perform view interpolation to address the aforementioned stereo view interpolation problem. Video frame interpolation can be seen as temporal novel view synthesis. However, this perspective is usually not considered and novel view synthesis generally concerns generating unseen views in space rather than time. For this reason, the set of sparse input images that is used for spatial novel view synthesis is commonly either captured at the same time, or it is assumed that the scene is static. This paradigm limits the applicability of novel view synthesis in real-world scenarios though. This thesis addresses three applications of novel view synthesis and provides practical solutions that do not require difficult-to-acquire multi-view imagery: video frame interpolation which performs temporal video-to-video synthesis, synthesizing the 3D Ken Burns effect from a single image which performs spatial image-to-video synthesis, synthesizing video action shots which performs spatiotemporal video-to-video and video-to-image synthesis. These applications not only explore different dimensions of time and space, they also perform novel view synthesis on everyday image and video footage. This is in stark contrast to the large body of existing work which focuses on spatial novel view synthesis while requiring multiple input views that were either captured at the same time or under the assumption of a static scene.

Acquiring qualified samples for RANSAC using geometrical constraints

In recent years polynomial solvers based on algebraic geometry techniques, and specifically the action matrix method, have become popular for solving minimal problems in computer vision. In this paper we develop a new method for reducing the computational time and improving numerical stability of algorithms using this method. To achieve this, we propose and prove a set of algebraic conditions which allow us to reduce the size of the elimination template (polynomial coefficient matrix), which leads to faster LU or QR decomposition. Our technique is generic and has potential to improve performance of many solvers that use the action matrix method. We demonstrate the approach on specific examples, including an image stitching algorithm where computation time is halved and single precision arithmetic can be used.

Recurrent neural networks are powerful tools for handling incomplete data problems in computer vision, thanks to their significant generative capabilities. However, the computational demand for these algorithms is too high to work in real time, without specialized hardware or software solutions. In this paper, we propose a framework for augmenting recurrent processing capabilities into a feedforward network without sacrificing much from computational efficiency. We assume a mixture model and generate samples of the last hidden layer according to the class decisions of the output layer, modify the hidden layer activity using the samples, and propagate to lower layers. For visual occlusion problem, the iterative procedure emulates feedforward-feedback loop, filling-in the missing hidden layer activity with meaningful representations. The proposed algorithm is tested on a widely used dataset and shown to achieve 2× improvement in classification accuracy for occluded objects. When compared to Restricted Boltzmann Machines, our algorithm shows superior performance for occluded object classification.

In the areas of computer vision and computer graphics, analyzing information about three-dimensional objects from two-dimensional images is essential. Shape from shading is a very fundamental problem in computer vision, which uses features and patterns in images to infer the shape of the object in our viewing direction. The influence of solving this problem is significant. For example, a typical application of shape from shading is in astronomy, in which we can obtain landscape features of a planet by analyzing the information from the image we get.

The simultaneous recovery of three-dimensional (3D) structure from motion (SfM) for the sequences of images, is one of the more difficult problems in computer vision. Classical approaches to the problem rely on using algebraic techniques to solve for these unknowns given two or more image. Motion analysis and 3D shape estimation based on the estimated motion is an important problem in computer vision. The correspondence problem is an important tool in SfM where in this paper a general 3D motion based on a simple rotation, tilt, roll and translation is proposed, and then is used for 3D shape estimation. The current work expands the 2D motion estimation (shift, rotation) to accommodate general 3D motion (shift, rotation, tilt and roll). The proposed work in this paper of motion estimation of moving object is based on Fourier Descriptor Transformation (FDT) analysis before and after motion. The FDT is also used to resolve the correspondence problem in a sequence of images. We test our method on several large-scale photo collections, show the efficacy of the introduced approach to improve reconstruction accuracy.

Which parts of an image evoke emotions in an observer? To answer this question, we introduce a novel problem in computer vision — predicting an Emotion Stimuli Map (ESM), which describes pixel-wise contribution to evoked emotions. Building a new image database, EmotionROI, as a benchmark for predicting the ESM, we find that the regions selected by saliency and objectness detection do not correctly predict the image regions which evoke emotion. Although objects represent important regions for evoking emotion, parts of the background are also important. Based on this fact, we propose using fully convolutional networks for predicting the ESM. Both qualitative and quantitative experimental results confirm that our method can predict the regions which evoke emotion better than both saliency and objectness detection.

One of the fundamental problems of computer vision is to extract 3D shape and motion from images. This can be achieved when a scene or object is observed from different viewpoints or over a period of time. There is a wide range of applications, ranging from digitizing cultural heritage to vision-based autonomous robot navigation. This talk will present several approaches to solve this problem. First, we’ll discuss techniques for 3D shape recovery for static objects and scenes. One particular case is the 3D mapping and localization in large environments from images, e.g. urban 3D reconstruction from vehicleborne cameras or localization from cell-phone images. Next, we’ll shift our focus to modeling dynamic scenes, e.g. people who are moving around. In addition to explicitly 3D modeling an event, we'll consider the possibility to perform video-based rendering from casually captured videos. Speaker Bio-Sketch: Marc Pollefeys is a full professor in the Dept. of Computer Science of ETH Zurich since 2007 where he is the head of the Institute for Visual Computing and leads the Computer Vision and Geometry lab. He currently also remains associated with the Dept. of Computer Science of the University of North Carolina at Chapel Hill where he started as an assistant professor in 2002 and became an associate professor in 2005. Before this he was a postdoctoral researcher at the Katholieke Universiteit Leuven in Belgium, where he also received his M.S. and Ph.D. degrees in 1994 and 1999, respectively. His main area of research is computer vision. One of his main research goals is to develop flexible approaches to capture visual representations of real world objects, scenes and events. Dr. Pollefeys has received several prizes for his research, including a Marr prize, an NSF CAREER award, a Packard Fellowship and a European Research Council Starting Grant. He is the author or co-author of more than 130 peer-reviewed publications. He is the General Chair for the European Conference on Computer Vision 2014 (ECCV), was a Program Co-Chair for the IEEE Conference on Computer Vision and Pattern Recognition 2009 (CVPR), was general/program co-chair of the Third Symposium on 3D Data Processing, Visualization and Transmission and has organized workshops and courses at major vision and graphics conferences and has served on the program committees of many conferences. Prof. Pollefeys is/was on the Editorial Board of the IEEE Transactions on Pattern Analysis and Machine Intelligence and the International Journal of Computer Vision as well as several other journals in computer vision, graphics and robotics.

No object is mysterious. The mystery is your eye. – Elizabeth Bowen Introduction There are two fundamentally different philosophies concerning understanding the brain. (1) Understand the brain first. If we can understand how the brain works, we can build smart machines. (2) Using any technique we can think of, make a smart machine. If we can accomplish that, it will give us some hints about how the brain works. This book is all about the second approach, although it draws from current understanding of biological computing. In this chapter, however, we define a few terms, introduce the greater localglobal problem, and then give a very brief introduction to the function of the mammalian brain. • (Section 1.2) From signal and systems perspective, we describe the differences between Computer Vision and some other closely related fields of studies, including, e.g., image processing and pattern recognition. • (Section 1.3) Since almost all problems in Computer Vision involve the issue of localness versus globalness, we briefly explain the “local-global” problem and the “consistency” principle used to solve this problem. • (Section 1.4) Computer Vision is deep-rooted in biological vision. Therefore, in this section, we discuss the biological motivation of Computer Vision and some amazing discoveries from the study of the human visual system. Some Definitions Computer Vision is the process whereby a machine, usually a digital computer, automatically processes an image and reports “what is in the image.” That is, it recognizes the content of the image. For example, the content may be a machined part, and the objective may be not only to locate the part but to inspect it as well. Students tend to get confused by other terms that often appear in the literature, such as Image Processing , Machine Vision , Image Understanding , and Pattern Recognition . We can divide the entire process of Image Processing into Low-Level Image Processing and High-Level Image Processing . If we interpret these processes from signal and systems perspective, it is more clear to describe their difference and similarity from the format of input/output of the system. When a Low-Level Image Processing system processes an input image, the output is still an image, but a somewhat different image. For example, it may be an image with noise removed, an image that does not take as much storage space as the input image, an image that is sharper than the input image, etc.