Sensorimotor adaptation compensates for distortions of 3D shape information

1. Subjects were presented with a cylinder, whose orientation with respect to the vertical and location in space varied from trial to trial. They grasped a similar cylinder in their hands and were instructed to align the grasped cylinder with the target cylinder. In some experiments the task was performed from memory, and subjects attempted to reproduce both location and orientation of the cylinder. In others, they attempted to reproduce only its orientation, either from memory or while the cylinder was in view. 2. Multivariate linear regression analysis was used to determine persistent and variable errors in performance. This analysis related the subjects' performance (reproduced orientation) to target parameters (location and orientation). 3. We have interpreted the experimental results starting from the assumption that there are two parallel neural processes underlying reaching and grasping: one relating proximal arm motion to target location and the other relating distal hand motion to target orientation. 4. Variable errors did not vary with task conditions, even when subjects were asked to dissociate target orientation from target location by matching target orientation with the arm at the side, irrespective of the location of the target. This finding suggests that the neural transformations involving target location and target orientation are performed in parallel. 5. Persistent errors did vary with task condition. The subjects made the largest errors in matching target orientation when the target was in view, but they were asked to match its orientation at a location that differed from that of the target. These errors depended mostly on the elevation of the target and on its slant (inclination relative to the vertical). They were related to the posture of the arm in a manner that suggested that matching orientation is influenced by both extrinsic (spatial) and intrinsic (arm posture) parameters. 6. The fact that persistent errors depend on target location and on arm posture as well as on target orientation implies that the neural transformation from target orientation to hand orientation is not independent of the transformation dealing with target location.

INTRODUCTION Traditionally, the perception of three-dimensional (3D) scenes and objects was assumed to be the result of the reconstruction of 3D surfaces from available depth cues, such as binocular disparity, motion parallax, texture, and shading (Marr, 1982). In this approach, the role of ‰gure-ground organization was kept to a minimum. Figure-ground organization refers to ‰nding objects in the image, diŠerentiating them from the background, and grouping contour elements into two-dimensional (2D) shapes representing the shapes of the 3D objects. According to Marr, solving ‰gureground organization was not necessary. e depth cues could provide the observer with local estimates of surface orientation at many points. ese local estimates were then integrated into the estimates of the visible parts of 3D shapes. Marr called this a 2.5D representation. If the objects in the scene are familiar, their entire 3D shapes can be identi‰ed (recognized) byCONTENTS Introduction 113 Figure-Ground Organization 114 Using Simplicity Principle to Recover 3D Shapes 118 Summary and Discussion 123matching the shapes in the observer's memory to the 2.5D sketch. If they are not familiar, the observer will have to view the object from more than one viewing direction and integrate the visible surfaces into the 3D shape representations.

We used functional magnetic resonance imaging to investigate the human cortical areas involved in processing 3-dimensional (3D) shape from texture (SfT) and shading. The stimuli included monocular images of randomly shaped 3D surfaces and a wide variety of 2-dimensional (2D) controls. The results of both passive and active experiments reveal that the extraction of 3D SfT involves the bilateral caudal inferior temporal gyrus (caudal ITG), lateral occipital sulcus (LOS) and several bilateral sites along the intraparietal sulcus. These areas are largely consistent with those involved in the processing of 3D shape from motion and stereo. The experiments also demonstrate, however, that the analysis of 3D shape from shading is primarily restricted to the caudal ITG areas. Additional results from psychophysical experiments reveal that this difference in neuronal substrate cannot be explained by a difference in strength between the 2 cues. These results underscore the importance of the posterior part of the lateral occipital complex for the extraction of visual 3D shape information from all depth cues, and they suggest strongly that the importance of shading is diminished relative to other cues for the analysis of 3D shape in parietal regions.

3D human shape and pose estimation from monocular images has been an active area of research in computer vision, having a substantial impact on the development of new applications, from activity recognition to creating virtual avatars. Existing deep learning methods for 3D human shape and pose estimation rely on relatively high-resolution input images; however, high-resolution visual content is not always available in several practical scenarios such as video surveillance and sports broadcasting. Low-resolution images in real scenarios can vary in a wide range of sizes, and a model trained in one resolution does not typically degrade gracefully across resolutions. Two common approaches to solve the problem of low-resolution input are applying super-resolution techniques to the input images which may result in visual artifacts, or simply training one model for each resolution, which is impractical in many realistic applications. To address the above issues, this paper proposes a novel algorithm called RSC-Net, which consists of a Resolution-aware network, a Self-supervision loss, and a Contrastive learning scheme. The proposed network is able to learn the 3D body shape and pose across different resolutions with a single model. The self-supervision loss encourages scale-consistency of the output, and the contrastive learning scheme enforces scale-consistency of the deep features. We show that both these new training losses provide robustness when learning 3D shape and pose in a weakly-supervised manner. Extensive experiments demonstrate that the RSC-Net can achieve consistently better results than the state-of-the-art methods for challenging low-resolution images.

This work addresses the problem of 3D human pose and shape estimation from a sequence of point clouds. Existing sequential 3D human shape estimation methods mainly focus on the template model fitting from a sequence of depth images or the parametric model regression from a sequence of RGB images. In this paper, we propose a novel sequential 3D human pose and shape estimation framework from a sequence of point clouds. Specifically, the proposed framework can regress 3D coordinates of mesh vertices at different resolutions from the latent features of point clouds. Based on the estimated 3D coordinates and features at the low resolution, we develop a spatial-temporal mesh attention convolution (MAC) to predict the 3D coordinates of mesh vertices at the high resolution. By assigning specific attentional weights to different neighboring points in the spatial and temporal domains, our spatial-temporal MAC can capture structured spatial and temporal features of point clouds. We further generalize our framework to the real data of human bodies with a weakly supervised fine-tuning method. The experimental results on SURREAL, Human3.6M, DFAUST and the real detailed data demonstrate that the proposed approach can accurately recover the 3D body model sequence from a sequence of point clouds.

The estimation of 3D body pose and shape has always been a challenging problem due to various reasons, such as the ambiguity in 2D images and complex articulated structure of the human body. In order to solve the ill-conditioned problems, in this paper, we bring up an end-to-end method to estimate 3D human shape and pose from multi-view RGB images. In the proposed framework, we first implement a CNN embedded with attention module to extract the image feature and design the view-pooling layer to combine the features from multiple views. Then we adopt a regression network with a novel geometric constraint of body limbs to estimate 3D human pose and shape. Additionally, during the training process, we employ the idea of adversarial learning in our model to help regress accurate pose and shape parameters. Extensive experiments are conducted on Human3.6M and MPI-INF-3DHP datasets, and our method achieves competitive results in the 3D pose and shape estimation task.

Explicit and implicit depth-cue integration: Evidence of systematic biases with real objects

The effects of augmented sensory feedback precision on the acquisition and retention of a simulated chiropractic task

Neurophysiological and neuroimaging work suggests that the cerebellum is critically involved in sensorimotor adaptation. Changes in cerebellar function alter behaviour when compensating for sensorimotor perturbations, as shown by non-invasive stimulation of the cerebellum and studies involving patients with cerebellar degeneration. It is known, however, that behavioural responses to sensorimotor perturbations reflect both explicit processes (such as volitional aiming to one side of a target to counteract a rotation of visual feedback) and implicit, error-driven updating of sensorimotor maps. The contribution of the cerebellum to these explicit and implicit processes remains unclear. Here, we examined the role of the cerebellum in sensorimotor adaptation to a 30° rotation of visual feedback of hand position during target-reaching, when the capacity to use explicit processes was manipulated by controlling movement preparation times. Explicit re-aiming was suppressed in one condition by requiring subjects to initiate their movements within 300ms of target presentation, and permitted in another condition by requiring subjects to wait approximately 1050ms after target presentation before movement initiation. Similar to previous work, applying anodal transcranial direct current stimulation (tDCS; 1.5mA) to the right cerebellum during adaptation resulted in faster compensation for errors imposed by the rotation. After exposure to the rotation, we evaluated implicit remapping in no-feedback trials after providing participants with explicit knowledge that the rotation had been removed. Crucially, movements were more adapted in these no-feedback trials following cerebellar anodal tDCS than after sham stimulation in both long and short preparation groups. Thus, cerebellar anodal tDCS increased implicit remapping during sensorimotor adaptation, irrespective of preparation time constraints. The results are consistent with the possibility that the cerebellum contributes to the formation of new visuomotor maps that correct perturbations in sensory feedback, even when explicit processes are suppressed during sensorimotor adaptation.

A wide range of applied imaging problems are concerned with the determination of the three dimensional structure of anomalous areas in a larger field of regard. In the context of medical imaging, there is great interest in the characterization of cancerous tumors using non-ionizing modalities such as diffuse optical tomography or quantitative ultrasonic imaging. In this work, we introduce a new, flexible approach to the modeling and estimation of 3D shapes. A complex three dimensional volume is defined by a set of 2D shape "primitives" representing the cross section of the object in essentially arbitrary planes. Each primitive is itself a 2D shape (specifically an ellipse for this paper) the structure of which is easily defined by a low dimensional vector of parameters (center location, axis lengths, and orientation angles). Given a set of primitives, we devise an interpolation scheme that correlates the structure of the individual primitives from one to the next. A nonlinear estimation algorithm is described for determining the parameters of our elliptic primitives i.e., the location of the centers in 3D, the lengths of their axes, and their orientation in space. Simulated results show the effectiveness of this method.

Three-dimensional human face modeling is a very popular topic with many applications,such as facial animation, face recognition and model-based facial video communication. Therefore, how to model the facial geometry, texture intensity and illumination variation is important in computer vision and graphics. Previous works on 3D head modeling from a single face image utilized prior information on 3D head models. However, it is difficult to accurately reconstruct the 3D face model from a single face image with expression since the facial expression induces 3D face model deformation in a complex manner. The main challenge is the coupling of the neutral 3D face model and the 3D deformation due to expression, thus making the 3D model estimation from a single face image with expression very challenging. On the other hand, the illumination condition also makes the problem more difficult. In this thesis,we focus on developing a 3D face model reconstruction system including surface registration and training of 3D face models with expressional deformations as well as the estimation of the 3D neutral shape and the 3D expressional deformation from a single face image. The proposed reconstruction algorithm integrates the linear and non-linear subspace representations for a prior 3D neutral morphable model and the probabilistic manifold-based 3D expressional deformation. We incorporate the face geometry, expression deformation, texture and illumination information into the problem so that it is well constrained. The reconstructed 3D face models can also be further extended and applied to many real-world applications.

The direct extension of 2D image learning to three-dimensional space is 3D point cloud learning. Recently, point cloud learning has shown significant results in 3D shape estimation and semantic segmentation. Despite these advancements, fundamental problems in point cloud learning still pose significant challenges. These problems include representation learning, shape generation, shape segmentation, and shape matching. In this paper, we propose a cognitive self-attention based learning approach to aggregate global representation of 3D shapes from point cloud data. We also integrate 3D point data with a binary tree structure to build a point cloud generator. We further design a novel Generative Adversarial Network (GAN) architecture to generate point clouds resembling the ground truth that could be used for unsupervised learning of 3D shapes. Relying on a self-attention mechanism, our framework, called SAPCGAN, aggregates the global graph features to correct the structural information of 3D shapes in the latent space. Finally, we compare the performance of two gradient penalty methods used in stabilizing the training of our GAN system. We show that our framework has high training efficiency and can provide state-of-the-art results in 3D point cloud generation. The performance of our is demonstrated with both quantitative and qualitative experimental evaluations. Furthermore, the generated 3D point clouds can be segmented into their natural semantic parts, such as, for example the four legs of a chair, the wings of an air plane, or the four wheels of a car.

Extracting 3D structure from disparity

3D Nose shape net for human gender and ethnicity classification

Several existing 3D systems for dental applications rely on obtaining an intermediate solid model of the jaw (cast or teeth imprints) from which the 3D information can be captured. In this paper, we propose a model-based shape-from-shading (SFS) approach which allows for the construction of plausible human jaw models in vivo, without ionizing radiation, using fewer sample points in order to reduce the cost and intrusiveness of acquiring models of patients teeth/jaws over time. The inherent relation between the photometric information and the underlying 3D shape is formulated as a statistical model where the effect of illumination is modeled using Spherical Harmonics (SH) and the partial least square (PLS) approach is deployed to carry out the estimation of dense 3D shapes. Moreover, shape and texture alignment is accomplished using a proposed definition of anatomical jaw landmarks which can be automatically detected. Vis-a-vis dental applications, the results demonstrate a significant increase in accuracy in favor of the proposed approach. In particular, our approach is able to recover geometrical details of tooth occlusal surface as well as mouth floor and ceiling as compared to SFS-based approaches.