Abstract Recently, the development of deep learning has facilitated continuous progress in the field of computer vision. Pixel‐level semantic segmentation serves as a fundamental task in computer vision. It achieves significant results by connecting wider and deeper backbone networks and building fine‐grained segmentation heads. However, applications such as self‐driving cars are more critical to the computational speed of the algorithms. The trade‐off between accuracy and real‐time performance of existing algorithms is still a challenging task. To address this challenge, this article proposes an adaptive multiscale segmentation fusion network to fuse multiscale contextual, which designs an adaptive multiscale segmentation fusion module based on an attention mechanism. Using segmentation fusion instead of feature fusion, the multiscale segmentation results are aggregated to obtain more precise segmentation results. The final results achieved 70.9% mIoU of accuracy in the Cityspace test set, processing images at 61 FPS when the input is 1024 × 2048. In addition, when adjusting the input size to 512 × 1024, the images are processed at 185 FPS.

Abstract A motion retargeting process is necessary as the body size and proportion of the actors are generally different from those of the target characters. However, the original spatial relationship between the multiple characters and the environment is easily broken when using previous motion retargeting methods, which are generally performed for each character independently. Therefore, time‐consuming manual adjustments by animators are usually required to obtain satisfactory results. To address these issues, we present a novel multicharacter motion retargeting method that preserves various types of spatial relationships between characters and environments. We establish a unified deformation‐based framework for the motion retargeting of multiple characters (more than two) or nonhuman characters with complex interactions. Also, an interactive motion editing interface with immediate feedback to the user is provided. We experimentally show that our method achieves a speedup when compared with previous motion retargeting methods.

Abstract While extensive research efforts have been made in semantic image segmentation, the state‐of‐the‐art methods still suffer from blurry boundaries and mismatched objects due to the insufficient multiscale adaptability. In this paper, we propose a two‐branch convolutional neural network (CNN) approach to capture the multiscale context and the boundary information with the two branches, respectively. To capture the multiscale context, we propose to embed self‐attention mechanism to the atrous spatial pyramid pooling network. To capture the boundary information, we propose to fuse the low‐level features in boundary feature extraction for refining the extracted boundaries via a feature fusion layer (FFL). With FFL, our method can improve the segmentation result with clearer boundaries. A new loss function is proposed which contains a segmentation loss and a boundary loss. Experiments show that our method can predict the boundaries of objects more clearly and have better performance for small‐scale objects.

Abstract In the process of simulating and modeling real objects, the phenomenon of objects penetrating each other may occur in the model, which is unrealistic and then the research of collision detection (CD) is generated. As a bottleneck of virtual environment simulation, researchers have conducted in‐depth research on CD, especially the CD of deformable objects. In this paper, we briefly review the related research on CD of deformable objects regarding relevant literature. First, we briefly introduce previous reviews of CD. Second, we review the popular research methods and limitations of CD between deformable objects. Third, we review the popular research methods and limitations of self‐collision detection in deformable objects. Finally, we discuss future directions of development. This review can be used as a reference for the application of CD in all directions.

Abstract This paper presents a unified framework to simulate surface and wave foams efficiently and realistically. The framework is designed first to project thee‐dimensional (3D) water particles from an underlying water solver onto two‐dimensional screen space to reduce the computational complexity of determining where foam particles should be generated. Because foam effects are often created primarily in fast and complicated water flows, we analyze the acceleration and curvature values to identify the areas exhibiting such flow patterns. Foam particles are emitted from the identified areas in 3D space, and each foam particle is advected according to its type, which is classified on the basis of velocity, thereby capturing the essential characteristics of foam wave motions. We improve the realism of the resulting foam by classifying it into two types: surface foam and wave foam. Wave foam is characterized by the sharp wave patterns of torrential flows, and surface foam is characterized by a cloudy foam shape, even in water with reduced motion. Based on these features, we propose a technique to correct the velocity and position of a foam particle. In addition, we propose a kernel technique using the screen space density to reduce redundant foam particles efficiently, resulting in improved overall memory efficiency without loss of visual detail in terms of foam effects. Experiments convincingly demonstrate that the proposed approach is efficient and easy to use while delivering high‐quality results.

Abstract In this paper, we propose a novel Multiview Stereo (MVS) method which can effectively estimate geometry in low‐textured regions. Conventional MVS algorithms predict geometry by performing dense correspondence estimation across multiple views under the constraint of epipolar geometry. As low‐textured regions contain less feature information for reliable matching, estimating geometry for low‐textured regions remains hard work for previous MVS methods. To address this issue, we propose an MVS method based on texture enhancement. By enhancing texture information for each input image via our multiscale bilateral decomposition and reconstruction algorithm, our method can estimate reliable geometry for low‐textured regions that are intractable for previous MVS methods. To densify the final output point cloud, we further propose a novel selective joint bilateral propagation filter, which can effectively propagate reliable geometry estimation to neighboring unpredicted regions. We validate the effectiveness of our method on the ETH3D benchmark. Quantitative and qualitative comparisons demonstrate that our method can significantly improve the quality of reconstruction in low‐textured regions.

Abstract We present a simplified force‐based heterogeneous traffic simulation model to facilitate consistent adjustment of the parameters involved. Different from previous work which requires the adjustment of multiple ad hoc parameters to produce satisfactory results, our approach can achieve similar results by using clear and meaningful parameters to simulate interactions between various kinds of road users. To simulate diverse and realistic motions of road users, we parameterize the coefficients of the force model for better detailed motion control. Our approach is also scalable to new types of road users, and facilitates an object‐oriented implementation with high performance. We validate our framework with extensive experiments.

Automatic animation synthesis has attracted much attention from the community. As most existing methods take a small number of discrete expressions rather than continuous expressions, their integrity and reality of the facial expressions is often compromised. In addition, the easy manipulation with simple inputs and unsupervised processing, although being important to the automatic facial expression animation applications, is relatively less concerned. To address these issues, we propose an unsupervised continuous automatic facial expression animation approach through action units (AU) transfer in the latent space of generative adversarial networks. The expression descriptor which is depicted with AU vector is transferred into the input image without the need of labeled pairs of images and even without their expressions and further network training. We also propose a new approach to quickly generate input image's latent code and cluster the boundaries of different AU attributes with their latent codes. Two latent code operators, vector addition and continuous interpolation, are leveraged for facial expression animation simulating align with the boundaries in the latent space. Experiments have shown that the proposed approach is effective on facial expression translation and animation synthesis.

Abstract Cross‐modal retrieval aims to do flexible retrieval among different modals, whose main issue is how to measure the semantic similarities among multimodal data. Though many existing methods have been proposed to enable cross‐modal retrieval, they rarely consider the preservation of content information among multimodal data. In this paper, we present a three‐stage cross‐modal retrieval method, named MMCA‐CMR. To reduce the discrepancy among multimodal data, we first attempt to embed multimodal data into a common representation space. We then combine the feature vectors with the content information into the semantic‐aware feature vectors. We finally obtain the feature‐aware and content‐aware projections via multimodal semantic autoencoders. With semantic deep autoencoders, MMCA‐CMR promotes a more reliable cross‐modal retrieval by learning feature vectors from different modalities and content information simultaneously. Extensive experiments demonstrate that the proposed method is valid in cross‐modal retrieval, which significantly outperforms state‐of‐the‐art on four widely‐used benchmark datasets.