Training and inference of convolutional neural networks (CNNs) on truncated signed distance fields (TSDFs) is a challenging task. Large parts of the scene are usually empty, which makes dense implementations inefficient in terms of memory consumption and compute throughput. However, due to the truncation distance, non-zero values are grouped around the surface creating small dense blocks inside the large empty space. We show that this structure can be exploited by storing the TSDF in a block sparse tensor and then decomposing it into rectilinear super blocks. A super block is a dense 3d cuboid of variable size and can be processed by conventional CNNs. We analyze the rectilinear decomposition and present a formulation for computing the bandwidth-optimal solution given a specific network architecture. However, this solution is NP-complete, therefore we also a present a heuristic approach for fast training and inference tasks. We verify the effectiveness of SuBloNet and report a speedup of 4x towards dense implementations and 1.7x towards state-of-the-art sparse implementations. Using the super block architecture, we show that recurrent volumetric fusion is now possible on large scale scenes. Such a systems is able to reconstruct high-quality surfaces from few noisy depth images.

Detailed shadows in large-scale environments are challenging. Our approach enables efficient detailed shadow computations for static environments at a low memory cost. It builds upon compressed precomputed multiresolution hierarchies but uses a directed acyclic graph to encode its tree structure. Further, depth values are compressed and stored separately and we use a bit-plane encoding for the lower tree levels entries in order to further reduce memory requirements and increase locality. We achieve between 20% to 50% improved compression rates, while retaining high performance.

ICAT-EGVE 2020 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments

In this paper, we present a monocular 3D fluid volume reconstruction technique that can alleviate challenging parameter tuning while vividly reproducing the inflow and outflow of the video scene. To reconstruct the geometric appearance and 3D motion of the fluid in the video, we propose a multilayer external force guiding model that formulates the effect of target particles on fluid particles. This multilayer model makes the whole 3D fluid volume subject to the shape and motion of the water captured by the input video, so we can avoid tedious and laborious parameter tuning and easily balance the smoothness of the fluid volume and the details of the water surface. Besides, for the inflow and outflow of the 3D fluid volume, we construct a generation and extinction model to add or delete fluid particles according to the 3D velocity field of target particles calculated by a hybrid model of coupling SfS with optical flow. Experiments show that our method compares favorably to the state-of-the-art in terms of reconstruction quality, and is more general to the real-captured fluid. Furthermore, the reconstructed 3D fluid volume can be effectively applied to any desired new scenario.

We introduce a Virtual Reality (VR) one-on-one tutoring system to support anatomy education. A student uses a fully immersive VR headset to explore the anatomy of the base of the human skull. A teacher guides the student by using the semi-immersive zSpace. Both systems are connected via network and each action is synchronized between both systems. The teacher is provided with various features to direct the student through the immersive learning experience. She can influence the student's navigation or provide annotations on the fly and, hereby, improve the students learning experience. The system is implemented using the \textit{Unity} game engine. A qualitative user study demonstrates that the one-on-one tutoring approach is feasible and sets a solid base for future research in the area of shared virtual environments for anatomy education.

Trajectories of moving objects are of interest in multiple research fields ranging from geographic information science to behavioral science. Movement patterns of the studied object are often analyzed. Therefore, similar trajectories are retrieved which introduces the need for a similarity measure of trajectories. Similarity measures taken the shape of the trajectory into account are widely researched. Though, there are more attributes that can be relevant to distinguish different movements. One of them is the object orientation along the trajectory. The orientation is of interest in research fields where it influences the movement behavior like the impact of external forces in particles simulations. Trajectory retrieval taking particle orientation into account is still an open research question. Therefore, this work presents a similarity measure for trajectory retrieval considering the complex interaction of linear and rotational movement of particles. Furthermore, the similarity measure applies partial matching allowing for exploration of trajectory parts such as events that may occur along a trajectory tracked over a long time. The proposed algorithm is incorporated into an application for exploratory trajectory visualization.

Framestore and WeightShift created a full-body physics simulation to deliver art-directed shots in highly dynamic scenarios in several days instead of several weeks. Our technique combines ragdoll simulation and keyframe animation in a new control scheme that is integrated in a standard Maya rig to produce a hybrid simulation workflow. The new workflow was used in over 50 completed shots, and is in production on numerous current projects. In this session we share our experience with full-body simulation, and discuss the numerous advantages and challenges of the technique.

In this paper we present a context-aware immersive teleoperation interface to assist operators during navigation tasks. This new interface strategy aims to address the problems associated with mental overload, often experienced by operators of teleoperated devices. Our approach simplifies the high complexity of information displayed in control rooms. Our approach includes a context-based human-robot interaction framework that detects relevant information and automatically adapts the displayed interface in virtual windshield. Results showed that the proposed approach enhances user immersion while maximizes task performances and minimizes the operator physical and cognitive workload.

Challenges in set visualization include representing overlaps among sets, changes in their membership, and details of constituent elements. We present a visualization technique that addresses these challenges. The approach uses set intersection graphs that explicitly visualize each set intersection as a rectangular node and elements as circles inside them. We represent the graph as a layered node-link diagram using colors to indicate the sets. The layers reflect different levels of intersections, from the base sets in the lowest layer to potentially the intersection of all sets in the highest layer. We provide different perspectives to show temporal changes in set membership. Graphs for individual, two, and all timesteps are visualized in static, diff, and aggregated views. Together with linked views and filters, the technique supports the detailed exploration of dynamic set data. We demonstrate the effectiveness of the proposed approach by discussing two application examples. The supplemental material contains the prototype itself and a video showing the proposed interactions.

Human Bio-Behavioral Rhythms (HBRs) such as sleep-wake cycles and their regularity have important health ramifications. Smartphones can sense HBRs by gathering and analyzing data from built-in sensors, which provide behavioral clues. The multichannel nature (multiple sensor streams) of such data makes it challenging to pin-point the causes of disruptions in HBRs. Prior work has utilized machine learning for HBR classification but has not facilitated deeper understanding or reasoning about the potential disruption causes. In this paper, we propose ARGUS, an interactive visual analytics framework to discover and understand HBR disruptions and causes. The foundation of ARGUS is a Rhythm Deviation Score (RDS) that extracts a user's underlying 24-hour rhythm from their smartphone sensor data and quantifies its irregularity. ARGUS then visualizes the RDS using a glyph to easily recognize disruptions in HBRs, along with multiple linked panes that overlay sensor information and user-provided or smartphone-inferred ground truth as supporting context. This framework visually captures a comprehensive picture of HBRs and their disruptions. ARGUS was designed by an expert lead goal-and-task analysis. To demonstrate its generalizability, two different smartphone-sensed datasets were visualized using ARGUS in conjunction with expert feedback.