Upper Body Pose Estimation from Stereo and Hand-Face Tracking

In this paper, we present $O^{3}$ REAL, a privacy-preserving distributed middleware for real-time collaborative editing of documents. $O^{3}$ REAL introduces a novel approach for building peer-to-peer real-time collaborative applications, using a reliable broadcast channel mechanism for network communication, but at the same time provides for persistent storage management of collaborative documents using the filesystem interface of a POSIX compliant filesystem. This approach enables real-time, completely decentralized collaboration among users, without the need for a third party to intervene, and significantly simplifies the creation of peer-to-peer collaborative applications. We demonstrate that $O^{3}$ REAL scales well for real-time collaboration use-cases. For example, with 33 users simultaneously collaborating on a document in real time over a WAN with a 50 ms link delay, the average perceived latency is approximately 54 ms, which is very close to the optimal baseline. In comparison, Etherpad exhibits nearly twice the perceived latency.

The I'Net age (the combination of the Internet, intranets, and extranets) supports real-time or synchronous interaction. Synchronous interactions are those in which a response typically occurs in less than 10 seconds. Relying on a standard Java-enabled 4.x browser, many organizations are collaborating in real time on decisions, proposals, customer service, e-commerce, marketing documents, and more. As defined by our consultancy, Collaborative Strategies, the functionality of a real-time collaboration (RTC) tool resides between the phone and email. RTC tools help communicate graphical and text-based information in a synchronous fashion. To date, users have typically run interactive sessions in conjunction with a conference call, since voice over IP can have unreliable quality, especially over the Internet.

We propose a complete image-based process that facilitates recovery of both gross-scale geometry and local surface structure to create highly detailed 3D models of building façades from photographs. We approximate both albedo and sufficient local geometric structure to compute complex self-shadowing effects, and fuse this with a gross-scale 3D model. Our approach yields a perceptually high-quality model, imparting the illusion of measured reflectance. The requirements of our approach are that image capture must be performed under diffuse lighting and surfaces in the images must be predominantly Lambertian. Exemplars of materials are obtained through surface depth hallucination, and our novel method matches these with multi-view image sequences that are also used to automatically recover 3D geometry. In this paper we illustrate the effictiveness of our approach through visually realistic reconstructions of historic buildings in two test cases, together with a table showing the breakdown of effort required to reconstruct each.

View-invariant representation of hand postures in the human lateral occipitotemporal cortex

It is envisioned that the real time collaboration and dynamic re-planning for target engagement would take place autonomously between unmanned systems (UMS) based on pre-mission planning profiles generated prior to mission initiation, with final decision on target engagement being left to the human operator. This leads directly to the need for the systems to be able to operate autonomously for extended periods, and also to be able to collaboratively engage hostile targets within specified rules of engagement. All assets are able to share a COP (common operating picture) of the battlespace and communicate sensor and target data in real time thus allowing for human or UMS target acquisition and location, transmission of the target to a weapon-target pairing system, and automated assignment of the target to a human or UMS asset as best suited. The collaborative engagement capability is developed as a distributed hardware/software processing component or components capable of insertion into multiple software architectures, and capable of use in multiple operating systems, to include real time embedded operating systems interfaced with on-board sensor, controller subsystems. This paper considers an automated target tracking, pointing and trajectory estimation system that includes components related to target acquisition, target coordinates determination, targeting information visualization and evaluation/simulation testing. Collaboration Engagement involves the target being acquired first by the Coremicro 4D GIS targeting system. Then, the real-time projectile trajectory is determined by the automated stabilization and pointing control system.

SIBI is the standardized sign language system offi-cially used in Indonesia. The application of SIBI is often found to be a hindrance because there are too many gestures that must be memorized. A mobile-based application is needed as gesture-to-text translator. From Rakun et al., Skin Color Segmentation was used as a method to segment hand and facial features using greenscreen background as dataset (3.367% of WER and 80.180% of SAcc). When this application is used, the gesture video is recorded on complex background but performed poorly (135.180% of WER and 0% of SAcc score). The computational time using Skin Color Segmentation is 0.013 s per frame. OpenPose was used to locate hand and facial position. OpenPose can give better performance (6.312% of WER and 69.293% of SAcc score) compared to Skin Color Segmentation but cannot be implemented on mobile application. The computational time using OpenPose is 0.410 s per frame. The focus of this study is to find a model that can locate hand and facial position on complex background and also can be implemented on mobile application. The model we use is RetinaNet. RetinaNet is proven to locate hand and facial position much better (4,100% of WER and 78,990 % of SAcc score) than Skin Color Segmentation and OpenPose. The computational time using RetinaNet is 0.038 s per frame.

With the rapid development of computer communication and web technology, especially the widely application of wireless sensor and embedded technology, Wireless Sensor Networks (WSNs) have been paid great attention in industry field and our daily life. WSNs based on ARM7TDMI-S CPU and ZigBee device has merit of rapid and simple in networking, node movable in application, abundant information in acquisition, real time collaboration in communication, low power consumption in continuous operation, so it is very suitable for targeting miner's position underground. In this paper, networking scheme of the WSN is presented, and distributed information processing, communication technology and real time collaboration are focused on. Miner's information in mobile is obtained accurately by sensor. After that position information is transmitted reliably to monitoring center. Test result in continuous operation shows there is no information loss or undetected. Therefore this scheme is stable and effective, which will play a positive role for security in coal mine by right of characteristics of Zigbee WSNs in my opinion.

The foot-to-ground contact model plays an important role in the simulation of highly dynamic motions, such as turns and kicks. In this paper, we propose a method for solving dynamically cumbersome slipping contact problems, which are frequently observed in highly dynamic motions. We employ and modify a combination of two different types of cones representing the inequality constraints of a contact model: the friction cone and the velocity cone. The friction cone makes character animation physically plausible while the velocity cone allows a character to perform a sharp turn without foot-to-ground penetration. Our system effectively simulates human behavior using an inverted pendulum on a cart (IPC) model and motion capture data. In the pre-processing step, we analyze motion capture data to extract meaningful information for the IPC model. At run-time, our system produces a physically simulated character by tracking the desired motion that is predicted by the IPC model. We formulate human motion control as a quadratic programming satisfying the proposed foot-to-ground contact constraints. Our examples show that the proposed system can produce physically plausible character animation without noticeable foot-to-ground contact artifacts.

This paper presents an immersive telepresence system for entertainment and meetings (ITEM). The system aims to provide a radically new video communication experience by seamlessly merging participants into the same virtual space to allow a natural interaction among them and shared collaborative contents. With the goal to make a scalable, flexible system for various business solutions as well as easily accessible by massive consumers, we address the challenges in the whole pipeline of media processing, communication, and displaying in our design and realization of such a system. Particularly, in this paper we focus on the system aspects that maximize the end-user experience, optimize the system and network resources, and enable various teleimmersive (TI) application scenarios. In addition, we also present a few key technologies, i.e., fast object-based video coding for real world data and spatialized audio capture and 3-D sound localization for group teleconferencing. Our effort is to investigate and optimize the key system components and provide an efficient end-to-end optimization and integration by considering user needs and preferences. Extensive experiments show the developed system runs reliably and comfortably in real time with a minimal setup requirement (e.g., a webcam or a color plus depth camera, an optional microphone array, a laptop/desktop connected to the Internet) for TI communication. With such a really minimal deployment requirement, we present a variety of interesting applications and user experiences created by ITEM.

It is important for beginners to imitate poses of experts in various sports; especially in sport climbing, performance depends greatly on the pose that should be taken for given holds. However, it is difficult for beginners to learn the proper poses for all patterns from experts since climbing holds are completely different for each course. Therefore, we propose a system that predict a pose of experts from the positions of the hands and feet of the climber--the positions of holds used by the climber--using a neural network. In other words, our system simulates what pose experts take for the holds the climber is now using. The positions of hands and feet are calculated from a image of the climber captured from behind. To allow users to check what pose is ideal in real time during practice, we have adopted a simple and lightweight network structure with little computational delay. We asked experts to compare the poses predicted by our system with the poses of beginners, and we confirmed that the poses predicted by our system were in most cases better than or as good as those of beginners.

Despite the current availability in resource-rich regions of advanced technologies in scanning and 3-D imaging in current ophthalmology practice, world-wide screening tests for early detection and progression of glaucoma still consist of a variety of simple tools, including fundus image-based parameters such as CDR (cup to disc diameter ratio) and CAR (cup to disc area ratio), especially in resource -poor regions. Reliable automated computation of the relevant parameters from fundus image sequences requires robust non-rigid registration and segmentation techniques. Recent research work demonstrated that proper non-rigid registration of multi-view monocular fundus image sequences could result in acceptable segmentation of cup boundaries for automated computation of CAR and CDR. This research work introduces a composite diffeomorphic demons registration algorithm for segmentation of cup boundaries from a sequence of monocular images and compares the resulting CAR and CDR values with those computed manually by experts and from 3-D visualization of stereo pairs. Our preliminary results show that the automated computation of CDR and CAR from composite diffeomorphic segmentation of monocular image sequences yield values comparable with those from the other two techniques and thus may provide global healthcare with a cost-effective yet accurate tool for management of glaucoma in its early stage.

The Internet and new easy-to-use software have transformed the educational paradigm from the traditional lecture and grand rounds in person to lecture attendance and learning online. Web-based education is more than just another delivery vehicle. It provides a new opportunity to provide ­distance learning, collaboration in either real time or asynchronously, incorporate rich multimedia and other visuals to illustrate teaching points, and assessment of the learner’s progress. Programming skills and computer savvy are no longer necessary but key attention to instructional design and content is critical. Many Web-based educational creations are now evaluated and promoted as scholarly activity.

There are many applications for which it is necessary to illustrate motion in a static image using visual cues which do not represent a physical entity in the scene, yet are widely understood to convey motion. For example, consider the task of illustrating the desired movements for exercising, dancing, or a given sport technique. Traditional artists have developed techniques to specify desired movements precisely (technical illustrators) and suggest motion (cartoonists) in an image.In this paper, we present an interactive system to synthesize a 2D image of an animated character by generating artist-inspired motion cues derived from 3D skeletal motion capture data. The primary cues include directed arrows, noise waves, and stroboscopic motion. First, the user decomposes the animation into short sequences containing individual motions which can be represented by visual cues. The system then allows the user to determine a suitable viewpoint for illustrating the movement, to select the proper level in the joint hierarchy, as well as to fine-tune various controls for the depiction of the cues themselves. While the system does provide adapted default values for each control, extracted from the motion capture data, it allows fine-tuning for greater expressiveness. Moreover, these cues are drawn in real time, and maintain a coherent display with changing viewpoints.We demonstrate the benefit of our interactive system on various motion capture sequences.

There are many applications that demand large quantities of natural looking motion. It is difficult to synthesize motion that looks natural, particularly when it is people who must move. In this paper, we present a framework that generates human motions by cutting and pasting motion capture data. Selecting a collection of clips that yields an acceptable motion is a combinatorial problem that we manage as a randomized search of a hierarchy of graphs. This approach can generate motion sequences that satisfy a variety of constraints automatically. The motions are smooth and human-looking. They are generated in real time so that we can author complex motions interactively. The algorithm generates multiple motions that satisfy a given set of constraints, allowing a variety of choices for the animator. It can easily synthesize multiple motions that interact with each other using constraints. This framework allows the extensive re-use of motion capture data for new purposes.

We present a physics-based approach to generate 3D biped character animation that can react to dynamical environments in real time. Our approach utilizes an inverted pendulum model to online adjust the desired motion trajectory from the input motion capture data. This online adjustment produces a physically plausible motion trajectory adapted to dynamic environments, which is then used as the desired motion for the motion controllers to track in dynamics simulation. Rather than using Proportional-Derivative controllers whose parameters usually cannot be easily set, our motion tracking adopts a velocity-driven method which computes joint torques based on the desired joint angular velocities. Physically correct full-body motion of the 3D character is computed in dynamics simulation using the computed torques and dynamical model of the character. Our experiments demonstrate that tracking motion capture data with real-time response animation can be achieved easily. In addition, physically plausible motion style editing, automatic motion transition, and motion adaptation to different limb sizes can also be generated without difficulty.