Cluster Haptic Texture Dataset: Haptic Texture Dataset with Varied Velocity-Direction Sliding Contacts.

The eye ground texture is disturbed by non ideal imaging factor such as noise, it will affect the clinical diagnosis in practice, an improved multi scale retina eye ground texture recognition algorithm is proposed based on fusion area threshold. The nonlinear sampling multi-scale transform is used to analyze the geometric space coefficient of retinal vessels with multi direction and shift invariant features, the regional threshold filtering is integrated, it is used to suppress the effect of non-uniform blocks for texture recognition. The maximum likelihood local mean standard deviation analysis is used for texture parameters estimation and recognition. The noise reduced greatly, accurate identification of texture feature is obtained. Simulation results show that the algorithm can well characterize the retinal vascular texture, it has good performance in different texture feature recognition, the recognition accuracy is improved, and it has good robustness.

Vision-based material recognition for automated monitoring of construction progress and generating building information modeling from unordered site image collections

Humans use periodically ridged fingertips to precisely perceive the characteristics of objects via ion-based fast- and slow-adaptive mechanotransduction. However, designing artificial ionic skins with fingertip-like tactile capabilities remains challenging because of the contradiction between structural compliance and pressure sensing accuracy (e.g., anti-interference from stretch and texture recognition). Inspired by the formation and modulus-contrast hierarchical structure of fingertips, an aesthetic ionic skin grown from a non-equilibrium Liesegang patterning process is introduced. This ionic skin with periodic stiff ridges embedded in a soft hydrogel matrix enables strain-undisturbed triboelectric dynamic pressure sensing as well as vibrotactile texture recognition. By coupling with another piezoresistive ionogel, an artificial tactile sensory system is further fabricated as a soft robotic skin to mimic the simultaneous fast- and slow-adaptive multimodal sensations of fingers in grasping actions. This approach may inspire the future design of high-performance ionic tactile sensors for intelligent applications in soft robotics and prosthetics.

The rapid advancements in robotics and automation have highlighted the need for robotic systems capable of adapting to the diverse physical properties of objects. Traditional grippers often lack the versatility to handle both hard and soft objects without extensive reprogramming or hardware adjustments. Previous studies have explored various approaches to this challenge, including tactile sensors and force feedback mechanisms to distinguish object properties. For instance, research by Calandra et al. (2018) utilized deep learning with tactile data to enable robotic hands to identify objects and adjust grip accordingly. Similarly, the paper by Li et al. (2020) “Design and performance characterization of a soft robot hand with fingertip haptic feedback for teleoperation”, focuses on designing and characterizing a soft robotic hand with fingertip haptic feedback for teleoperation emphasizing real-time tactile sensing and feedback mechanisms. However, these studies primarily focus on tactile feedback or specialized hardware, limiting their applicability in scenarios where such systems are not available or practical.This study introduces a novel machine learning-based approach, focusing on the use of visual data alone to classify and adapt to the hardness or softness of objects. By leveraging the CIFAR-100 dataset, we trained a deep learning model based on the ResNet50 architecture, achieving significant results in binary classification of hard and soft objects. The CIFAR-100 dataset, consisting of 100 diverse object categories/classes, was reorganized into two classes: hard (39 categories) and soft (61 categories). The ResNet50 model, pre-trained on ImageNet, was fine-tuned specifically for this task, with modifications to the last 210 layers to enhance its adaptability.Data augmentation techniques, including rotations, translations, shearing, zooming, and horizontal flipping, were applied to simulate real-world variations, ensuring robust learning. The model was further refined with additional fully connected layers, dropout, and batch normalization to prevent overfitting. Optimized using the AdamW optimizer, the model achieved a training accuracy of 83.31% and a validation accuracy of 80.25%, with a test accuracy of 80%. The precision, recall, and F1-scores were 0.82, 0.86, and 0.84 for the soft object class, and 0.76, 0.71, and 0.73 for the hard object class, demonstrating the model’s effectiveness in distinguishing between hard and soft objects without the need for specialized sensors. It is also observed that the accuracy of the model in relation to hard objects is significantly lesser as compared to soft objects. It is understood that the CIFAR-100 dataset (comprising of 100 classes) is inadequate for model training, so we are exploring the ILSVRC (ImageNet subset) dataset for model training in future. The research is useful in a variety of fields where the focus lies on object handling. In everyday life, we encounter a wide array of objects with varying degrees of hardness, requiring different levels of care and precision during handling. By relying on visual data and machine learning algorithms, as demonstrated in this research, robotic systems can become more autonomous and versatile, reducing the burden on human operators and improving overall efficiency. This approach can lead to cost savings by reducing the need for specialized hardware, such as tactile sensors, which are often expensive and difficult to integrate.

‘Applications of virtual reality’ documents some of the wider applications of VR and considers the value of good sound design across such applications. Here, we take a look at data visualisation, telepresence, education, creative applications, rapid product prototyping and health applications. Throughout this chapter, sound is positioned as an underappreciated yet vitally powerful contributor across all wider applications of VR; from increasing the ease of comprehension in VR visualisations, enabling more complex data sets to be more readily interpreted, to facilitating the retention of more information in an eLearning application by increasing users’ engagement and immersion in the material.

In robotics, tactile perception is important for fine control using robot grippers and hands. To effectively incorporate tactile perception in robots, it is essential to understand how humans use mechanoreceptors and proprioceptors to perceive texture. Thus, our study aimed to investigate the impact of tactile sensor arrays, shear force, and the positional information of the robot's end effector on its ability to recognize texture. A deep learning network was employed to classify tactile data from 24 different textures that were explored by a robot. The input values of the deep learning network were modified based on variations in the number of channels of the tactile signal, the arrangement of the tactile sensor, the presence or absence of shear force, and the positional information of the robot. By comparing the accuracy of texture recognition, our analysis revealed that tactile sensor arrays more accurately recognized the texture compared to a single tactile sensor. The utilization of shear force and positional information of the robot resulted in an improved accuracy of texture recognition when using a single tactile sensor. Furthermore, an equal number of sensors placed in a vertical arrangement led to a more accurate distinction of textures during exploration when compared to sensors placed in a horizontal arrangement. The results of this study indicate that the implementation of a tactile sensor array should be prioritized over a single sensor for enhanced accuracy in tactile sensing, and the use of integrated data should be considered for single tactile sensing.

A real-time face tracker based on both sound and visual cues is presented. Initial talker locations are estimated acoustically from microphone array data while precise localization and tracking are derived from visual data. The image processing employs a hierarchical structure which utilizes source motion, contour geometry, color data, and facial features. The resulting system is capable of tracking multiple persons in complex backgrounds and robustly discriminating faces from similar objects. While the direct focus of this work is automated videoconferencing, the face tracking capability has utility to many multimedia and virtual reality applications.

As robotics prehension systems and virtual reality applications are in constant evolution, the need for high-fidelity haptic interaction increases. This helps ensure and enhance user immersion and handling precision. While commercial haptic interfaces offer high performance, their prohibitive cost limits their widespread adoption in general-purpose robotics. Furthermore, many low-cost solutions suffer from limited transparency, where the operator constantly fights the friction of the actuator even during free motion. This article presents the design and development of an innovative, cost-effective master-slave robotic system aimed at democratizing efficient haptic feedback devices. The solution is intended for remote manipulation of objects with a maximum mass of 1 kg, while limiting the gripping force to 50 N, thus ensuring the integrity of objects being manipulated. The device includes a master haptic module in the form of a clamp that reproduces the thumb-index-middle finger gripping motion performed by the user. The system relies on a custom haptic interface measuring the angular position of the master gripper, which is transmitted in real time to the slave gripper, so as to adjust the position of the clamp accordingly, thus optimizing the grasping control loop. As soon as an object is detected, using a force sensor integrated into the slave gripper, the master motor renders a resistive force, preventing the user from closing the haptic module. The other part of the system is the slave mechanical gripper with three fingers, each with three phalanges based on human anatomy, allowing the clamp to mechanically conform to irregular object geometries with a single actuator. The last but not least innovative aspect lies in the implementation of a current sensor, which provides the haptic feedback. The force applied by the user is reproduced by the slave gripper using current sensors, eliminating the need for expensive force-torque sensors while maintaining a responsive feedback loop.

We are witnessing research expansion and application of robotics in almost all segments of human's activity, from industry to space exploration. Similar happen with humanoid robotics, but it has to be pointed out that its applications areas are mostly situated in environment close to humans. This imposes some specificities that have to be encountered. Two of them are, in our opinion, of essential importance. First one is that human environment is either loosely structured or completely unstructured (as a consequence, walking realization by programming it in advance is not possible and on-line generation of walk is inevitable), while second one is that robot's physical contact with surrounding objects, humans or other robots almost can not be avoided. If unexpected (and unwanted) such contact can be considered as disturbance and can jeopardize humanoid's dynamic balance, whose preservation is prerequisite for any biped activity. In both situations new approaches are needed. We have proposed new method for on-line walking generation based on set of predefined primitive movements that can be easily combined into more complex ones. Thus, basic leg's motion is always same (combined of primitives), and just parameters of primitives (step length, height of foot trajectory, leg bending intensity, walking velocity, …) have to be adjusted on-line during walking. The second issue, closely related to walking, is preservation of dynamic balance, either in static posture or during walking. Because existence of dynamic balance is assumed for any biped performing gait, its preservation is of primary importance. However, compensation strategies are not same for disturbances of various intensities. For real application this is extremely important and complex issue, particularly in case of large disturbances. Compensation action must simultaneously ensure two, often contradictory, requirements (to prevent immediate biped overturn and to bring system into state which enables previous activity to be continued when compensation action is over) and we propose compensation to be composed as synergetic action of more DOFs simultaneously.

Because of their outstanding electrical and mechanical properties, graphene strain sensors have attracted extensive attention for electronic applications in virtual reality, robotics, medical diagnostics, and healthcare. Although several strain sensors based on graphene have been reported, the stretchability and sensitivity of these sensors remain limited, and also there is a pressing need to develop a practical fabrication process. This paper reports the fabrication and characterization of new types of graphene strain sensors based on stretchable yarns. Highly stretchable, sensitive, and wearable sensors are realized by a layer-by-layer assembly method that is simple, low-cost, scalable, and solution-processable. Because of the yarn structures, these sensors exhibit high stretchability (up to 150%) and versatility, and can detect both large- and small-scale human motions. For this study, wearable electronics are fabricated with implanted sensors that can monitor diverse human motions, including joint movement, phonation, swallowing, and breathing.

Decoupling geometry and appearance in Gaussian splatting for reflective surface reconstruction: A glossy image prior-guided approach.

A virtual reality (VR) study was conducted both in our lab setting and remotely to investigate the effects of virtual hand appearance and tactile feedback on ownership, external appearance, and tactile sensation embodiment dimensions when participants were instructed to perform an assembly task in a virtual environment. Six experimental conditions that combine hand appearances (abstract, mannequin, and realistic) and tactile feedback (tactile and no tactile) levels were examined. The results of our study indicated that: (1) the more realistic hand had a stronger effect on tactile sensation and external appearance, while the mannequin hand was associated with a greater ownership effect; (2) tactile feedback was associated with a stronger effect on external appearance compared to no tactile feedback; (3) the realistic hand model in conjunction with tactile feedback significantly affected the tactile sensation of the participants; (4) the participants in the lab setting rated the external appearance of the realistic hand model higher than the remote participants; (5) the participants exposed for less than one hour per week to VR rated the tactile sensation in the presence of tactile feedback higher, while they similarly rated the external appearance of the virtual hand model in the presence of a realistic hand model higher when compared to those who were exposed for more than one hour per week to VR; and finally, (6) the younger participants rated the tactile sensation higher than the older participants in the presence of the realistic hand model. In this paper, we discuss our findings and provide design considerations for virtual reality applications that consider virtual hands and tactile feedback.

Human postural control is regulated by the vestibular, somatosensory, and visual systems. These types of sensory information are integrated in the central nervous system to ascertain the body’s position in space. Proper functioning of the vestibular, somatosensory, and visual senses is necessary for the body to maintain equilibrium. Bilateral vestibulopathy (BVP) is a condition in which bilateral peripheral vestibular function is reduced. Its treatment includes vestibular rehabilitation (VeR), balance training, counseling, treating the underlying cause, and avoiding further damage to the vestibular system. As VeR is often tedious for patients, patient motivation is required or patients may drop out of the program. To solve this problem, in recent years, there have been increasing reports of VeR using virtual reality, which increases vestibulo-ocular reflex gain and decreased dizziness by inducing adaptation. In this review, we discuss VeR, particularly for BVP, and VeR using virtual reality.

In this thesis, concepts for developing Spatial Data Infrastructures with an emphasis on visualizing 3D landscape and city models in distributed environments are discussed. Spatial Data Infrastructures are important for public authorities in order to perform tasks on a daily basis, and serve as research topic in geo-informatics. Joint initiatives at national and international level exist for harmonizing procedures and technologies. Interoperability is an important aspect in this context - as enabling technology for sharing, distributing, and connecting geospatial data and services. The Open Geospatial Consortium is the main driver for developing international standards in this sector and includes government agencies, universities and private companies in a consensus process. 
\n3D city models are becoming increasingly popular not only in desktop Virtual Reality applications but also for being used in professional purposes by public authorities. Spatial Data Infrastructures focus so far on the storage and exchange of 3D building and elevation data. For efficient streaming and visualization of spatial 3D data in distributed network environments such as the internet, concepts from the area of real time 3D Computer Graphics must be applied and combined with Geographic Information Systems (GIS). For example, scene graph data structures are commonly used for creating complex and dynamic 3D environments for computer games and Virtual Reality applications, but have not been introduced in GIS so far. 
\nIn this thesis, several aspects of how to create interoperable and service-based environments for 3D spatial data are addressed. These aspects are covered by publications in journals and conference proceedings. The introductory chapter provides a logic succession from geometrical operations for processing raw data, to data integration patterns, to system designs of single components, to service interface descriptions and workflows, and finally to an architecture of a complete distributed service network.
\nDigital Elevation Models are very important in 3D geo-visualization systems. Data structures, methods and processes are described for making them available in service based infrastructures. A specific mesh reduction method is used for generating lower levels of detail from very large point data sets. An integration technique is presented that allows the combination with 2D GIS data such as roads and land use areas. This approach allows using another optimization technique that greatly improves the usability for immersive 3D applications such as pedestrian navigation: flattening road and water surfaces. It is a geometric operation, which uses data structures and algorithms found in numerical simulation software implementing Finite Element Methods. 
\n3D Routing is presented as a typical application scenario for detailed 3D city models. Specific problems such as bridges, overpasses and multilevel networks are addressed and possible solutions described. The integration of routing capabilities in service infrastructures can be accomplished with standards of the Open Geospatial Consortium. An additional service is described for creating 3D networks and for generating 3D routes on the fly. Visualization of indoor routes requires different representation techniques.
\nAs server interface for providing access to all 3D data, the Web 3D Service has been used and further developed. Integrating and handling scene graph data is described in order to create rich virtual environments. Coordinate transformations of scene graphs are described in detail, which is an important aspect for ensuring interoperability between systems using different spatial reference systems. The Web 3D Service plays a central part in nearly all experiments that have been carried out. It does not only provide the means for interactive web-visualizations, but also for performing further analyses, accessing detailed feature information, and for automatic content discovery.
\nOpenStreetMap and other worldwide available datasets are used for developing a complete architecture demonstrating the scalability of 3D Spatial Data Infrastructures. Its suitability for creating 3D city models is analyzed, according to requirements set by international standards. A full virtual globe system has been developed based on OpenStreetMap including data processing, database storage, web streaming and a visualization client.
\nResults are discussed and compared to similar approaches within geo-informatics research, clarifying in which application scenarios and under which requirements the approaches in this thesis can be applied.
\n

Propelled by the video game industries, virtual reality (VR) systems are on the verge of entering the electronic mass media market. VR not just allows generating imaginary worlds, but also visualizing real geographies and possible past and future scenarios of real-world environments. However, VR capabilities of current geographic information systems (GIS) are limited as yet, thus challenging the transport of geodata into VR applications. Given the shortcomings of GIS, the present article proposes a practical workflow for the visualization of geographic information with immersive environments. Principal work steps are 3D-modeling, composition, performance optimization and the final dissemination on VR platforms. We will illustrate all elements of our workflow using the example of an immersive VR application of a coral reef in the Mexican Caribbean, thus providing both theoretical and practical background on the visualization of real-world data in VR space.