An angle-changeable tracked robot with human-robot interaction in unstructured environments

Advanced robots are desired to improve their reasoning ability in increasingly practical situations to quickly respond to urgent challenges such as search and rescue, public security, and autonomous driving. Advances in sensor technology and intelligent algorithms are accelerating the development of autonomous systems, allowing them to interact with dynamic environments more independently and intelligently, completing relatively complicated tasks. Nevertheless, the safety and security problems are yet to be comprehensively addressed. Currently, many efforts have been made to solve this problem by improving the unimodal perception ability of intelligent systems. Methods using surveillance cameras to collect large amounts of visual data and process them using deep networks. Various types of range scanner can sense and reconstruct the surrounding environment through precise distance measurements. However, in the above cases, there may be a lack of training data, fixed cameras cannot adequately and dynamically model the surroundings, unimodal analysis is hard to capture both appearance and depth information. Therefore, higher levels of perception and reasoning abilities are required to handle unexpected conditions and scenarios efficiently. In such cases, mobile robots should be fully aware of the dynamic environment, prioritize, and continuously analyze critical situations, share information, and collaboration under limited computing power and resources. Furthermore, considering the robots are operating alongside humans, reliable and interpretable robot behavior is highly preferred for more natural interactions.
\n
\nThis study aims to develop reasoning strategies for autonomous robots to improve robotic intelligence under different assumptions and conditions. The framework allows one or several robots to address safety or security problems in a targeted manner in partially known environments, where time, dynamics, and interactions play a significant role. The objective is to identify the unique individual whose activity, in space and time, appear different from the other people in the scene. The activities considered not only the motion pattern, but also the object associated with this person. Unlike traditional abnormal detection, the uniqueness reasoning in this work does not require a predefined expected pattern. A modular analysis is also applied to decompose the entire reasoning process into submodules such as perception, analysis, and inference, to achieve algorithmic flexibility and interpretability. Probabilistic modeling is used to associate spatial relationships between semantic labels and their temporal changes. The relationships are described in either instant spatial spaces (semantic-interaction feature) or spatiotemporal spaces (spatiotemporal feature), and it provides essential information for robot navigation, situation awareness, and task planning. Unlike existing research, this is the first time that such a mission is implemented on mobile robot platforms by using the probabilistic fusion of multimodal information.
\n
\nConsidering how evidence information is obtained and linked in the process of reasoning, we study the problem of uniqueness reasoning at three levels. The first level is called knowledge-based single robot uniqueness reasoning and runs on a single robot platform. At this level, we focus on a knowledge-based approach, where knowledge is provided in terms of how motion and an object associated with a person can be used to infer his/her uniqueness, as determined by human expertise. Because such a method has advantages in dealing with structured missions and with limited training data. Qualitative rules are applied to extract human activities from observations, and then the uniqueness of each person is judged by associating the prior knowledge with the performed activities. Experiments were conducted in indoor environments with various degrees of occlusion, and the experimental results showed the rationality and accuracy of the reasoning.
\n
\nHowever, though the knowledge-based method can successfully address specific tasks, it lacks flexibility and generality to adapt to different situations. The assumption of prior knowledge should be replaced by general analytical modeling. Thus, the second level is analytics-based single robot uniqueness reasoning. For one thing, it applies quantitative analysis to model human activities from observations rather than using qualitative rules. For another, instead of leveraging prior knowledge, the distribution of the features characterizing a human activity is constructed and the activity of an individual is analyzed within the distribution to determine if it is unique. Experiments were conducted in both indoor and urban outdoor environments. The experimental results demonstrated the rationality, accuracy, and efficiency of the reasoning.
\n
\nMoreover, since many applications require to work in an unstructured and large-scale environment as well as a long-term operation, the sensing and reasoning capabilities of a single robot are limited by occlusion, sensor failure, and narrow field of view. Robots are desired to share information and perform collaborative reasoning. Therefore, if say the first two levels are mainly using sensory data to implement the algorithm, the third level is more focused on applying the algorithm to the robot platform. The third level is analytics-based multi-robot uniqueness reasoning, in which a multi-robot system is adopted as the platform. By obtaining the relative pose between robots, local observations can be transformed and transmitted among the multi-robot system. Since it is a distributed system, each robot still first performs its local uniqueness reasoning in the case of long-distance and poor communication. Meanwhile, the system performs a re-judgment based on the shared global observation and then shares the judgment results back to each robot. Experiments were carried out in various unstructured environments and contained day and night situations. The experimental results showed the rationality, accuracy, and robustness of the reasoning. This is the first time that reasoning is categorized by considering the way of obtaining and linking evidence information. This study has potential applications in a variety of situations, and it can serve as a guide for further robotic motion planning and human-robot interaction.

Enhancing Humans Trust in Robots through Explanations

Research on autonomous driving in unstructured outdoor environments is less advanced than in structured urban settings due to challenges like environmental diversities and scene complexity. These environments–such as rural areas and rugged terrains–pose unique obstacles that are not common in structured urban areas. Despite these difficulties, autonomous driving in unstructured outdoor environments is crucial for applications in agriculture, mining, and military operations. Our survey reviews over 250 papers for autonomous driving in unstructured outdoor environments, covering offline mapping, pose estimation, environmental perception, path planning, end‐to‐end autonomous driving, datasets, and relevant challenges. We also discuss emerging trends and future research directions. This review aims to consolidate knowledge and encourage further research for autonomous driving in unstructured environments. To support ongoing work, we maintain an active repository of up‐to‐date literature and open‐source projects at: https://github.com/chaytonmin/Survey‐Autonomous‐Driving‐in‐Unstructured‐Environments .

Intrinsically soft and fully recyclable robotic sensors with quadruple sensing functions for reliable human-robot interactions.

Robotic fingers made of soft material and compliant structures usually lead to superior adaptation when interacting with the unstructured physical environment. In this paper, we present an embedded sensing solution using optical fibers for an omni-adaptive soft robotic finger with exceptional adaptation in all directions. In particular, we managed to insert a pair of optical fibers inside the finger's structural cavity without interfering with its adaptive performance. The resultant integration is scalable as a versatile, low-cost, and moisture-proof solution for physically safe human-robot interaction. In addition, we experimented with our finger design for an object sorting task and identified sectional diameters of 94\% objects within the $\pm$6mm error and measured 80\% of the structural strains within $\pm$0.1mm/mm error. The proposed sensor design opens many doors in future applications of soft robotics for scalable and adaptive physical interactions in the unstructured environment.

Development of robotics and growing use of robots in many areas of the economy and social life require research which will identify problems related to the use of modern technologies and the development of human-robot interaction (HRI, Human-Robot Interaction). Robotics, autonomous systems and artificial intelligence also create a number of new legal and ethical challenges. Research in this area has been expanding in recent years, but there are still issues that require in-depth scientific exploration. The objective of this paper is to systematize knowledge about interactions between people and robots, with particular emphasis on interactions in the work environment, and to identify research gaps in the analysed problem area. The following research questions were formulated: 1) What determines people's attitudes towards robots? 2) What are the key challenges in human-robot communication? 3) What ethical challenges determine human-robot interaction? In the studies the the method of narrative literature review has been used. The latest scientific publications on the researched issue were analysed. The results of the study indicate that the main factors which determine people's attitudes towards robots are apart from demographic variables and experiences gained by people from contacts with robots, the perceived usefulness of using robots, the image of robots created in the mass media and religiousness. The key challenges in human-robot interaction apart from effective communication is the reflection of emotions. The research results enable the identification of further directions of scientific research on human-robot interaction. They can also enrich practitioner’s perspective in the studied area.

This article provides a general description of numerical modeling over complex terrain, including both diagnostic and prognostic approaches and their associated advantages and limitations. Diagnostic models solve simplified versions of the equations of motion and are widely used in the engineering community for simulations of flow over complex terrain. These models produce a description of the steady‐state flow, while requiring few inputs and being computationally efficient. Diagnostic models do not predict time evolution of flow, and are often only appropriate for terrain with small slopes. Prognostic models solve the full equations of motion and are used for both engineering and numerical weather prediction applications. Details of the implementation of prognostic models vary widely, but they are capable of producing time‐varying descriptions of the flow and handling very complex terrain with no limitations on terrain slope. These models can require detailed three‐dimensional inputs and large computational resources. Additional simplifications can reduce these requirements, but generally at the expense of functionality, such as producing a steady‐state solution or the inability to handle arbitrarily complex terrain.

Towards establishing future-oriented relations between humans and robots, not only verbal, but also nonverbal robot functions have been studied. However, it is not clear how these different functions should be combined and implemented in robots in order to achieve ‘natural’ human-robot communication or interaction. In this paper, we focus on the timing for the coordination between utterance as a verbal function and body motion as a nonverbal function. To begin with, we investigated the temporal order and correlation between utterance and gesture in an interpersonal (human-human) interaction scenario as a preliminary experiment (Experiment 1). Then, we controlled the robot's response timing based on findings of the first experiment and evaluated the effectiveness of the robot's timing control in human-robot interaction (Experiment 2). It was shown that elderly participants significantly preferred the controlled robot's response. In addition, the timing of utterance and gestures affected the impression of the robot and interpretation of the verbal message. These results are important findings for designing socially interactive robots for the aging society.

Rugged topography considerably regulates the surface downwelling long‐wave radiation (SDLR) flux and further affects the surface radiation and energy balances. The three dimensional sub‐grid terrain long‐wave radiative effect (3DSTLRE) is absent in most current numerical models, which usually adopt plane‐parallel schemes to simulate the SDLR flux. This study has developed a clear‐sky 3DSTLRE parameterization scheme based on the isotropic assumption of SDLR at rugged terrains and systematically evaluated its ability over the Tibetan Plateau (TP). Results show that the 3DSTLRE scheme achieves good and stable performance regardless of the horizontal resolution, time of the year, and sub‐grid terrain complexity. At different model horizontal resolutions ranging from 0.025° to 0.8°, the normalized mean absolute errors (NMAE) of the daily SDLR flux simulated by the clear‐sky 3DSTLRE scheme over most of TP are less than 0.9%, and the NMAE of the daily SDLR flux produced by the clear‐sky 3DSTLRE scheme regionally averaged over the grids with different sub‐grid terrain complexity are less than 0.25% in different months. Neglecting the 3DSTLRE in the plane‐parallel schemes may lead to clearly underestimated SDLR flux over the rugged areas, and the underestimation increases with the horizontal resolution and sub‐grid terrain complexity. At different model horizontal resolutions, the mean underestimation of the clear‐sky daily SDLR flux simulated by the plane‐parallel scheme over most of TP ranges from 5 to 20 W · m−2 with a relative underestimation of 4∼10%. The 3DSTLRE scheme can clearly reduce the biases of plane‐parallel scheme and exhibits wide application prospects in various numerical models.

Topographic correction models (TCMs) are valid on satellite image data preprocessing steps. Theillumination angle may be sensitive to different terrain slope and aspect conditions base on sun-terrain-sensorgeometry. Although the topographic correction is influenced by the sun azimuth and zenith angle, the correction resultcan be equally in the same image status. By contrast, the terrain factors change with different digital elevation model(DEM) resolution in the topographic correction equations and cause a significant effect. Slope is sensitive in ruggedterrain, and aspect is impressionable at flat surface at a coarse DEM resolution data. As the DEM resolution lead adistinct result on TCMs, this research is aimed to examine the impact of DEM resolution on the accuracy of terrainrepresentation and of the gradient determined. In this study, five TCMs, including cosine correction, C correction, SCS correction, SCS+C correction and Minnaert correction models are compared by different resolutions using SPOTimage data. The 5 meter DEM obtained from Ministry of the interior will be resampled to 10 to 500 meters to testthose topographic models sustainability on Lienhuachih Research Center. The accuracy of five topographic correctionmodels base on different DEM resolution will be evaluated by root-mean-square error (RMSE).

Topography impacts the fraction of radiation that reaches a target surface from the sun, sky, and surrounding terrains, which leads to distortions in the radiance levels that are observed by sensors. Although several mountain radiative transfer (RT) models have been developed, the surrounding topographic effects are rarely considered, which can cause non-negligible uncertainty in forward radiance modeling. In this study, we first extended the canopy reflectance model to be suitable for both continuous and discontinuous canopies over sloping terrain (BOST) to rugged terrain, and used it to investigate the impact of the surrounding topography on the reflected radiance and evaluate the contributions of diffuse irradiance from the sky and adjacent terrains. Discrete anisotropic RT (DART) simulations and remote sensing (RS) observations in a real mountain environment were used to evaluate the physical mechanism and model performance. The results suggested that the extended BOST model can capture terrain-induced variations in direct and diffuse radiation, which can be successfully used to simulate the reflected radiance in a real mountainous region. The model shows significant improvement compared with that before extension (i.e., root-mean-square error (RMSE) decreases (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$R^{2}$ </tex-math></inline-formula> increases) of 1.252 (0.025) and 2.035 (0.054) in the red and near-infrared (NIR) bands, respectively). In addition, we discovered that the surrounding topographic effects on the diffuse sky and terrain irradiance are significantly influenced by the wavelength, solar direction, and visible area of the sky and terrains. The extended BOST model serves as an effective tool for improving simulations of the observed radiance and facilitates the development of RT models for rugged terrains.

Teleoperation has been widely used to perform tasks in dangerous and unreachable environments by replacing humans with controlled agents. The idea of human-robot interaction (HRI) is very important in teleoperation. Conventional HRI input devices include keyboard, mouse and joystick, etc. However, they are not suitable for handicapped users or people with disabilities. These devices also increase the mental workload of normal users due to simultaneous operation of multiple HRI input devices by hand. Hence, HRI based on gaze tracking with an eye tracker is presented in this study. The selection of objects is of great importance and occurs at a high frequency during HRI control. This paper introduces gaze gestures as an object selection strategy into HRI for drone teleoperation. In order to test and validate the performance of gaze gestures selection strategy, we evaluate objective and subjective measurements, respectively. Drone control performance, including mean task completion time and mean error rate, are the objective measurements. The subjective measurement is the analysis of participant perception. The results showed gaze gestures selection strategy has a great potential as an additional HRI for use in agent teleoperation.

Robots instead of humans work in unstructured environments, expanding the scope of human work. The interactions between humans and robots are indirect through operating terminals. The mental workloads of human increase with the lack of direct perception to the real scenes. Thus, mental workload assessment is important, which could effectively avoid serious accidents caused by mental overloading. In this paper, the operating object is a dual-arm robot. The classification of operator’s mental workload is studied by using the heart rate variability (HRV) signal. First, two kinds of electrocardiogram (ECG) signals are collected from six subjects who performed tasks or maintained a relaxed state. Then, HRV data is obtained from ECG signals and 20 kinds of HRV features are extracted. Last, six different classifications are used for mental workload classification. Using each subject’s HRV signal to train the model, the subject’s mental workload is classified. Average classification accuracy of 98.77% is obtained using the K-Nearest Neighbor (KNN) method. By using the HRV signal of five subjects for training and that of one subject for testing with the Gentle Boost (GB) method, the highest average classification accuracy (80.56%) is obtained. This study has implications for the analysis of HRV signals characteristic of mental workload in different subjects, which could improve operators’ well-being and safety in the human-robot interaction process.

With the aim of reproducing the grasping and manipulation capabilities of humans, many robotic devices have been developed all over the world in more than 50 years of research, starting from very simple grippers, normally used in industrial activities, to very complex anthropomorphic robotic hands. Unfortunately, the reduced functionality and/or reliability of the devices developed so far prevent, together with the cost, their usability in unstructured environments, and in particular in human everyday activities. The adoption of design solutions inherited from conventional mechanics and the lack of purposely developed sensors and actuators are among the main causes of the partial fail in achieving the final goal of reproducing human manipulation capabilities. Our research activity aims at developing innovative solutions concerning the mechanical design, the sensory equipment and the actuation system for the implementation of anthropomorphic robotic hands with improved reliability, functionality and reduced complexity and cost, considering also aspects related to safety during human–robot interaction, paving the way toward the next generation of robotic hands.KeywordsAngular DisplacementTriangular ElementInnovative TechnologyTactile SensorFuse Deposition ModelingThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Human‐in‐the‐loop robotic system is an emerging technique in recent years. Human intelligence as well as machine intelligence are incorporated to accomplish tasks efficiently and effectively. However, grasp‐and‐lift (GAL) tasks through human–robot interactions are still a problem in an unstructured environment like urban search and rescue. Human assistive GAL tasks enable robots to complete search or rescue procedures quickly and accurately. Brain–machine interface (BMI) controlled robots have demonstrated promising applications in human–robot collaborative manipulations. In this study, an architecture of human–robot team is proposed for performing GAL tasks in BMI‐based human–robot systems. The proposed architecture contains several workflows from both human and robot aspects to improve performance. In addition, human brain activities are generally considered as non‐stationary signals with varying spatial and temporal distributions. To enhance robustness and stability of brain‐controlled robot's GAL tasks, a new method via adaptive boosting mechanism is proposed. The proposed multiple subjects' adaptive boosting is able to suppress noisy data and outliers in multiple subjects’ electroencephalogram signals, and therefore enhance accuracy and robustness of intention and sensation signal classification in GAL tasks. Preliminary results show that the new architecture is feasible with ethical establishment and the proposed method can outperform traditional methods.