An Electronic Travel Aid based on multi-sensor fusion using extended Kalman filter

Mobile robot localization and map building are very important for a mobile robot to perform a navigation-based service task to aid the aged or disabled in an indoor environment (office, facilities or at home). In order to enable mobile robot to perform service tasks autonomously, the information of environment in which the mobile robot moves is needed. To obtain the accurate map, many different sensors system and techniques have been developed. Approaches of mapping can roughly be classified according to the kind of sensor data processed and the matching algorithms [S. Thrun et al., 1996, 1998, 2000, 2004; Chong, K.S et al., 1996; C.-C. Wang et al., 2002; G. Dissanayake et al., 2000; B. barshan et al., 1995; Weimin Shen et al., 2005]. In this paper, we proposed a method of map building using interactive GUI for a mobile robot. The reason we proposed this method is that it is difficult for a mobile robot to generate an accurate map although many kinds of sensors are used. The proposed method enables the operator to modify map built by sensors, compared with the real-time video from web camera by using modification tool in interactive GUI. Laser Range Finder (LRF) was used to get the information of the environment in which mobile robot moving. In order to improve self-localization of mobile robot, Extended Kalman Filter (EKF) was introduced. By the results of simulation and experiments, the developed EKF is effective in self-localization of mobile robot. This paper introduces the architecture of the system and gives some experimental results to verify the effectiveness of the developed system. The rest of the paper consists of 5 sections. Section 2 describes the structure of the proposed system. Section 3 presents the developed algorithm of self-localization of mobile robot and gives some simulation and experimental results to verify the effectiveness of the EKF in improving precision of positioning of mobile robot. Section 4 details the developed LRF data processing algorithm. Section 5 introduces the developed interactive GUI. The experimental results are given in Section 6. Section 7 concludes the paper. 13

Blind and visually impaired people often use an object-to-object strategy to explore unknown spaces through physical contact via their canes or bodies. Camera-based mobile electronic travel aids (ETAs) not only offer a larger work range than a white cane while detecting obstacles, but also enable users to recognise objects of interest without physical contact. In this paper, we conducted a case study with seven visually impaired participants, to investigate how they use a depth-sensing camera-based ETA for exploring unknown spaces, and how they reconstruct cognitive mappings of surrounding objects. We found that camera-based mobile ETAs would assist visually impaired users to explore the surrounding environment effectively and without physical contact. Furthermore, our original study indicates ETA users change their strategies for exploring the surrounding environment from using an object-to-object approach to an to observation-point strategy. Participants improved their cognitive mappings of the surrounding environment compared to white cane users.

With the development of advanced human-machine interface, effective information processing algorithms, and more powerful microprocessors, it is possible to enable the blind to achieve additional perception of the environment. In 1970s, the blind-assistant facility based on sensory substitution was introduced[1]. It called as ETA (short for Electronic Travel Aids), bases on the other natural senses of the blind, such as hearing, touch, smell, feeling and etc. Then the research of vision aids has been broadly extended. After introducing the traditional methods for guiding blind, two typical input modes of travel aid are presented in this paper. One is sonar-based ETAs and the other technique is camera-based ETAs. Its principle of each modes is discussed in detail. Besides these, comparisons between the above two techniques have also been indicated in this paper.

The Extended Kalman Filter (EKF) is both the historical algorithm for\nmulti-sensor fusion and still state of the art in numerous industrial\napplications. However, it may prove inconsistent in the presence of\nunobservability under a group of transformations. In this paper we first build\nan alternative EKF based on an alternative nonlinear state error. This EKF is\nintimately related to the theory of the Invariant EKF (IEKF). Then, under a\nsimple compatibility assumption between the error and the transformation group,\nwe prove the linearized model of the alternative EKF automatically captures the\nunobservable directions, and many desirable properties of the linear case then\ndirectly follow. This provides a novel fundamental result in filtering theory.\nWe apply the theory to multi-sensor fusion for navigation, when all the sensors\nare attached to the vehicle and do not have access to absolute information, as\ntypically occurs in GPS-denied environments. In the context of Simultaneous\nLocalization And Mapping (SLAM), Monte-Carlo runs and comparisons to OC-EKF,\nrobocentric EKF, and optimization-based smoothing algorithms (iSAM) illustrate\nthe results. The proposed EKF is also proved to outperform standard EKF and to\nachieve comparable performance to iSAM on a publicly available real dataset for\nmulti-robot SLAM.\n

We have developed a dead reckoning sensor system and a tracking algorithm for mobile robots to estimate the path when a mobile robot explores an unknown, enclosed region where GPS access or landmarks are unavailable. A dead reckoning sensor system consists of a low-cost MEMS IMU and a navigation sensor (used in laser mice), which provide complementary functions. The IMU has benefits such as compact size, a self-contained system, and an extremely low failure rate but has a bias drift problem, which can accumulate substantial error over time. A navigation sensor measures the motion of a mobile robot directly without the slip error in the case of a wheel-type odometer, but it often fails to read a surface. A tracking algorithm consists of an extended Kalman filter (EKF) to fuse data from the IMU and the navigation sensor and a least-squares method to estimate acceleration bias in the EKF. We obtained experimental data by driving a radio-controlled car equipped with the sensor system in a 3D pipeline and compared the path estimated by the tracking algorithm with the path of the pipeline. The tracking algorithm combined data from the IMU and the navigation sensor and correctly estimated the path of the radio-controlled car. Our study can be applied to estimate position or path of mobile robots without external aids such as GPS, landmarks, and beacons.

This paper presents a study on quantitative dynamics analysis of human lower limb using developed wearable sensor systems that can measure reaction force and detect the following gait phases: initial contact, loading response, mid stance, terminal stance, pre-swing, initial swing, mid swing and terminal swing. Since conventional camera-based motion analysis system and reaction force plate system require costly devices, vast space as well as time-consuming calibration experiments, the wearable sensor-based system is much cheaper. Gyroscopes and two-axis accelerometers are incorporated in this wearable sensor system. The former are attached on the surface of the foot, shank and thigh to measure the angular velocity of each segment, and the latter are used to measure inclination of the attached leg segment (shank) in every single human motion cycle for recalibration. Ground reaction forces during human walking are synchronously measured using a wearable force sensor integrated in a shoes mechanism. Finally, experiment has been performed to compare the measurement results from the wearable sensor system with the data obtained from an optical motion analysis system and a force plate. The results showed that the measurement of human lower limb orientations and reaction forces for human dynamics analysis could be reliably implemented using the wearable sensor systems.

Development and Experimental Validation of an Adaptive Estimation Algorithm for the On-Line Localization of Mobile Robots by Multisensor Fusion

Objective: Movement impairments such as those due to stroke often result in the nervous system adopting atypical movements to compensate for movement deficits. Monitoring these compensatory patterns is critical for improving functional outcomes during rehabilitation. The purpose of this study was to test the feasibility and validity of a wearable sensor system for detecting compensatory trunk kinematics during activities of daily living. Approach: Participants with no history of neurological impairments performed reaching and manipulation tasks with their upper extremity, and their movements were recorded by a wearable sensor system and validated using a motion capture system. Compensatory movements of the trunk were induced using a brace that limited range of motion at the elbow. Main results: Our results showed that the elbow brace elicited compensatory movements of the trunk during reaching tasks but not manipulation tasks, and that a wearable sensor system with two sensors could reliably classify compensatory movements (~90% accuracy). Significance: These results show the potential of the wearable system to assess and monitor compensatory movements outside of a lab setting.

Estimation of vivo muscle forces during human motion is important for understanding human motion control mechanisms and joint mechanics. This paper combined the advantages of the convolutional neural network (CNN) and long-short-term memory (LSTM) and proposed a novel muscle force estimation method based on CNN-LSTM. A wearable sensor system was also developed to collect the angles and angular velocities of the hip, knee, and ankle joints in the sagittal plane during walking, and the collected kinematic data were used as the input for the neural network model. In this paper, the muscle forces calculated using OpenSim based on the Static Optimization (SO) method were used as the standard value to train the neural network model. Four lower limb muscles of the left leg, including gluteus maximus (GM), rectus femoris (RF), gastrocnemius (GAST), and soleus (SOL), were selected as the studying objects in this paper. The experiment results showed that compared to the standard CNN and the standard LSTM, the CNN-LSTM performed better in muscle forces estimation under slow (1.2 m/s), medium (1.5 m/s), and fast walking speeds (1.8 m/s). The average correlation coefficients between true and estimated values of four muscle forces under slow, medium, and fast walking speeds were 0.9801, 0.9829, and 0.9809, respectively. The average correlation coefficients had smaller fluctuations under different walking speeds, which indicated that the model had good robustness. The external testing experiment showed that the CNN-LSTM also had good generalization. The model performed well when the estimated object was not included in the training sample. This article proposed a convenient method for estimating muscle forces, which could provide theoretical assistance for the quantitative analysis of human motion and muscle injury. The method has established the relationship between joint kinematic signals and muscle forces during walking based on a neural network model; compared to the SO method to calculate muscle forces in OpenSim, it is more convenient and efficient in clinical analysis or engineering applications.

Computer vision is the area of imitation of biological vision system using computer algorithms and technologies. One of the main usages of this innovative technology is for navigation. Some of the cardinal and challenging parts in navigation are recognition of safe routes and distinguishing them from obstacles. In this paper an innovative algorithm is described and developed for this purpose. This algorithm can be utilized in camera based electronic travel aid (ETA) devices or autonomous vehicles. Furthermore, the focus here is to use this algorithm in an ETA device for helping visually impaired person (VIP). Different methods and techniques can be used in order to develop such an algorithm. Here, the algorithm is designed in a way that generates depth map using block matching technique and process it using Random Sample Consensus (RANSAC) with extra criteria obtained from Inertial Measurement Unit (IMU) in order to extract the ground plane and discover the obstacles. Finally, the algorithm is implemented in a prototype and several experiments are performed in order to measure the process time and success rate.

Location based service (LBS) cannot be realized unless the location of the user is available. For indoor LBS, indoor positioning must be utilized and many researchers have been working on indoor positioning and tracking. For example, extended Kalman filter (EKF) was exploited in Bluetooth based indoor positioning. Nowadays, WLAN (wireless local area network) is available virtually everywhere. Thus, WLAN based indoor positioning and tracking is more economical than Bluetooth based ones. The main purpose of this paper is to propose a new WLAN based EKF indoor tracking method by extending existing Bluetooth based EKF positioning method. Experimental results comparing our method with other previous methods are discussed.

With the advancement of intelligent manufacturing and flexible automation, mobile industrial robots are increasingly being deployed in scenarios such as material handling, inspection, and collaborative operations. As one of the core technologies for mobile robots, the localization system has a direct impact on the stability and accuracy of path planning and task execution. This paper, built on the Robot Operating System (ROS) platform, systematically reviews and analyzes the implementation mechanisms and applicable scenarios of mainstream localization algorithms, with a focus on the performance characteristics of the Extended Kalman Filter (EKF) and Adaptive Monte Carlo Localization (AMCL) in industrial settings. By constructing an experimental platform that fuses multiple sensors—lidar, IMU, and wheel odometry—a series of tests comparing localization accuracy and robustness are conducted. We evaluate each algorithm's adaptability to complex conditions including dynamic occlusions, uniform environmental textures, and multipath interference. The results indicate that AMCL achieves higher positioning accuracy in static, structured environments, whereas EKF is better suited to dynamic applications suffering from sensor drift and data latency. Finally, we propose an optimization approach that integrates visual SLAM and deep-learning–based feature extraction, offering guidance for designing highly reliable localization systems for future industrial robots.

A navigation algorithm is proposed to increase the inertial navigation performance of a ground vehicle using magnetic measurements and dynamic constraints. The navigation solutions are estimated based on inertial measurements such as acceleration and angular velocity measurements. To improve the inertial navigation performance, a three-axis magnetometer is used to provide the heading angle, and nonholonomic constraints (NHCs) are introduced to increase the correlation between the velocity and the attitude equation. The NHCs provide a velocity feedback to the attitude, which makes the navigation solution more robust. Additionally, an acceleration-based roll and pitch estimation is applied to decrease the drift when the acceleration is within certain boundaries. The magnetometer and NHCs are combined with an extended Kalman filter. An experimental test was conducted to verify the proposed method, and a comprehensive analysis of the performance in terms of the position, velocity, and attitude showed that the navigation performance could be improved by using the magnetometer and NHCs. Moreover, the proposed method could improve the estimation performance for the position, velocity, and attitude without any additional hardware except an inertial sensor and magnetometer. Therefore, this method would be effective for ground vehicles, indoor navigation, mobile robots, vehicle navigation in urban canyons, or navigation in any global navigation satellite system-denied environment.

A wearable sensor system is proposed to determine several walking features. including stride length, cadence, walking speed, incline. The sensor system is composed of an inertial measurement unit (IMU) mounted on the foot instep and a biaxial accelerometer mounted on the waist. The IMU integrates a biaxial accelerometer and a rate gyroscope. Both the foot gyroscope and the waist accelerometer are capable to accurately estimate the cadence. Strapdown integration applied to the inertial data allows to reconstruct the trajectory in the sagittal plane of the sensed anatomical point on the foot, yielding the opportunity to determine the desired walking features. Double-integration of the waist accelerometer signals to determine the trunk vertical displacement and root mean square analysis of the waist acceleration modulus allow to construct two predictors of the walking speed, which integrate the information gathered from the inertial estimator. In this paper, preliminary results concerning the accuracy of level walking speed prediction by different methods are presented.

In recent years, with the rapid development of robotics, indoor mobile robots have gradually penetrated many aspects of human production and life. High accuracy localization is important for mobile robots in a variety of scenarios, including domestic and public places. In this paper, we present a fusion approach for indoor localization of a smartphone-based robot, which fuses the information of inertial sensors, encoder and indoor maps. In our approach, we first propose a dead reckoning model to obtain the position and attitude of the robot. Secondly, the information obtained by the encoder is used to identify the turning area. The difference between the turning position by our proposed algorithm and the known turning position is compared with a preset threshold. The estimated position converge on its true position. Finally, we design an Extended Kalman Filter (EKF)based fusion algorithm to fuse the inertial data, encoder data and map information. We evaluate our proposed method in an office building. The results show that the mean positioning error of the proposed method on the two routes is 0.7220m and 0.7150m respectively, which is a 60.4% and 68.3% reduction compared with the fusion of DR/Encoder approach.