Towards Active Machine-Vision-Based Driver Assistance for Urban Areas

In this study emotion-based face expression recognition framework has been proposed using a machine vision (MV) approach. The face emotion dataset has been collected local survey in Bahawalpur city dataset divide into 3 classes happy, sad, and angry. A total of 600 images of size (256 x 256) were transformed into a gray level format and employed a median filter for noise removal. Three non-overlapping regions of interest (ROIs) of size (50 x 50) have been taken and analyze 1800 (600 x 3) ROIs on the overall dataset. Total 45 Statistical features named as texture, histogram, and binary features were extracted. Select optimize features using the correlation-based feature section technique. The optimized dataset employed of MV classifiers namely random forest (RF), logistic (Lg), and J48 are obtained very promising accuracy 96.33%, 95.67%, and 95.33% respectively.

The Japan Digital Road Map Association (JDRMA) was founded to establish the standards for digital road map (DRM) and to produce a DRM database in Japan. The JDRMA is now in the process of refining the database using 1/25,000 scale topographical maps. The JDRMA supplies users with an updated version of the database every year. Car navigation is currently the largest application of DRM. Police departments, fire departments, ambulance services, and taxi companies use location systems that use DRM. Significant application of DRM is expected to come in the future in the planning and research fields. Road sign checking system, traffic accident analysis system, road census reference system and logistics system are now being studied. As the applications expand, more advanced DRMs must be made available, To meet this requirement, development of more detailed and precise DRMs and linkage to dynamic information systems is being pursued.

Japan Digital Road Map Association (JDRMA) started to construct the digital road map database for the whole Japan in 1988. The first database was completed in 1989, and consequently the updating started in 1990. There are two kinds of data sources for updating the JDRMA database. One is the newly issued topographic maps, which include both roads and other features but are not updated every year. The other is the information from the road administrators; it is used to update the road networks. Every year the information of the newly opened or improved roads is collected and used to update the road network in the database. Other information is also used to supplement the two main information sources. Documents of transportation facilities other than roads and information from the users are utilized. The investigation to improve the updating procedure is continued. >

The GPS-based navigation systems provide an accurate positioning of land-vehicles but this service is not always continuous in urban areas. These systems are usually enhanced with dead reckoning sensors, like odometers, and the use of a digital road map can improve the vehicle location when GPS fails. In this work, we focus on these GPS outages where the main problem is to combine another available measurements intelligently. We present here a global estimation method to solve the fusion problem of the GPS, odometer and digital road map measurements. It relies on an unscented Kalman filter whose main benefit is to deal with non-linear equations directly. In the case of an urban transport scenario, the comparison with an extended Kalman filter shows that the vehicle positioning is improved without requiring more computing time.

PurposeThe purpose of this paper is to describe innovative machine vision methods that have been employed for the capture and analysis of 3D skin textures; and the resulting potential for assisting with identification of suspicious lesions in the detection of skin cancer.Design/methodology/approachA machine vision approach has been employed for analysis of 3D skin textures. This involves an innovative application of photometric stereo for the capture of the textures, and a range of methods for analysing and quantifying them, including statistical methods and neural networks.Findings3D skin texture has been identified as a useful indicator of skin cancer. It can be used to improve realism of virtual skin reconstructions in tele‐dermatology. 3D texture features can also be combined with 2D features to obtain a more robust classifier for improving diagnostic accuracy, thereby assisting with the long‐term goal of implementing computer‐aided diagnostics for skin cancer.Originality/valueThe device developed for capturing 3D skin textures is known as the “Skin Analyser”, and as far as the authors know it is unique in the world in being able to recover 3D textures from pigmented lesions in vivo. There currently exist numerous methods for analysing lesions, including manual inspection (using established heuristics commonly known as ABCD rules), dermoscopy and SIAoscopy. The ability to capture and analyse 3D lesion textures complements these existing techniques and forms a valuable additional indicator for assisting with the early detection of dangerous skin cancers such as melanoma.

Reliable and accurate measurement of particle size and subsequent analysis of particle size distribution (PSD) is central to characterization of particulate minerals. Traditionally PSD of particulate materials was determined using standard mechanical sieving, but it was shown to be deficient in accurate particle size measurements, hence PSD analysis. In this paper a highly inexpensive machine vision based approach was proposed as an alternative to mechanical sieving. The test mineral used was ball-milled celestite. This machine vision approach used a document scanner as imaging device and a user-coded java ImageJ plugin performed the image processing and automated the PSD analysis. The acquired color images were preprocessed to binary images and the particles analyzed after grouping them based on their distinct length. Volumes of all particles were evaluated assuming a prolate spheroid geometry from measured lengths and widths. A new approach of using sum of volumes as weighting factor (Svolume) was utilized for particle grouping in ASABE Standard PSD analysis. The plugin also evaluated several significant dimensional (16) and distributional (27) parameters that characterize the PSD of samples. Standard mechanical sieving of samples was performed for experimental verification and results compared with machine vision method. The cumulative undersize PSD followed a log-normal distribution, and the plot against particle size exhibited a linear trend. Shapes of log-normal plot of cumulative undersize PSD were similar between both methods; however, the mechanical sieving curves lagged by 0.2-0.5 mm. The deviation was attributed to the falling-through effect of longer particles through sieve openings.

The objective of this study was to observe the potential of machine vision (MV) approach for the classification of eight citrus varieties. The leaf images of eight citrus varieties that were grapefruit, Moussami, Malta, Lemon, Kinow, Local lemon, Fuetrells, and Malta Shakri. These were acquired by a digital camera in an open environment without any complex laboratory setup. The acquired digital images dataset was transformed into the multi-feature dataset that was the combination of binary, histogram, texture, spectral, rotational, scalability and translational (RST) invariant features. For each citrus leaf image, total 57 multi-features were acquired on every non-overlapping region of interest (ROI), i.e. (32x32), (64x64), (128x128), and (256x256). Furthermore, the optimized 15 features using the supervised correlation-based feature selection (CFS) technique were acquired. The optimized multi-features dataset to different MV classifiers namely Multilayer Perceptron (MLP), Random Forest (RF), J48 and Naïve Bayes using10-fold cross-validation method were plugged-in. The results produced by MLP presented an average overall accuracy of 98.14% on ROIs (256x256) outperforming the other classifiers. The classification accuracy values by MLP on the eight citrus leaf varieties, namely; Grapefruit, Moussami, Malta, Lemon, Kinow, Local lemon, Fuetrells, and Malta Shakri were observed 98%, 98.75%, 99.25%, 97.5%, 97%, 95.87%, 95.5%, and 99.37% respectively.

Research and/or Engineering Questions/Objective The vehicle of the future will support its driver by advising him regarding potential hazards. Essential prerequisite therefore is the sensor based perception of the traffic situation. For the recognition of traffic related objects, camera based sensors, deepness cameras, vehicle sensors as well as radar and lidar sensors are used. For the future development of ADAS the fusion of multiple sensor data to a consistent environmental picture will play a key role. The evaluation approach of real world driving tests will no longer be sufficient due to the complexity of the system interactions. New simulation methods are needed to evaluate ADAS by using virtual test driving with realistic vehicle behavior and complex traffic environment. Methodology Therefore it is important to integrate camera based components in a “closed loop”-simulation platform to be able to test sensor data fusion technologies under realistic conditions. To test new driver assistance systems in a simulation environment today animation data is filmed, subsequently this data is used to test an image processing algorithm or a fusion algorithm. But this method cannot be applied if wide-angle cameras such as cameras with fisheye lenses will be used. Within a research frame work for autonomous driving functions a new simulation technology was developed to integrate virtual cameras beside the well know environment sensor in the vehicle dynamic simulation CarMaker. For this purpose the real-time animation was extended with a sophisticated virtual camera model so called “VideoDataStream” to generate simultaneous video data (also PMD for 3D images). The camera positions as well as the camera properties could be applied individually. Additionally it is possible to freely define the type of the camera lens (e.g. fisheye) with lens settings like opening angle and the typical lens failures (e.g. distortion and vignetting). With this new technology it is possible that e.g. camera and radar data can be provided time and place synchronal for the fusion algorithm which should be tested! Results The video data could be used for evaluating image processing and sensor data fusion in Model-/Software-/Hardware-in-the-Loop applications within virtual test driving conditions. Here the created method and examples of image based perception of the vehicle environment as well as sensor data fusion algorithms shall be presented. Among others this covers first of all the recognition of traffic lanes, traffic signs and other traffic partners as well as the fusion of the single information up to a comprehensive environment picture. A further field of application will be the conjunction with navigation systems and digital maps, by which the virtual vehicle supports the navigation system with related GPS position and gets back the “MPP—Most Probable Path” with the “electronic horizon”, which is a type of predictive sensor, with all related preview information in front of the vehicle which are defined in the ADASIS protocol. Conclusion By using the introduced method the capability and efficiency of function development and testing in the area of Advanced Driver Assistant Systems will significantly be improved. Due to a powerful simulation environment a broad range of validation tests can be shifted into simulation because also complex test scenarios can be replicated and the tests are reproducible. The simulation data can be provided time and place synchronal, which is absolutely important, e.g. for a fusion algorithm which should be tested.

The paper focuses on the development of GDF as a standard for digital road maps. GDF is a geographical data format in which the data are defined in three different levels: level O contains the geometry and the topology of the information; level 1 defines real world objects; and on level 2, functionally related level 1 features are grouped into higher order objects. The creation of the European digital road map, which was made possible by the standard and on the expected growth of mass market applications such as car information systems, is also discussed.

Location based services (LBS) are capturing global spotlights increasingly, in which users tend to pick up navigation service as their favorites. This paper examines the dynamic routing problems based on real-time traffic information in LBS. At first, the overall structure of dynamic navigation system is illustrated, which is composed of four ingredients: traffic data collection, digital road map, static routing and dynamic re-routing. Traffic data collection is the task of traffic information center (TIC), where various sources of data are fused into link travel times that are used as criteria to choose candidates in path determining. Digital road map includes digital road network and historical traffic data, where the topology relationship of road network is elementarily explored and the method to represent statistical traffic data is referred to. Routing with historical traffic data and dynamic re-routing with live traffic data are two main functions of dynamic navigation service. The flow of dynamic re-routing algorithm is exhibited. Lastly, a first-in-first-out (FIFO) time-depend shortest path algorithm is discussed for dynamic routing, which adopts the adaptive A^* algorithm based on binary heap priority queue and RB-tree.

125 years after Bertha Benz completed the first overland journey in automotive history, the Mercedes Benz S-Class S 500 INTELLIGENT DRIVE followed the same route from Mannheim to Pforzheim, Germany, in fully autonomous manner. The autonomous vehicle was equipped with close-to-production sensor hardware and relied solely on vision and radar sensors in combination with accurate digital maps to obtain a comprehensive understanding of complex traffic situations. The historic Bertha Benz Memorial Route is particularly challenging for autonomous driving. The course taken by the autonomous vehicle had a length of 103 km and covered rural roads, 23 small villages and major cities (e.g. downtown Mannheim and Heidelberg). The route posed a large variety of difficult traffic scenarios including intersections with and without traffic lights, roundabouts, and narrow passages with oncoming traffic. This paper gives an overview of the autonomous vehicle and presents details on vision and radar-based perception, digital road maps and video-based self-localization, as well as motion planning in complex urban scenarios.

Image sequences recorded with cameras mounted in a moving vehicle provide information about the vehicle's environment which has to be analysed in order to really support the driver in actual traffic situations. One type of information is the lane structure surrounding a vehicle. Therefore, we systematically developed and investigated driver assistance functions which make explicit use of the lane structure represented by lane borders and lane markings. With increasing computing power of standard PCs it was possible to realize more complex driver assistance with general purpose hardware. Investigations with a video-based lane departure warning system and a lane change assistant for highways will be discussed in detail. We integrated our lane keeping assistant in some experimental cars and performed systematic experiments in real traffic situations which enables the experience of video-based driver assistance on a high level—the action of a system. This allows us to assess whether a driver assistance system really understands the actual traffic situation which is the basis for reliable systems accepted by the user.

Advanced Driver Assistance Systems (ADAS) and Collision Avoidance Systems (CAS) are the primary modules of modern human-centric and autonomous driving applications, such as forward and rear-end collision warnings. To enhance the performance of ADAS and CAS systems in foggy weather, an Efficient and Cost-Effective Vehicle Detection and Tracking with Driver Assistance (ECE-VDTDA) system is proposed. The proposed ECE-VDTDA system comprises vehicle detection, tracking, and driver assistance modules. An optimized SimYOLO-V5s_WIOU vehicle detection algorithm is proposed, based on the SimSPPF module, the baseline You Only Look Once (YOLO) algorithm (YOLO-V5s), and the Wise Intersection Over Union (WIOU) localization loss function. State-of-the-art Deep-SORT, Strong-SORT, and optimized Deep-SORT algorithms are utilized for vehicle tracking. The vehicle detection and tracking performance of the ECE-VDTDA system is rigorously evaluated on DAWN, foggy driving, foggy cityscapes, BDD100K, web-collected, and self-collected foggy weather datasets. Optimized SimYOLO-V5s_WIOU algorithm outperformed on the foggy driving dataset with a 17.45% increase in mAP50, and foggy cityscapes dataset with a 0.32%, 1.05%, 1.58%, 2%, 0.54% increase in the multiclass mAP50, mAP50-95, F1 score, precision, and recall scores, respectively, compared to the baseline YOLO-V5s. Furthermore, the SimYOLO-V5s_WIOU algorithm also outperformed the state-of-the-art methods and enables Deep-SORT, Strong-SORT, and optimized Deep-SORT vehicle tracking algorithms to track vehicles with high confidence. The driver assistance module of the ECE-VDTDA system helps prevent imminent road collisions in foggy weather by estimating distance, speed, and time-to-collision and by issuing timely collision warnings. The experimental results demonstrate the robustness and computational efficiency of the proposed ECE-VDTDA system.

Various kinds of level 2 driver assistance systems have been commercialized in the market to reduce traffic fatalities. However, there is a concern that if drivers ware assisted by such systems for a long of time, their level of alertness would tend to decline. To resolve these kinds of issues, most existing research has focused on preventing excessive dependence on driving assistance and arousal reduction caused by the assistance. In this paper, we focus on the driver excitement in the level 2 automated driving (advanced driving support) and propose a method for estimating the excitement using the SD (semantic differential) method. We also show that there is a clear correlation between excitement and arousal levels during driving with a level 2 driver assistance system. We evaluate the effectiveness of our proposal through a driving simulator-based experiment on the Tokyo Metropolitan Expressway.

Using multisensory signals in advanced driver assistance is continuously increasing as a way to increase the attention and reduce the reaction time. It is essential that driver assistance system is capable of providing directional cues to the driver to direct his attention to the sides of the car as usually the focus is in front of the car. These signals could be for blind spot information, navigation, lane departure warning, collision warning, etc. This study investigated the effect of auditory and vibrotactile on directional attention in driver assistance systems. Moreover, two types of immersive displays were used in the driving simulation, namely the Head Mounted Display (HMD) and CAVE display, to study the effect of the type of display on the human performance. Lane Chang Task was used to assess the attention by measuring the response time to directional information. Vibrotactile and Auditory cues induced equal response times, meanwhile, vibrotactile signal was significantly gained higher satisfaction than auditory.