Data-driven visual model development and 3D visual analytics framework for underground mining

Three-dimensional (3D) real-time volume rendering has demonstrated improvements in clinical care for several areas of radiological imaging. We test whether advanced real-time rendering techniques combined with an effective user interface will allow radiologists and surgeons to improve their performance for cardiothoracic surgery planning and diagnostic evaluation. An interactive combination 3D and 2D visualization system developed at the University of North Carolina at Chapel Hill was compared against standard tiled 2D slice presentation on a viewbox. The system was evaluated for 23 complex cardiothoracic computed tomographic (CT) cases including heart-lung and lung transplantation, tumor resection, airway stent placement, repair of congenital heart defects, aortic aneurysm repair, and resection of pulmonary arteriovenous malformation. Radiologists and surgeons recorded their impressions with and without the use of the interactive visualization system. The cardiothoracic surgeons reported positive benefits to using the 3D visualizations. The addition of the 3D visualization changed the surgical plan (65% of cases), increased the surgeon's confidence (on average 40% per case), and correlated well with the anatomy found at surgery (95% of cases). The radiologists reported fewer and less major changes than the surgeons in their understanding of the case due to the 3D visualization. They found new findings or additional information about existing findings in 66% of the cases; however, they changed their radiology report in only 14% of the cases. With the appropriate choice of 3D real-time volume rendering and a well-designed user interface, both surgeons and radiologists benefit from viewing an interactive 3D visualization in addition to 2D images for surgery planning and diagnostic evaluation of complex cardiothoracic cases. This study finds that 3D visualization is especially helpful to the surgeon in understanding the case, and in communicating and planning the surgery. These results suggest that including real-time 3D visualization would be of clinical benefit for complex cardiothoracic CT cases.

Construction equipment generally consists of a set of rigid bodies connected by joints, which is termed as a kinematic chain. The relative motion and constraints between successive bodies of the chain make the real-time 3D visualization a challenge, which requires minimizing the number of sensors as needed by practical applications in construction. In this paper, we investigate a methodology for efficient real-time 3D visualization of articulated construction equipment. The Denavit-Hartenberg (DH) technique which has been widely used in robotics research is introduced and adapted for computing relative motions of various components in the articulated equipment system based on a minimal quantity of input parameters. The 3D position of articulated construction equipment can be analytically fixed by analyzing point coordinates and joint movement parameters. This analytical method is validated by simulating a backhoe excavator in a computer environment, where the absolute location of the backhoe’ tracks is computed by tracking three control points, and the relative positioning states of the cabin, boom, stick and bucket are deduced by using the DH technique.

This project uses Arduino Uno and Unity 3D to create a digital twin with IoT capabilities for real-time battery monitoring and lifespan prediction. The system monitors battery metrics including voltage, current, temperature, State of Charge (SoC), and State of Health (SoH) using sensors and an Arduino microcontroller. The obtained data is transmitted to a cloud-based IoT platform via a Wi-Fi module (ESP8266), allowing for real-time remote access. Unity 3D's digital twin simulates the behavior of a real battery, providing interactive display of important parameters and performance trends. Coulomb counting is used to compute SoC, whereas watching the loss of battery capacity over time determines SoH. Predictive analytics are used in the system to predict the battery's remaining useful life (RUL). When performance thresholds are met, alerts and notifications are sent via the IoT dashboard and Unity interface. The digital twin enhances monitoring and enables predictive maintenance, resulting in optimal utilization and preventing unexpected problems. This concept has several applications, including electric cars, renewable energy storage systems, and consumer electronics. This system offers real-time IoT connectivity, 3D visualization, and predictive modeling, making it a powerful tool for battery management, analytics, and health tracking

As the volume of network data continues to increase and networks become more complex, the ability to accurately manage and analyze data quickly becomes a difficult problem. Many network management tools already use two-dimensional (2D) and three-dimensional (3D) visualization techniques to help support decision-making and reasoning of network anomalies and activity. However, a poor user interface combined with the massive amount of data could obfuscate important network details. As a result, administrators may fail to detect and identify malicious network behavior in a timely manner. 3D visualizations address this challenge by introducing monocular and binocular visual cues to portray depth and to increase the perceived viewing area. In this work, we explore these cues for 3D network security applications, with a particular emphasis on binocular disparity or stereoscopic 3D. Currently, no network security tool takes advantage of the enhanced depth perception provided by stereoscopic 3D technologies for vulnerability assessment. Compared to traditional 3D systems, stereoscopic 3D helps improve the perception of depth, which can, in turn reduce the number of errors and increase response times of network administrators. Thus, we introduce a stereoscopic 3D visual Framework for Rendering Enhanced 3D Stereoscopic Visualizations for Network Security (FRE3DS). Our novel framework uses state-of-the art 3D graphics rendering to assist in 3D visualizations for network security applications. Moreover, utilizing our framework, we propose a new 3D Stereoscopic Vulnerability Assessment Tool (3DSVAT). We illustrate the use of 3DSVAT to assist in rapid detection and correlation of attack vulnerabilities in a subset of a modified local area network data set using the enhanced perception of depth in a stereoscopic 3D environment.

In underground mining, hydraulic backfill materials, such as waste tailings, river sand, and cement, are often used to fill underground mined stopes.In cut-and-fill mining methods with blasthole stoping and delayed backfill, after extraction of adjacent pillars that contain economic minerals, the backfill is often subject to exposure of free standing on at least one side.A key concern for mining engineers is the stability of this immediately-bordered backfill body, because the backfill stability has a significant effect on the dilution/loss rate and the safety of mining operations.It is found that backfill stability is one of the mining subjects most dominated by uncertainty.Rock and backfill properties, environmental conditions, and analytical models are such factors contributing to uncertainty.Conventional methods simplified the problem by considering the uncertain parameters to be deterministic, and accounted for the uncertainties through the use of empirical factors of safety.This paper aims to conduct stability analysis of backfill in underground mining using a probabilistic reliability method, which is an extension of conventional deterministic methods.The parameters of backfill properties are modelled as random variables.In order to determine the failure probability of the backfill in a cut-and-fill mining method of an underground mine in China, a three-dimensional wedge model is set up for the backfill and a corresponding limit state function is established to characterize the backfill stability for the purpose of reliability analysis.The influences of the mean values, coefficients of variation, probability distribution types, and correlation between random variables are carefully investigated through sensitivity analysis.The results obtained give insights into the mechanism of backfill stability and could provide some useful clues on how to choose the right backfill materials.

The goal of the Image Guidance Laboratories (IGL) is to provide the highest quality, real-time visualization of 3D ultrasound images in a manner that is most acceptable and suited for adoption by surgeons. To this end, IGL has developed an optically tracked, freehand, 3D volume- rendering ultrasound system for image-guided surgery. Other systems temporally separate frame acquisition from volume construction and display; the data must be stored before being loaded into the volume construction engine for visualization. By incorporating novel methods to reduce the computational expense associated with frame insertion, volume maintenance, and 2D texture-based rendering, the IGL system is able to simultaneously acquire and display 3D ultrasound data. The work presented here focuses on methods unique to achieving near real-time 3D visualization using 2D ultrasound images and discusses the potential of this system to address clinical situations such as liver resection, tumor ablation, and breast biopsy.

SWAV, which stands for Simulated Wave Visualizer, is a novel real-time 3D surf zone animation and visualization application under development at Templeman Automation, LLC. SWAV visualizes many types of breaking waves in real-time, including true plunging waves, integrated into a realistic 3D environment.

Real-time telemetry-based, 3D situational-awareness visualizations for deep space and earth-orbiting missions have proven to be vital during all mission phases from project formulation, mission development, design, verification, through post-launch operations. These visualizations play key roles in sending flight system health and status to engineers and communicating mission goals to stakeholders and the public. This real-time telemetry-based visualization technology has been used for many previous NASA/JPL missions including Mars 2020, Mars Science Laboratory (MSL), Low-Density Supersonic Decelerator (LDSD), NASA's Soil Moisture Active Passive (SMAP), Galileo, and many NASA/JPL robotic systems research and development projects. Despite the number of missions that use these visualizations to carry out mission operations, there currently exists no multi-project environment. The current development model of building new and unique telemetry visualization software systems for each new mission is extremely cost-ineffective. In this paper, we, from JPL's Mission Control Systems Section, introduce Theia, our multi-project space mission telemetry-based visualization system which we offer as a low-cost, easily configurable, multi-project system inspired by our recent work on the Entry, Descent, and Landing (EDL) sequence for the Mars 2020 Perseverance Rover mission. The Theia telemetry visualization system is an end-to-end solution that listens to a real-time telemetry or simulation data source, processes, packages, and streams that data through secure WebSockets to multiple simultaneously connected web browser clients executing our TheiaJS 3D visualization engine. TheiaJS presents the data to the user through an interactive 3D interface, extendable as required by individual missions. We describe Theia's complete system design and architecture, the construction and rendering of a flight-system Digital Twin, and the data pipeline needed to accurately and continuously update the Digital Twin and its environment. We illustrate Theia's use using historical data from two previous missions: Mars Science Laboratory (MSL) and Mars Perseverance Rover 2020, as well as data from future missions such as NISAR and Psyche. In addition, we discuss challenges, system verification, validation, and integration with predictive models and analysis tools. We further explore Theia as the core of a fully integrated real-time flight mission analysis and visualization environment.

The main contribution of this paper is to introduce a framework for integrating Machine Learning (ML), Human, and Virtual Reality (VR) into one platform to promote a collaborative visualisation environment that can assist in better analysis and improve the human-machine teaming capability. This platform was demonstrated using a case study in ana-lysing road traffic conditions. The ‘Ab-normal Machine Learning Road Traffic Detection in VR (AbnMLRTD-VR)’ prototype system was developed to assist the human analyst. The proposed system has two main integrative components: a data-driven ML model and a 3D real-time visualisation in a VR environment. An unsupervised ML model was built using real traffic data. The AbnMLRTD-VR system highlights the outliers in the road sections in actual road contexts of a road traffic network. This gives the human analyst a 3D real-time immersive visualisation in a VR environment to evaluate road conditions. The AbnMLRTD-VR system demonstrated that it could help minimise the need for human pre-labelling of the data. It enables the visualisation of the road traffic conditions more meaningfully and to understand the context of the road traffic conditions of road sections at any given time.

Instant 3D design concept generation and visualization by real-time hand gesture recognition

To determine the teaching effects of a real-time three dimensional (3D) visualization system in the operating room for early-stage phacoemulsification training. A total of 10 ophthalmology residents of the first-year postgraduate were included. All the residents were novices to cataract surgery. Real-time cataract surgical observations were performed using a custom-built 3D visualization system. The training lasted 4wk (32h) in all. A modified International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubric (ICO-OSCAR) containing 4 specific steps of cataract surgery was applied. The self-assessment (self) and expert-assessment (expert) were performed through the microsurgical attempts in the wet lab for each participant. Compared with pre-training assessments (self 3.2±0.8, expert 2.5±0.6), the overall mean scores of post-training (self 5.2±0.4, expert 4.7±0.6) were significantly improved after real-time observation training of 3D visualization system (P<0.05). Scores of 4 surgical items were significantly improved both self and expert assessment after training (P<0.05). The 3D observation training provides novice ophthalmic residents with a better understanding of intraocular microsurgical techniques. It is a useful tool to improve teaching efficiency of surgical education.

Electrical capacitance tomography (ECT) has been developed to provide 2D images, representing the cross-sectional permittivity distribution of industrial processes, such as oil pipelines and pharmaceutical fluidised beds. In some cases, 3D visualisation is more attractive than 2D. While some researchers worked on 3D ECT, the imaging speed was very slow, far away from real time. This paper introduces an approach to real-time 3D visualization in ECT, which can generate 3D images at almost the same speed as the traditional 2D images.

Ensuring safety in the mining industry is a critical concern for a nation's industrial advancement. Industry 4.0, characterized by the integration of advanced technologies, is at the forefront of efforts to enhance mining practices. Coal seams contain a range of hydrocarbon gases, predominantly methane, which is released in significant quantities during mining operations. Effectively mitigating methane emissions is imperative. The inclusion of methane forecasting allows for the early identification of potential methane emissions, hence resulting in significance enhancement in mine safety. The research work is focused on real-time remote monitoring and cloud-based forecasting of methane levels in underground coal mines. An Industrial Internet of Things (IIoT) device is developed for data acquisition in underground coal mines, capturing essential parameters such as methane concentration, temperature, and humidity. The collected data are utilized to train a long short-term memory based multivariate forecasting model. The trained model is subsequently deployed in the cloud. The experiment is performed in a mine of Eastern Coalfields Limited, India. After the deployment of the proposed model, the developed IIoT device transmits real-time data, obtained from the mine, to the cloud. Based on the real-time data, our model conducts methane forecasting and communicates results back to the IIoT device. The device issues immediate alerts when methane levels surpass predefined thresholds. This ensures enhanced safety in mining operations by providing warnings for both current and forecasted methane concentrations. The forecasted methane concentrations, along with real-time data, are accessible through mobile applications and a web-based dashboard. The accuracy of the proposed model is measured by mean absolute error, mean absolute percentage error, and root mean square error, which demonstrate values of 156.95 ppm, 4.23%, and 191.53 ppm, respectively. A comparative study is performed where our model is evaluated against the multivariate multilayer perceptron, vector autoregression, and auto-regressive integrated moving average models. The comparative study demonstrates that our developed model outperforms the others, showing superior results.

Skill testing is a key component of sports science, yet research on golf performance assessment remains limited. This study aims to systematically review existing methods for evaluating golf performance and to visually present their design principles and indicator systems using a 3D bubble chart, ultimately proposing an optimized and standardized testing framework for golf skills. Following the PRISMA guidelines, this study conducted a systematic search of Web of Science, PubMed, SPORTDiscus, Scopus, and Google Scholar, ultimately including 13 studies for analysis. Combined with 3D visualization indicate that existing research is overly concentrated on driving and putting, while iron and chipping tests are markedly underrepresented, limiting the comprehensive understanding of the four core golf skills. The number of shots and selection of measurement variables lack standardization. Although complex biomechanical and club-ball dynamics indicators can help explain performance mechanisms, overemphasis on these measures often overlooks the core objective of golf-getting the ball into the hole. This study proposes a standardized modular testing framework centered on "target-hitting." Consists of four modules: ball-striking zone (e.g., grass, bunkers, slopes), target zone (e.g., fairway, green, and varying sizes), shot distance, and club selection. By adjusting the combination of modules, it can simulate complex scenarios and accommodate athletes of different skill levels, providing a scientific reference for research, training, and performance evaluation. This study also provides guidance on the test modules and test site setup.

Museums and cultural institutions, in general, are in a constant challenge of adding more value to their collections. The attractiveness of assets is practically tightly related to their value obeying the offer and demand law. New digital visualization technologies are found to give more excitements, especially to the younger generation as it is proven by multiple studies. Nowadays, museums around the world are currently trying to promote their collections through new multimedia and digital technologies such as 3D modeling, virtual reality (VR), augmented reality (AR), and serious games. However, the difficulty and the resources required to implement such technologies present a real challenge. Through this paper, we propose a 3D acquisition and visualization framework aiming mostly at increasing the value of cultural collections. This framework preserves cost-effectiveness and time constraints while still introducing new ways of visualization and interaction with high-quality 3D models of cultural objects.