Enhancing point cloud analysis with multi-scale state space modeling and edge graph augmentation

Previous studies on Sound Event Localization and Detection (SELD) have primarily focused on CNN- and Transformer-based designs. While CNNs possess local receptive fields, making it difficult to capture global dependencies over long sequences, Transformers excel at modeling long-range dependencies but have limited sensitivity to local time–frequency features. Recently, the VMamba architecture, built upon the Visual State Space (VSS) model, has shown great promise in handling long sequences, yet it remains limited in modeling local spatial details. To address this issue, we propose a novel state space model with an attention-enhanced feature fusion mechanism, termed FFMamba, which balances both local spatial modeling and long-range dependency capture. At a fine-grained level, we design two key modules: the Multi-Scale Fusion Visual State Space (MSFVSS) module and the Wavelet Transform-Enhanced Downsampling (WTED) module. Specifically, the MSFVSS module integrates a Multi-Scale Fusion (MSF) component into the VSS framework, enhancing its ability to capture both long-range temporal dependencies and detailed local spatial information. Meanwhile, the WTED module employs a dual-branch design to fuse spatial and frequency domain features, improving the richness of feature representations. Comparative experiments were conducted on the DCASE2021 Task 3 and DCASE2022 Task 3 datasets. The results demonstrate that the proposed FFMamba model outperforms recent approaches in capturing long-range temporal dependencies and effectively integrating multi-scale audio features. In addition, ablation studies confirmed the effectiveness of the MSFVSS and WTED modules.

In order to solve the problem of low efficiency of image feature matching in traditional remote sensing image database, this paper proposes the feature matching optimization of multimedia remote sensing images based on multiscale edge extraction, expounds the basic theory of multiscale edge, and then registers multimedia remote sensing images based on the selection of optimal control points. In this paper, 100 remote sensing images with a size of 3619∗825 with a resolution of 30 m are selected as experimental data. The computer is configured with 2.9 ghz CPU, 16 g memory, and i7 processor. The research mainly includes two parts: image matching efficiency analysis of multiscale model; matching accuracy analysis of multiscale model and formulation of model parameters. The results show that when the amount of image data is large, feature matching takes more time. With the increase of sampling rate, the amount of image data decreases rapidly, and the feature matching time also shortens rapidly, which provides a theoretical basis for the multiscale model to improve the matching efficiency. The data size is the same, 3619 × 1825, which makes the matching time between images have little difference. Therefore, the matching time increases linearly with the increase of the number of images in the database. When the amount of image data in the database is large, a higher number of layers should be used; when the amount of image data in the database is small, the number of layers of the model should be reduced to ensure the accuracy of matching. The availability of the proposed method is proved.

The concept of non-Archimedean multiscale space and models of plastic media with structure

Multi-Scale State Space Modeling for Cross-Modal Infrared and Visible Image Fusion

In this paper, a new high-order multiscale finite element method (MsFEM) is developed for elliptic problems with highly oscillating coefficients. The method is inspired by the MsFEM developed in [G. Allaire and R. Brizzi, Multiscale Model. Simul., 4 (2005), pp. 790--812], but a more explicit multiscale finite element space is constructed. The approximation space is nonconforming when an oversampling technique is used. We use a Petrov--Galerkin formulation suggested in [T. Y. Hou, X.-H. Wu, and Y. Zhang, Commun. Math. Sci., 2 (2004), pp. 185--205] to simplify the implementation and to improve the accuracy. The method is natural for high-order finite element methods used with the advantage of solving the coarse grained problem. We prove optimal error estimates in the case of periodically oscillating coefficients and support the findings by various numerical experiments.

In order to support the functional design and simulation and the final fabrication processes for functional surfaces, it is necessary to obtain a multi-scale modelling approach representing both macro geometry and micro details of the surface in one unified model. Based on the fractal geometry theory, a synthesized model is proposed by mathematically combining Weierstrass-Mandelbrot fractal function in micro space and freeform CAGD model in macro space. Key issues of the synthesis, such as algorithms for fractal interpolation of freeform profiles, and visualization optimization for fractal details, are addressed. A prototype of the integration solution is developed based on the platform of AutoCAD’s Object ARX, and a few multi-scale modelling examples are used as case studies. With the consistent mathematic model, multi-scale surface geometries can be represented precisely. Moreover, the visualization result of the functional surfaces shows that the visualization optimization strategies developed are efficient.

It is important for product designer and material designer to forecast the performance and nature of the designing product by computer simulation in the early stage of the design process. The computational model of multi-scale space is the core part of the multi-scale product design system. Phase field method (PFM) is able to simulate the phenomena of phase transition of material. A space change propagates in the computational model of multi-scale space. PFM is applied into the computational model of multi-scale space. A method to simulate the propagation of phase transition in multi-scale space is proposed.

Rotating machinery plays a critical role in industrial systems, making timely fault detection essential for operational safety and economic efficiency. Traditional methods relying on manual feature extraction and expert knowledge are often insufficient for handling complex and variable fault conditions. While supervised and semi-supervised learning approaches partially address these limitations, they remain constrained by reliance on labeled data, poor identification of novel faults, and inadequate modeling of multi-feature interactions and temporal dependencies. To overcome these challenges, this paper proposes a self-supervised fault detection method termed KAN-Multi-Scale Convolutional State Space Model (KMSC-SSM), eliminating label dependency through signal reconstruction. Specifically, KMSC-SSM leverages a Kolmogorov–Arnold Network (KAN) to enhance nonlinear feature representation, employs a Multi-scale Convolutional Bidirectional State Space Module (MCB-BiSSM) to effectively capture spatiotemporal dynamics of vibration signals, and integrates Reversible Instance Normalization to mitigate signal non-stationarity. Moreover, a dynamic weighting mechanism optimizes module collaboration, further enhancing adaptability to complex operational environments. Experimental validations conducted on publicly available datasets demonstrate that the proposed KMSC-SSM achieves superior performance, obtaining high F1-scores without requiring labeled fault data, thus confirming its effectiveness and robustness in practical industrial applications.

The thesis is going to focus on point clouds analysis, especially on 3D object classification and part segmentation of point clouds. Point clouds analysis of 3D models and 3D scenes attracts extensive research attention recently, not only due to the availability of several large repositories but also because of its wide applications. Point clouds are commonly used in industry, e.g. autonomous vehicles and robotics use LIDAR to collect point clouds of scenes, which works as an auxiliary recognition tool of 3D space. Therefore, proposing realiable and feasible algorithms for point clouds recognition and analysis is significant and benefitial.

A new multiscale, ensemble-based data assimilation (DA) method, multiscale local gain form ensemble transform Kalman filter (MLGETKF), is introduced. MLGETKF allows simultaneous update of multiple scales for both the ensemble mean and perturbations through assimilating all observations at once. MLGETKF performs DA in independent local volumes, which lends the algorithm a high degree of computational scalability. The multiscale analysis is enabled through the rapid creation of many pseudoensemble perturbations via a multiscale ensemble modulation procedure. The Kalman gain that is used to update the raw background ensemble mean and perturbations is based on this modulated ensemble, which intrinsically includes multiscale model space localization. Experiments with a noncycled statistical model show that the full background covariance estimated by MLGETKF more accurately resembles the shape of the true covariance than a scale-unaware localization. The mean analysis from the best-performing MLGETKF is statistically significantly more accurate than the best-performing scale-unaware LGETKF. The accuracy of the MLGETKF analysis is more sensitive to small-scale band localization radius than large-scale band. MLGETKF is further examined in a cycling DA context with a surface quasigeostrophic model. The root-mean-square potential temperature analysis error of the best-performing MLGETKF is 17.2% lower than that of the best-performing LGETKF. MLGETKF reduces analysis errors measured in kinetic energy spectra space by 30%–80% relative to LGETKF with the largest improvement at large scales. MLGETKF deterministic and ensemble mean forecasts are more accurate than LGETKF for full and large scales up to 5–6-day lead time and for small scales up to 3–4-day lead time, gaining ~12 h–1 day of predictability.

Conformational dynamics of proteins can be interpreted as itinerant motions as the protein traverses from one state to another on a complex network in conformational space or, more generally, in state space. Here we present a scheme to extract a multiscale state space network (SSN) from a single-molecule time series. Analysis by this method enables us to lift degeneracy--different physical states having the same value for a measured observable--as much as possible. A state or node in the network is defined not by the value of the observable at each time but by a set of subsequences of the observable over time. The length of the subsequence can tell us the extent to which the memory of the system is able to predict the next state. As an illustration, we investigate the conformational fluctuation dynamics probed by single-molecule electron transfer (ET), detected on a photon-by-photon basis. We show that the topographical features of the SSNs depend on the time scale of observation; the longer the time scale, the simpler the underlying SSN becomes, leading to a transition of the dynamics from anomalous diffusion to normal Brownian diffusion.

A comparison between neural network and traditional approaches to nonlinear system identification is investigated with respect to aspects of model performance. Two neural network models, the state space and input-output model structures, are considered. A global recurrent RMLP and a teacher forcing RMLP are categorized as the state space models, and a global feedback FMLP and a teacher forcing FMLP are considered as the input-output models. In the traditional methods an AutoRegressive eXogeneous (ARX) Input model and a Nonlinear AutoRegressive eXogeneous (NARX) Input model are considered. Basic algorithms of models are described, and simulation results are also presented through the system output response. Performance of models is compared based on the Mean-Square-Errors (MSE). Noise-added sinusoidal, pulse and step signals are chosen as the test inputs for the validation of the obtained models. Two different noise levels are augmented to the chosen input signals.

The use of virtual reality technologies for data visualization and analysis has been an emerging topic of research in the past years. However, one type of data has been left neglected, the point cloud. While some strides have been made in the visualization and analysis of point clouds in immersive environments, these have yet to be used for direct manipulation interactions. It is hypothesized that as with other types of data, bringing direct interactions and 3D visualization to point clouds may increase the ease of performing basic handling tasks. An immersive application for virtual reality HMDs was developed in Unity to help research this hypothesis. It is capable of parsing classified point cloud files with extracted objects and representing them in a virtual environment. Several editing tools were also developed, designed with the HMD controllers in mind. The end result allows the user to perform basic transformative tasks to the point cloud with an ease of use and intuitive feeling unmatched by the traditional desktop-based tools.

Modern remote sensing technologies have enabled the creation of high-resolution 3D point clouds of infrastructure systems. In particular, photogrammetric reconstructions using Dense-Structure-from-Motion algorithm can now yield point clouds with the necessary resolution to capture small-strain displacements. By tracking changes in these point clouds over time, displacements can be measured, leading to strain and stress estimates for long-term structural evaluations. This study determines the accuracy of a comparative point cloud analysis technique for measuring deflections in high-resolution point clouds of structural elements. Utilizing a combination of a recently developed point cloud generation process and localized nearest-neighbors cloud comparisons, the analytical technique is designed for long-term field scenarios and requires no artificial tracking, targets, and camera calibrations. A series of flexural laboratory experiments were performed in order to test the approach. The results indicate sub-millimeter accuracy in measuring the vertical deflection, making it suitable for the small-displacement analysis of a variety of large-scale infrastructure systems. Ongoing work seeks to extend this technique for comparison with as-built and finite element models.

The study of strain effects in thermally-forced rock masses has gathered growing interest from engineering geology researchers in the last decade. In this framework, digital photogrammetry and infrared thermography have become two of the most exploited remote surveying techniques in engineering geology applications because they can provide useful information concerning geomechanical and thermal conditions of these complex natural systems where the mechanical role of joints cannot be neglected. In this paper, a methodology is proposed for generating point clouds of rock masses prone to failure, combining the high geometric accuracy of RGB optical images and the thermal information derived by infrared thermography surveys. Multiple 3D thermal point clouds and a high-resolution RGB point cloud were separately generated and co-registered by acquiring thermograms at different times of the day and in different seasons using commercial software for Structure from Motion and point cloud analysis. Temperature attributes of thermal point clouds were merged with the reference high-resolution optical point cloud to obtain a composite 3D model storing accurate geometric information and multitemporal surface temperature distributions. The quality of merged point clouds was evaluated by comparing temperature distributions derived by 2D thermograms and 3D thermal models, with a view to estimating their accuracy in describing surface thermal fields. Moreover, a preliminary attempt was made to test the feasibility of this approach in investigating the thermal behavior of complex natural systems such as jointed rock masses by analyzing the spatial distribution and temporal evolution of surface temperature ranges under different climatic conditions. The obtained results show that despite the low resolution of the IR sensor, the geometric accuracy and the correspondence between 2D and 3D temperature measurements are high enough to consider 3D thermal point clouds suitable to describe surface temperature distributions and adequate for monitoring purposes of jointed rock mass.