MLGCN: an ultra efficient graph convolutional neural model for 3D point cloud analysis.

Unlike 2-dimensional (2D) images, direct 3-dimensional (3D) point cloud processing using deep neural network architectures is challenging, mainly due to the lack of explicit neighbor relationships. Many researchers attempt to remedy this by performing an additional voxelization preprocessing step. However, this adds additional computational overhead and introduces quantization error issues, limiting an accurate estimate of the underlying structure of objects that appear in the scene. To this end, in this article, we propose a deep network that can directly consume raw unstructured point clouds to perform object classification and part segmentation. In particular, a Deep Feature Transformation Network (DFT-Net) has been proposed, consisting of a cascading combination of edge convolutions and a feature transformation layer that captures the local geometric features by preserving neighborhood relationships among the points. The proposed network builds a graph in which the edges are dynamically and independently calculated on each layer. To achieve object classification and part segmentation, we ensure point order invariance while conducting network training simultaneously—the evaluation of the proposed network has been carried out on two standard benchmark datasets for object classification and part segmentation. The results were comparable to or better than existing state-of-the-art methodologies. The overall score obtained using the proposed DFT-Net is significantly improved compared to the state-of-the-art methods with the ModelNet40 dataset for object categorization.

With the increasing use of commodity RGB-D cameras for computer vision, robotics, mixed and augmented reality and other areas, it is of significant practical interest to calibrate the relative pose between a depth (D) camera and an RGB camera in these typ

3D model classification has many applications in CAD, 3D object retrieval, and so on. The description of 3D model is crucial but difficult, which leads to the difficulty of classification. The traditional classifier has its limitation in classification of 3D model description. In this paper, we present 3D model classification-based nonparametric discriminant analysis with kernels combined with geometry projection-based histogram model for invariable feature extraction. Firstly, we present nonparametric discriminant analysis with kernels, and secondly, we proposed the invariable feature extraction method with geometry projection-based histogram model. Thirdly, we present the framework of 3D model classification using the proposed nonparametric discriminant analysis with kernels and geometry projection-based histogram model. Finally, we testify the feasibility of the proposed algorithm and performance on 3D model classification. The experimental results show that the proposed scheme is feasible and effective on 3D model classification on the public datasets.

The development and structuring of an augmented reality (AR) platform integrating image processing, deep learning, and spatial tracking techniques have yielded promising results. AR technology, which enhances real-world objects with computer-generated information, holds vast potential for education, entertainment, and industry. Image processing enables real-time object detection, recognition, and tracking within the user's environment, while deep learning models enhance pattern recognition accuracy. Spatial tracking techniques ensure seamless integration of virtual content with the physical world, providing immersive AR experiences. Ant colony optimization (ACO) algorithms further enhance the AR platform's functionality by optimizing object placement and interaction. ACO mimics ant behavior to find optimal paths through graphs, effectively optimizing spatial layout and user interaction patterns in dynamic environments. The integration of ACO into the AR platform facilitates efficient pathfinding for augmented objects and enhances the system's responsiveness to user inputs. The successful implementation of the AR platform underscores its potential to revolutionize various industries, including healthcare, education, and marketing. By leveraging AI-driven algorithms and optimization techniques, AR systems can deliver personalized and engaging experiences that blur the boundaries between the digital and physical worlds. These results pave the way for future advancements in AR technology, driving innovation and reshaping human-computer interaction paradigms.

Traffic forecasting is a crucial aspect of modern Intelligent Transportation Systems (ITS) and the Internet of Vehicles (IoV), playing a vital role in improving the safety and efficiency of daily transportation activities. Despite the valuable contributions of traditional machine learning (ML) models and advanced deep learning (DL) techniques, there persist challenges in capturing the intricate spatial and temporal dependencies inherent in traffic flow. In response to these challenges, we present GRAPHITE, an innovative framework that combines Graph Neural Networks (GNNs) and Generative Adversarial Networks (GANs) to leverage generative reasoning for efficient traffic management. Our model seamlessly integrates historical traffic volume data collected by road sensors with local spatial information encoded through knowledge graphs (KGs) associated with each sensor. These KGs offer a structured representation of relationships between traffic sensors and points of interest (POIs) in their neighborhood, thereby enhancing the comprehension of the urban context and leading to more accurate traffic predictions. Extensive experiments conducted on diverse datasets underscore the efficacy of GRAPHITE. Notably, we achieved a maximum decrease in RMSE of 31.05% compared to GAN-GRU and a maximum increase in R2 of 8.15% compared to GAN-RNN, positioning GRAPHITE as a standout solution among the current state-of-the-art approaches. Our code is available at: https://github.com/MODAL-UNINA/GRAPHITE.

This paper presents a new Intelligent Anti-Money Laundering Transaction Pattern Recognition System based on Graph Neural Networks (GNNs). The proposed method addresses the limitations of anti-money laundering (AML) by using graphical representation and deep learning techniques. We introduce general methods for creating financial markets that are integrated according to different shapes, including structural and physical. A custom GNN architecture is developed, featuring heterogeneous graph convolution, listening mechanisms, and physical models to capture the exchange patterns. The analysis of the world’s big data shows that the best performance of our method achieved a 35.2% Money Laundering Detection Rate (MLDR) in the top 1% of the flag business, do the best way.

Recommender systems act as filtering algorithms to provide users with items that might meet their interests according to the expressed preferences and items' characteristics. As of today, the collaborative filtering paradigm, along with deep learning techniques to learn high-quality users' and items' representations, constitute the de facto standard for personalized recommendation, showing remarkable recommendation accuracy performance. Nevertheless, recommendation remains a highly-challenging task. Among the most debated open issues in the community, this thesis considers two algorithmic and conceptual ones, namely: (i) the inexplicable nature of users' preferences, especially when they come in the form of implicit feedback; (ii) the effective exploitation of the collaborative information in the designing and training of recommendation models. In domains such as fashion, food, and media content recommendation, the shallow item's profile can be enhanced through the multimodal characteristics describing items [Malitesta et al., 2023]. Driven by these assumptions, in the first part of this thesis, we apply multimodal deep learning strategies for multimedia recommendation; the scope is to study and design recommendation algorithms based upon the principles of multimodality to possibly match each item's characteristic to the implicit preference expressed by the user [Deldjoo et al., 2022], thus addressing the (i) issue. Recent collaborative filtering approaches profile users and items through embedding vectors in the latent space. However, such models disregard structural properties naturally encoded into the user-item interaction data. Indeed, recommendation datasets are easily describable under the topology of a bipartite and undirected graph, with users and items being the graph nodes connected at multiple distance hops. In this respect, the application of graph neural networks , recent deep learning techniques specifically tailored to learn from non-euclidean data, can provide a refined representation of users and items to mine near- and long-distance relationships in the user-item graphs [Anelli et al., 2023b]. Indeed, this is one possible solution to exploit the collaborative information, which is effectively propagated within the user-item graph, addressing the (ii) issue. Conclusively, this thesis aims to match the two families of recommendation strategies by leveraging graph neural networks and multimodal information data [Anelli et al., 2022]. In doing so, other numerous micro-aspects within the two macro-areas (introduced above) are examined. Indeed, the thesis is a systematic compendium of careful analyses regarding, among others, reproducibility, novel evaluation dimensions [Anelli et al., 2023a], and tasks/scenarios complementary to recommendation. Awarded by : Politecnico di Bari, Bari, Italy on 30 January 2024. Supervised by : Tommaso Di Noia. Available at : https://hdl.handle.net/11589/264941.

Recently, graph neural networks (GNNs) have been improved under the influence of various deep learning techniques, such as attention, autoencoders, and recurrent networks. However, real-world graphs may have multiple types of vertices and edges, such as graphs of social networks, citation networks, and e-commerce data. In these cases, most GNNs that consider a homogeneous graph as input data are not suitable because they ignore the heterogeneity. Meta-path-based methods have been researched to capture both heterogeneity and structural information of heterogeneous graphs. As a meta-path is a type of graph pattern, we extend the use of meta-paths to exploit graph patterns. In this study, we propose TP-HAN, a heterogeneous graph attention network for exploiting triangle patterns. In the experiments using DBLP and IMDB, we show that TP-HAN outperforms the state-of-the-art heterogeneous graph attention network.KeywordsGraph neural networksHeterogeneous graphTriangle

Designing of VehiNet Using Convolutional Neural Networks and Deep Learning Techniques

Accurate object classification and position estimation is a crucial part of executing autonomous pick-and-place operations by a robot and can be realized using RGB-D sensors becoming increasingly available for use in industrial applications. In this paper, we present a novel unified framework for object detection and classification using a combination of point cloud processing and deep learning techniques. The proposed model uses two streams that recognize objects on RGB and depth data separately and combines the two in later stages to classify objects. Experimental evaluation of the proposed model including classification accuracy compared with previous works demonstrates its effectiveness and efficiency, making the model suitable for real-time applications. In particular, the experiments performed on the Washington RGB-D object dataset show that the proposed framework has 97.5% and 95% fewer parameters compared to the previous state-of-the-art multimodel neural networks Fus-CNN, CNN Features and VGG3D, respectively, with the cost of approximately 5% drop in classification accuracy. Moreover, the inference of the proposed framework takes 66.11%, 32.65%, and 28.77% less time on GPU and 86.91%, 51.12%, and 50.15% less time on CPU in comparison to VGG3D, Fus-CNN, and CNN Features. The potential applicability of the developed object classification and position estimation framework was then demonstrated on an experimental robot-manipulation setup realizing a simplified object pick-and-place scenario. In approximately 95% of test trials, the system was able to accurately position the robot over the detected objects of interest in an automatic mode, ensuring stable cyclic execution with no time delays.

Deep learning has revolutionized the performance of many computer vision systems in recent years. In particular deep convolutional neural networks have demonstrated ground-breaking performance in object classification from imagery. However, these techniques typically require sizeable volumes of training data in order to derive the large number of parameters that these types of approaches must learn. In many situations sufficient volumes of imagery for the object types of interest are unavailable. One solution to this problem is to use an initial training set which has properties similar to those of the objects of interest, but is available with a large number of labelled examples. These can be used for initial network training and the resulting partially-learned solution may subsequently be tuned using a smaller sample of the actual target objects. This type of approach, transfer learning, has shown considerable success in conventional imaging domains. Unfortunately, for Synthetic Aperture Radar imaging sensors, large volumes of labelled training samples of any type are hard to come by. The challenge is exacerbated when variations in imaging geometry and sensor configuration are taken into account. This paper examines the use of simulated SAR imagery in pre-training a deep neural network. The simulated imagery is generated using a straightforward process which has the capability to generate sufficient volumes of training exemplars in a modest amount of time. The samples generated are used to train a deep neural network which is then retrained using a comparatively small volume of MSTAR SAR imagery. The value of such a pre-training process is assessed using techniques to explain model performance by visualization. The assessment highlights some interesting aspects of the MSTAR SAR image set with regard to bias.

This paper proposes an intuitive teleoperation scheme by using human gesture in conjunction with multimodal human-robot interface. Further, in order to deal with the complication of dynamic daily environment, the authors apply haptic point cloud rendering and the virtual collaboration to the system. all these functions are achieved by a portable hardware that is proposed by authors newly, which is called "the mobile iSpace". First, a surrounding environment of a teleoperated robot is captured and reconstructed as the 3D point cloud using a depth camera. Virtual world is then generated from the 3D point cloud, which a virtual teleoperated robot model is placed in. Operators use their own whole-body gesture to teleoperate the humanoid robot. The Gesture is captured in real time using the depth camera that was placed on operator side. The operator recieves both the visual and the vibrotactile feedback at the same time by using a head mounted display and a vibrotactile glove. All these system components, the human operator, the teleoperated robot and the feedback devices, are connected with the Internet-based virtual collaboration system for a flexible accessibility. This paper showcases the effectiveness of the proposed scheme with experiment that were done to show how the operators can access the remotely placed robot in anytime and place.

Efficient graph representation in graph neural networks for stress predictions in stiffened panels

Abstract. A series of panoramic images are usually used to generate a 720° panorama image. Although panoramic images are typically used for establishing tour guiding systems, in this research, we demonstrate the potential of using panoramic images acquired from multiple sites to create not only 720° panorama, but also three-dimensional (3D) point clouds and 3D indoor models. Since 3D modeling is one of the goals of this research, the location of the panoramic sites needed to be carefully planned in order to maintain a robust result for close-range photogrammetry. After the images are acquired, panoramic images are processed into 720° panoramas, and these panoramas which can be used directly as panorama guiding systems or other applications. In addition to these straightforward applications, interior orientation parameters can also be estimated while generating 720° panorama. These parameters are focal length, principle point, and lens radial distortion. The panoramic images can then be processed with closerange photogrammetry procedures to extract the exterior orientation parameters and generate 3D point clouds. In this research, VisaulSFM, a structure from motion software is used to estimate the exterior orientation, and CMVS toolkit is used to generate 3D point clouds. Next, the 3D point clouds are used as references to create building interior models. In this research, Trimble Sketchup was used to build the model, and the 3D point cloud was added to the determining of locations of building objects using plane finding procedure. In the texturing process, the panorama images are used as the data source for creating model textures. This 3D indoor model was used as an Augmented Reality model replacing a guide map or a floor plan commonly used in an on-line touring guide system. The 3D indoor model generating procedure has been utilized in two research projects: a cultural heritage site at Kinmen, and Taipei Main Station pedestrian zone guidance and navigation system. The results presented in this paper demonstrate the potential of using panoramic images to generate 3D point clouds and 3D models. However, it is currently a manual and labor-intensive process. A research is being carried out to Increase the degree of automation of these procedures.

Grasp detection plays a pivotal role in robotic manipulation, allowing robots to interact with and manipulate objects in their surroundings. Traditionally, this has relied on three-dimensional (3D) point cloud data acquired from specialized depth cameras. However, the limited availability of such sensors in real-world scenarios poses a significant challenge. In many practical applications, robots operate in diverse environments where obtaining high-quality 3D point cloud data may be impractical or impossible. This paper introduces an innovative approach to grasp generation using color images, thereby eliminating the need for dedicated depth sensors. Our method capitalizes on advanced deep learning techniques for depth estimation directly from color images. Instead of relying on conventional depth sensors, our approach computes predicted point clouds based on estimated depth images derived directly from Red-Green-Blue (RGB) input data. To our knowledge, this is the first study to explore the use of predicted depth data for grasp detection, moving away from the traditional dependence on depth sensors. The novelty of this work is the development of a fusion module that seamlessly integrates features extracted from RGB images with those inferred from the predicted point clouds. Additionally, we adapt a voting mechanism from our previous work (VoteGrasp) to enhance robustness to occlusion and generate collision-free grasps. Experimental evaluations conducted on standard datasets validate the effectiveness of our approach, demonstrating its superior performance in generating grasp configurations compared to existing methods. With our proposed method, we achieved a significant 4% improvement in average precision compared to state-of-the-art grasp detection methods. Furthermore, our method demonstrates promising practical viability through real robot grasping experiments, achieving an impressive 84% success rate.