Set-Based State Estimators with Iterative Update

The L1-regularized models are widely used for sparse regression or classification tasks. In this paper, we propose the orthant-wise passive descent algorithm (OPDA) for solving L1-regularized models, as an improved substitute of proximal algorithms, which are the standard tools for optimizing the models nowadays. OPDA uses a stochastic variance-reduced gradient (SVRG) to initialize the descent direction, then apply a novel alignment operator to encourage each element keeping the same sign after one iteration of update, so the parameter remains in the same orthant as before. It also explicitly suppresses the magnitude of each element to impose sparsity. The quasi-Newton update can be utilized to incorporate curvature information and accelerate the speed. We prove a linear convergence rate for OPDA on general smooth and strongly-convex loss functions. By conducting experiments on L1-regularized logistic regression and convolutional neural networks, we show that OPDA outperforms state-of-the-art stochastic proximal algorithms, implying a wide range of applications in training sparse models.

IntroductionIn December 2019, the first cases of COVID-19 associated with SARS-CoV-2 viral infection were described in Wuhan, Hubei Province, China. Since then, it has spread rapidly affecting 188 countries and...

A hybrid global maximum power point tracking method for photovoltaic arrays under partial shading conditions

In recent years, terrain-aided navigation (TAN) has been rapidly developed for the localization of autonomous underwater vehicles (AUVs). TAN provides a position estimate for vehicles relative to a priori topographic map and is typically a nonlinear filtering model. Particle filtering has attracted considerable attention for addressing nonlinear and non-Gaussian challenges. However, it has problems with particle degradation and particle impoverishment. To tackle the problem in this study, the particle swarm optimization (PSO) optimization approach is employed to increase particle variety while driving particles towards the high probability zone through iterative updates. In addition, prior information is introduced to the particle fitness to reduce the interference of pseudo-peaks and increase the estimate precision. Finally, simulation tests are implemented to verify the effectiveness of the proposed method. Simulation results indicate that the improved GPSO-PF method performs better in the case of likelihood function with multiple peaks.

The Internet of Things (IoT) has penetrated into various application fields. If the multimedia information obtained by the IoT device is tampered with, the subsequent information processing will be affected, resulting in an incorrect service and even security threat. Therefore, it is very necessary to study multimedia forensics technology for IoT security. In the edge-cloud IoT environment, an image anomaly detection technology for security service is proposed in this paper. First, preprocessing is performed before image anomaly detection. Then, we extracted sparse features from the image to roughly localize the region of anomaly detection. Feature extraction based on the polar cosine transform (PCT) is then performed only on the candidate region of anomaly detection. To further improve the detection accuracy, we use iterative updating. This method makes use of the feature that the edge node is closer to the multimedia source in physical location and migrates the complex computing task of image anomaly detection from the cloud computing center to the edge node. Provide a security service for abnormal data and deploy it to the edge-cloud server to reduce the pressure on the cloud. Overall, preprocessing improves the ability of feature extraction in smooth or small region of anomaly detections, and the iterative strategy enhances the security service. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.

The booming growth in AI and machine learning is drastically reshaping the landscape of high performance computing. Traditional HPC addresses scientific problems that are driven by simulation, modeling and analysis inscience domains. Algorithms like linear algebra and methods like differential equations are at the core of solutions to such problems. However, emerging machine learning tasks rest on a new set of algorithms and models. The computation and data movement patterns inherent in learning algorithms cannot be directly mapped to the computation motifs in the physics, chemistry, or biology simulation problems. As a result, the high performance libraries that originate in the traditional scientific domains thus cannot be straightforwardly applied to emerging ML tasks to deliver required performance. This thesis focuses on performance optimizations of computation kernelsin emerging machine learning applications, spanning across a diverse range from dense, regular to sparse and irregular kernels. In this work, we demonstrate how code specialization and generation together with expert-built performance models and learned dispatch strategies can together enable ML motifs to achieve better performance on modern processors. First, we investigate the performance optimization of dense kernels with a focus on the convolutional neural networks (CNN). The computation of convolution layers in deep neural networks typically relies on high performance matrix-multiplication routines to improve performance. However, these routines are not optimized for performing convolution. Extra memory overhead is incurred due to data transformation, and the performance obtained is also less than conventionally expected. We demonstrate that direct convolution, when implemented correctly, eliminates all memory overhead, and yields performancethat is between 10% to 4x better than existing high performance implementations on conventional and embedded CPU architectures. We present a model-guided optimization approach which utilizes the characteristics of system architectures to guide the optimization choices of loop ordering, blocking, and memory layout transformation. We show that a high performance direct convolution exhibits better performance scaling than expert-tuned matrix implementation, i.e. suffers less performance drop, when increasing the number of threads. Sparse kernel is an equally important computation kernel appearing in many machine learning applications such as graph analytics and genetic sequencing. One factor that prevents sparse kernels from achieving high performance on modern processors results from the prohibitively large number of different implementations and data structures for sparse problems. We start with the observation that the complicated sparse computations can be distilled into primitive set of operators such as join, merge, and difference. To acceleratethose operators on modern processors with data parallelism, we propose a vectorization and code specialization approach which can eliminate the control divergences of these operators. Next, we explore the design space for vectorization on CPUs with various vector width, based on which we present the code generation algorithm that takes the data width and operations as input and generates various implementations. We then demonstrate the acceleration of the General Sparse Matrix-Matrix Multiplication (SpGEMM) on GPUs. We show how the SpGEMM implementation can leverage join/merge operatorsto compose a variety of implementations. Another challenge when optimizing sparse kernels is that their performance behavior is data dependent, while the input characteristics may change online during iterative updates. To leveragethe different implementations offered by the code generator, we propose a low-overhead mechanism that collects the data characteristic information to learn online dispatch decisions over iterations. Overall, in this thesis, we demonstrate the interplay of code specialization and generation, together with performance modeling, learned dispatch, can enable high performance kernels for the emerging machine learning applications.

Effective provision of integrated sensing, communication, and computation (ISCC) services in future networks will inevitably increase their operational complexity. The distinct requirements of diverse tasks for tailored ISCC devices further exacerbate the challenge of adaptively allocating constrained resources among concurrent tasks. To address these difficulties, a task-specific joint resource orchestration scheme is proposed in this paper to enhance the effectiveness of ISCC operation and heterogeneous tasks completion. Specifically, the completion of concurrent heterogeneous tasks by different devices relies on the task-specific sharing of limited resource among sensing, real-time data computing and delay-tolerant data processing. Consequently, a value of multi-task completion (VoC) indicator is designed to connect and balance among the diverse demands from concurrent tasks, including computing rate, time delay, and sensing performance. The VoC is then maximized by collaborative optimization of multi-dimensional resources, including transmit beamformer, local and offloading CPU-cycle frequency, data factor assignment and computation capacity. To solve this challenging optimization problem with the lack of close-form solution and coupling of multi-variables, we first transform it into an equivalent form that is tractable to handle. Next, the problem is decomposed into several subproblems, which can be approximately solved by iterative updates. Simulation results demonstrate the performance enhancement of the proposed scheme is superior to the benchmarks.

In the initial construction of unit-level virtual models in the discrete production system, the understanding of physical entities and the actual operating environment is lacking, and the virtual models need to be iteratively updated to resemble the physical entities more closely. To solve the above problems, an updating method for knowledge in the virtual model driven by data is studied. Firstly, the behavior model in the virtual model can be divided into basic functional models such as forward kinematics, trajectory planning, and material generation, and the knowledge components to support functional model computation are constructed. Secondly, the evaluation indexes for the quality, maintainability, and reliability of the knowledge components are established, and appropriate knowledge component combinations can be selected from the knowledge component base. Then, based on the measurements, it is used to the triggering judgment, and a knowledge component combination with the smallest difference between the simulation output of the virtual model and the real data of physical entities is selected from multiple combinations to realize the iterative update of the model. Finally, the feasibility and effectiveness of the method are verified by examples.

Text summarization using Wikipedia

This survey reviews the recent development of gradient domain mesh deformation method. Different to other deformation methods, the gradient domain deformation method is a surface-based, variational optimization method. It directly encodes the geometric details in differential coordinates, which are also called Laplacian coordinates in literature. By preserving the Laplacian coordinates, the mesh details can be well preserved during deformation. Due to the locality of the Laplacian coordinates, the variational optimization problem can be casted into a sparse linear system. Fast sparse linear solver can be adopted to generate deformation result interactively, or even in real-time. The nonlinear nature of gradient domain mesh deformation leads to the development of two categories of deformation methods: linearization methods and nonlinear optimization methods. Basically, the linearization methods only need to solve the linear least-squares system once. They are fast, easy to understand and control, while the deformation result might be suboptimal. Nonlinear optimization methods can reach optimal solution of deformation energy function by iterative updating. Since the computation of nonlinear methods is expensive, reduced deformable models should be adopted to achieve interactive performance. The nonlinear optimization methods avoid the user burden to input transformation at deformation handles, and they can be extended to incorporate various nonlinear constraints, like volume constraint, skeleton constraint, and so on. We review representative methods and related approaches of each category comparatively and hope to help the user understand the motivation behind the algorithms. Finally, we discuss the relation between physical simulation and gradient domain mesh deformation to reveal why it can achieve physically plausible deformation result.

With the rapid growth of web data, information about the same target gathered from multiple sources often exhibits conflicts. This problem motivates the need for truth discovery, which is to automatically resolve conflicts and find the truth from multiple conflicting claims. Existing truth discovery methods are mainly based on iterative updates or probability models. A common limitation of these methods is that their models are complex to be built. In this paper, we propose a concise end-to-end deep neural network for truth discovery, which regards the task as a classification problem. Firstly, for each target, we extract a unique claim, and for each unique claim, we construct a source-unique-claim vector depending on whether the source provides this value. Then on the training dataset, we label the vector as true/false according to the ground truth. Finally, we use a deep neural network to build a classification model for each target to judge which claim is the truth. Experimental results on two real-world datasets show that our proposed model has better performance than existing state-of-the-art methods.

Understanding scene image includes detecting and recognizing objects, estimating the interaction relationships of the detected objects, and describing image regions with sentences. However, since the complexity and variety of scene image, existing methods take object detection or vision relationship estimate as the research targets in scene understanding, and the obtained results are not satisfactory. In this work, we propose a Multi-level Semantic Tasks Generation Network (MSTG) to leverage mutual connections across object detection, visual relationship detection and image captioning, to solve jointly and improve the accuracy of the three vision tasks and achieve the more comprehensive and accurate understanding of scene image. The model uses a message pass graph to mutual connections and iterative updates across the different semantic features to improve the accuracy of scene graph generation, and introduces a fused attention mechanism to improve the accuracy of image captioning while using the mutual connections and refines of different semantic features to improve the accuracy of object detection and scene graph generation. Experiments on Visual Genome and COCO datasets indicate that the proposed method can jointly learn the three vision tasks to improve the accuracy of those visual tasks generation.

In recent days, the rapid growth of advanced communication links among smart consumer appliances has enabled the integration of intelligent energy management system which combines sensing, communication, computing and control technologies within consumer electronics. A synthetic IoT sensor stream is developed to simulate real-time electricity consumption data from multiple utilities, transmitting sensor packets over a wireless uplink with realistic network conditions including jitter and packet loss. The main objective of this study is to design a lightweight hybrid framework which ensures convergence between cloud-level learning and edge-level control while maintaining bounded prediction error and latency. The convergence of the distilled-tree inference model with the cloud-based XGBoost model is achieved through minimization of distillation loss, guaranteeing stable edge predictions with steady-state variance below 3% across iterative updates. The proposed model ensures lightweight inference while the gradient boosting contributes to high predictive accuracy. The experimental results shows that the proposed model achieves about 17.6% simulated energy savings compared to traditional approaches like feedforward neural networks (12%) and random forest (10.3%). In addition, this study includes a prototype for consumer dashboard which showcases real-time device integration and futuristic energy management.

Quantitative analysis using a standardized uptake value (SUV) has become possible for single-photon emission computed tomography-computed tomography (SPECT-CT) of bone. However, previous research was targeted to the trunk area, and there are few studies for the head and neck region. Therefore, the purpose of this study was to determine the optimal image reconstruction conditions for bone SPECT of the head and neck using a phantom study. The radioactivity concentration of the 99mTc solution enclosed in the cylindrical phantom was set to the same count rate as in clinical cases, and six hot spheres (10, 13, 17, 22, 28, 37 mm) with four times the concentration were placed within it. The image reconstruction was 3D-OSEM, and the reconstruction conditions were varied by the number of iterative updates and the width of the Gaussian filter. Quantitative evaluations of the image quality were performed using the % contrast, background variability, and SUV for the hot spheres and background. A visual evaluation was performed by four observers to determine the optimal image reconstruction conditions for bone SPECT of the head and neck region. The concentration of the 99mTc solution enclosed in the phantom was 6.95 (kBq/ml). Based on the results of the quantitative and visual evaluations, the optimal image reconstruction conditions were iterative updates=60 (subset: 10, iteration: 6) and a Gaussian filter of 7.8 mm. The optimal image reconstruction conditions were subset=10, iterations=6, and a Gaussian filter of 7.8 mm.

Speech recognisers are very sensitive to degradation of the acoustical environment. In order to limit the influence of noise, reverberation and interference, several strategies can be applied. A new method for tracking location (i.e., range and bearing) of a single talker of interest for microphone arrays in the very near field (the range of the talker is less than or nearly equal to the array length) is proposed. In the proposed method, an array is divided into two subarrays, focusing transformations only for the talker are used, and an iteration update is proceeded. The proposed method is suitable for cases of low signal-to-noise ratio (SNR) or/and strong interfering signals (even it is stronger than the talker's signal). Computer simulations are given to support the proposed method.