Fast Post-Hoc Normalization for Brain Inspired Sparse Coding on a Neuromorphic Device

K-Singular Value Decomposition (K-SVD) is an excellent feature analysis method, but it has a large amount of calculation and low algorithm efficiency. To solve the above problems, a dictionary learning method for image block classification training is proposed. This method uses image entropy as the classification index of image samples, selects more complex texture image blocks with more analytical value, and uses Discrete Cosine Transform- Discrete Wavelet Transformation (DCT-DWT) dictionary to train them in K-Singular Value Decomposition. Through the feedback of sparse coefficients, the threshold value is dynamically changed and the precision of sparse coding is controlled. In the sparse solution, a variable-step Orthogonal Matching Pursuit (OMP) algorithm is used. The experimental comparison shows that the classification K-Singular Value Decomposition dictionary learning algorithm runs faster, and the Peak signal-to-noise Ratio (PSNR) and Structural Similarity (SSIM) value of the reconstructed image are higher.

Sparse coding employs low-dimensional subspaces in order to encode high-dimensional signals. Finding the optimal subspaces is a difficult optimization task. We show that stochastic gradient descent is superior in finding the optimal subspaces compared to MOD and K-SVD, which are both state-of-the art methods. The improvement is most significant in the difficult setting of highly overlapping subspaces. We introduce the so-called “Bag of Pursuits” that is derived from Orthogonal Matching Pursuit. It provides an improved approximation of the optimal sparse coefficients, which, in turn, significantly improves the performance of the gradient descent approach as well as MOD and K-SVD. In addition, the “Bag of Pursuits” allows to employ a generalized version of the Neural Gas algorithm for sparse coding, which finally leads to an even more powerful method.

Hyperspectral images (HSI) have high spectral and low spatial resolutions. However, multispectral images (MSI) usually have low spectral and high spatial resolutions. In various applications HSI with high spectral and spatial resolutions are required. In this paper, a new method for spatial resolution enhancement of HSI using high resolution MSI based on sparse coding and linear spectral unmixing (SCLSU) is introduced. In the proposed method (SCLSU), high spectral resolution features of HSI and high spatial resolution features of MSI are fused. In this case, the sparse representation of some high resolution MSI and linear spectral unmixing (LSU) model of HSI and MSI is simultaneously used in order to construct high resolution HSI (HRHSI). The fusion process of HSI and MSI is formulated as an ill-posed inverse problem. It is solved by the Split Augmented Lagrangian Shrinkage Algorithm (SALSA) and an orthogonal matching pursuit (OMP) algorithm. Finally, the proposed algorithm is applied to the Hyperion and ALI datasets. Compared with the other state-of-the-art algorithms such as Coupled Nonnegative Matrix Factorization (CNMF) and local spectral unmixing, the SCLSU has significantly increased the spatial resolution and in addition the spectral content of HSI is well maintained.

Purpose. The article discusses the use of mobile technologies in teaching mathematics to engineering students. The relevance of the paper is due to the need to introduce new methods and means of teaching in connection with the increasing integration of mobile technologies into all areas of life, the interests and characteristics of the new, networked generation of students, the limited possibilities of using innovative methods and means of teaching mathematics due to the conduct of classes outside the computer lab.Materials and methods. The peculiarity of this study is that the integration of mobile technologies into the process of teaching mathematics is considered as a system of three components: technological, informational and social. The analysis of these components made it possible to obtain answers to research questions: where and how mobile devices should be used when studying mathematics; what mobile applications will provide the most effective learning of the discipline; how to organize the educational process using mobile technologies to increase interest and motivation in learning mathematics.Results. The main reasons for the lack of interest in studying mathematics and low motivation include the high level of abstraction of the discipline and the characteristics of the modern, networked generation of students. Analysis of scientific sources and our own observations made it possible to identify such characteristics as problems with understanding complex and long text, the ability to easily read from a small screen and process information in parallel, the choice of sound and images instead of text, the desire for short-term learning and instant results, and attitude towards mobile devices as vital items. Taking into account the interests and wishes of students, and relying on the didactic capabilities of mobile technologies, the paper proposes mobile applications for studying various topics of a mathematical course, assignment options that help increase motivation, develop group work skills and digital literacy. The importance and effectiveness of using social services for visualizing mathematical concepts and calculation schemes, as well as the WhatsApp messenger for instant messaging between students and receiving timely individual and group consultation from a lecturer, is shown. An important characteristic of the messenger is the presence of video communication.Conclusion. The analysis of scientific literature, the study of the requirements of employers and the interests of the modern, networked generation of students, indicated the need to integrate mobile technologies into the process of teaching mathematics. Working with different groups in terms of both the level of mathematical knowledge and the level of proficiency in mobile technologies made it possible to identify and outline ways to solve the main issues of using mobile technologies in teaching mathematics. With limited time to study mathematics, it is difficult for a lecturer to allocate additional time to present the capabilities and effectiveness of mobile applications. A solution to this issue may be to include a section “Mobile Technologies” in a course related to the study of information and communication technologies. It is noted that group work also contributes to a more effective study of the capabilities and techniques of working with mobile applications.

Modern industrial control systems for complex objects are created using the latest achievements in the microprocessor technology and based on a range of technical tools from leading manufacturers, such as: Honeywell, Siemens, Schneider Electric, etc. The automation of large industrial enterprises of oil and gas, metallurgy, aerospace and other industries is carried out taking into account the requirements of reliability, safety and efficiency of the equipment. An important factor is the timely analysis and diagnosis of control systems, since even minor, unpredictable equipment failures can lead to emergency situations, as well as to economic production losses. Distributed enterprise management systems are overloaded with data streams, most of which are archived and are not further analyzed. An effective solution to this problem is the integration and application of the latest achievements in the field of artificial intelligence (AI). In turn, the bio-inspired approach of artificial immune systems is rapidly developing, which has the following advantages: the ability to process a large amount of production data, the ability to process information in parallel, self-training, the presence of memory, and the ability to predict at the class boundaries. Since there are currently no universal artificial intelligence algorithms capable of equally efficient forecasting for various types of production data, it is effective to develop modified algorithms for artificial immune systems. The researches are devoted to the development of a diagnostic system for industrial equipment based on the AMDEC (l'Analyse des Modes de Defaillances, de leurs Effets et de leur Criticite) mode, failure and criticality analysis approach and modified AIS algorithms, using the industrial equipment from Schneider Electric as an example. The AMDEC approach identifies equipment weaknesses and is used to predict potential failures. The disadvantage of this method is its complexity. Extending the AMDEC model, using modified algorithms of artificial immune systems, allows, on the basis of data mining, to predict the state of industrial equipment, to assess the severity of individual failures, and to make recommendations for decision making on the elimination of failures.

Quantum computing (QC) is not a futuristic notion in agriculture, though its full potential has yet to be realized. QC is an emerging field at the intersection of physics and computer science that holds immense potential to revolutionize various sectors, including agriculture production and artificial intelligence (AI) modeling. While QC is still in the early stages of development and practical applications within agriculture are not yet widespread, researchers are actively exploring its potential benefits in various agricultural domains, including crop optimization, livestock breeding, and environmental monitoring. QC harnesses the principles of quantum mechanics to perform computations using quantum bits or qubits, which can exist in multiple states simultaneously. Unlike classical computers, which rely on binary bits representing 0 or 1, quantum computers exploit phenomena such as superposition and entanglement to process information in parallel, potentially offering exponential speedup for certain types of problems. In agriculture production, particularly in animal science, QC offers promising avenues for optimizing processes and enhancing productivity. Quantum algorithms can analyze vast amounts of genomic data to improve breeding programs, leading to the development of more resilient and productive livestock breeds. Furthermore, QC can facilitate precision farming techniques by modeling complex environmental factors and animal behavior to optimize feeding strategies, disease management, and overall farm management practices. Moreover, QC can significantly benefit AI modeling by accelerating computations and enabling more efficient training of AI models. Quantum algorithms can enhance the performance of AI algorithms in various tasks, including pattern recognition, natural language processing, and predictive analytics. By leveraging quantum-enhanced optimization algorithms, AI models can achieve better convergence and accuracy, leading to more effective decision-making and problem-solving capabilities. While hybrid intelligent models also represent a novel frontier in agriculture, QC has the potential to expedite the merging of mechanistic and AI modeling paradigms, facilitating a more holistic understanding of complex systems in agriculture and beyond. By integrating mechanistic models, which describe the underlying physical processes, with AI models, which learn patterns from data, quantum computing can enable comprehensive simulations and predictions of agricultural systems. This fusion of modeling paradigms can lead to more accurate and robust predictions of crop yields, livestock performance, and environmental impacts, facilitating informed decision-making for farmers and policymakers. The application of QC in agriculture, however, requires interdisciplinary collaborations between physicists, computer scientists, agronomists/animal scientists, and AI researchers. These collaborations can drive the development of quantum algorithms tailored to agricultural applications, the integration of quantum-enhanced AI techniques into existing modeling frameworks, and the deployment of QC resources in real-world agricultural systems. Ultimately, harnessing the power of QC holds the potential to revolutionize agriculture production practices, including regenerative agriculture, and advance AI modeling capabilities, paving the way for a more sustainable and efficient agricultural industry.

Reservoir fluid properties data are very important in reservoir engineering computations such as material balance calculations, well testing, reserve estimates, and numerical reservoir simulations. Ideally, those data should be obtained experimentally. On some occasions, these data are not either available or reliable; then, empirically derived correlations are used to predict PVT properties. However, the success of such correlations in prediction depends mainly on the range of data at which they were originally developed. These correlations were developed using linear, non-linear, multiple regression or graphical techniques. Recently, researchers utilized artificial neural networks (ANN) to develop more accurate PVT correlations. ANNs are biologically inspired non-algorithmic, non-digital, massively parallel distributive and adaptive information processing systems. They resemble the brain in acquiring knowledge through learning process, and storing knowledge in inter-neuron connection strengths. The present study presents new models developed to predict the bubble point pressure and, the formation volume factor at the bubble point pressure. The models were developed using 283 data sets collected from Saudi reservoirs. These data were divided into three groups: the first was used to train the ANN models, the second was used to cross-validate the relationships established during the training process and, the last was used to test the models to evaluate their accuracy and trend stability. Trend tests were performed to ensure that the developed model would follow the physical laws. Results show that the developed models outperform the published correlations in terms of absolute average percent relative error, and standard deviation.

Sparse coding models of natural scenes can account for several physiological properties of primary visual cortex (V1), including the shapes of simple cell receptive fields (RFs) and the highly kurtotic firing rates of V1 neurons. Current spiking network models of pattern learning and sparse coding require direct inhibitory connections between the excitatory simple cells, in conflict with the physiological distinction between excitatory (glutamatergic) and inhibitory (GABAergic) neurons (Dale's Law). At the same time, the computational role of inhibitory neurons in cortical microcircuit function has yet to be fully explained. Here we show that adding a separate population of inhibitory neurons to a spiking model of V1 provides conformance to Dale's Law, proposes a computational role for at least one class of interneurons, and accounts for certain observed physiological properties in V1. When trained on natural images, this excitatory-inhibitory spiking circuit learns a sparse code with Gabor-like RFs as found in V1 using only local synaptic plasticity rules. The inhibitory neurons enable sparse code formation by suppressing predictable spikes, which actively decorrelates the excitatory population. The model predicts that only a small number of inhibitory cells is required relative to excitatory cells and that excitatory and inhibitory input should be correlated, in agreement with experimental findings in visual cortex. We also introduce a novel local learning rule that measures stimulus-dependent correlations between neurons to support "explaining away" mechanisms in neural coding.

A new class of neuromorphic processors promises to provide fast and power-efficient execution of spiking neural networks with on-chip synaptic plasticity. This efficiency derives in part from the fine-grained parallelism as well as event-driven communication mediated by spatially and temporally sparse spike messages. Another source of efficiency arises from the close spatial proximity between synapses and the sites where their weights are applied and updated. This proximity of compute and memory elements drastically reduces expensive data movements but imposes the constraint that only local operations can be efficiently performed, similar to constraints present in biological neural circuits. Efficient weight update operations should therefore only depend on information available locally at each synapse as non-local operations that involve copying, taking a transpose, or normalizing an entire weight matrix are not efficiently supported by present neuromorphic architectures. Moreover, spikes are typically non-negative events, which imposes additional constraints on how local weight update operations can be performed. The Locally Competitive Algorithm (LCA) is a dynamical sparse solver that uses only local computations between non-spiking leaky integrator neurons, allowing for massively parallel implementations on compatible neuromorphic architectures such as Intel's Loihi research chip. It has been previously demonstrated that non-spiking LCA can be used to learn dictionaries of convolutional kernels in an unsupervised manner from raw, unlabeled input, although only by employing non-local computation and signed non-spiking outputs. Here, we show how unsupervised dictionary learning with spiking LCA (S-LCA) can be implemented using only local computation and unsigned spike events, providing a promising strategy for constructing self-organizing neuromorphic chips.

Speech signals are almost invariably corrupted with either background noise or mixed with other coherent speech. Various techniques are used for speech enhancement like Nonnegative matrix factorization (NMF), Independent component analysis (ICA) etc. One of the techniques is sparse coding and dictionary learning. For this standard algorithmic approaches use iterative techniques like KSVD and Orthogonal Matching Pursuit (OMP) which require significant memory and computation time to process successfully. We, however, use a novel approach of using k-sparse autoencoders which has not been previously used in speech processing. The proposed approach extends k-sparse autoencoders as a denoising autoencoder which allows us to achieve significantly better performance. This research work demonstrate that the use of k-sparse autoencoder has number of advantages especially it does not need any prior knowledge on the statistical characteristics of the noise and it performs much better on signals more heavily corrupted with noise. In addition to standard datasets,it's superior performance over other dictionary learning techniques are demonstrated on speech signals that are sensed on android phones.

Simple cells in primary visual cortex were famously found to respond to low-level image components such as edges. Sparse coding and independent component analysis (ICA) emerged as the standard computational models for simple cell coding because they linked their receptive fields to the statistics of visual stimuli. However, a salient feature of image statistics, occlusions of image components, is not considered by these models. Here we ask if occlusions have an effect on the predicted shapes of simple cell receptive fields. We use a comparative approach to answer this question and investigate two models for simple cells: a standard linear model and an occlusive model. For both models we simultaneously estimate optimal receptive fields, sparsity and stimulus noise. The two models are identical except for their component superposition assumption. We find the image encoding and receptive fields predicted by the models to differ significantly. While both models predict many Gabor-like fields, the occlusive model predicts a much sparser encoding and high percentages of ‘globular’ receptive fields. This relatively new center-surround type of simple cell response is observed since reverse correlation is used in experimental studies. While high percentages of ‘globular’ fields can be obtained using specific choices of sparsity and overcompleteness in linear sparse coding, no or only low proportions are reported in the vast majority of studies on linear models (including all ICA models). Likewise, for the here investigated linear model and optimal sparsity, only low proportions of ‘globular’ fields are observed. In comparison, the occlusive model robustly infers high proportions and can match the experimentally observed high proportions of ‘globular’ fields well. Our computational study, therefore, suggests that ‘globular’ fields may be evidence for an optimal encoding of visual occlusions in primary visual cortex.

Neuromorphic computing provides a promising solution to the von Neumann bottleneck and has received a lot of research attention. However, due to the limitation of static threshold activation of traditional neuromorphic devices, it is difficult to simulate the dynamic sparsity characteristics of biological neuron, resulting in more than 90% computational redundancy in fully connected neural networks architectures based on classical devices, which has become a key issue for efficient neuromorphic computing. Here, a dynamically adaptive activation neuron‐transistor based on asymmetric electrodes and indium gallium zinc oxide thin films is proposed, overcoming the static activation limitations of conventional neuromorphic devices. The device achieves a dynamic adaptive activation similar to biometric neuron via gate voltage or UV irradiation, achieving a wide range of activation times (65 ms–13.5 s) and adjustable activation thresholds (2.5–7.7 V). Leveraging this device, a dynamic sparse spiking neural network (DS‐SNN) is constructed that enables in situ Hadamard‐based weight pruning/regeneration. Applied to autonomous driving object detection, the DS‐SNN achieves 85% accuracy with 42% sparsity, outperforming dense convolutional/spiking neural networks (320k /140k weights) while utilizing only ≈80k parameters. This hardware‐algorithm co‐design establishes a new paradigm for energy‐efficient edge‐computing electronics, exploring 3D integration of large‐scale neuromorphic processors.

A Specialized Area in Limbic Cortex for Fast Analysis of Peripheral Vision

A novel method which employs the recently developed theory of compressed sensing (CS) in method of moments (MoM) is presented. With a suitable discrete wavelet transform (DWT), the known impedance matrix, excitation and the unknown solution are localized, so the CS technique is introduced to construct the underdetermined equation that can be efficiently solved by the stagewise orthogonal matching pursuit technique (StOMP) method. Based on the fact of the significant correlation between the localized solution and excitation vector, the underdetermined equation can be constructed systematically and deterministically. Due to the computational complexity of StOMP depends not only on the size of equation, but also on the sparsity of solution, the computa¬tional cost of our method can be saved dramatically. The perfor¬mance of the proposed approach is illustrated by numerical studies of electromagnetic field integral equations.

Algorithms that encode images using a sparse set of basis functions have previously been shown to explain aspects of the physiology of a primary visual cortex (V1), and have been used for applications, such as image compression, restoration, and classification. Here, a sparse coding algorithm, that has previously been used to account for the response properties of orientation tuned cells in primary visual cortex, is applied to the task of perceptually salient boundary detection. The proposed algorithm is currently limited to using only intensity information at a single scale. However, it is shown to out-perform the current state-of-the-art image segmentation method (Pb) when this method is also restricted to using the same information.