Reducing Computational Complexity in CNNs: A Focus on VGG19 Pruning and Quantization

A Refined Model of Computation for Continuous Problems

Morphological computation proposes the idea that in a physical system, certain computational processes can be off-loaded to the body. However, the concept has still eluded serious theoretical quantification attempts, unlike traditional computational theory. This perspective examines the notion of morphological computation from the well established theories of traditional computation and computational complexity, drawing parallels between the two, to understand the differences and similarities. Further, we look at the quantification efforts of morphological computation and attempt to link it to the unexplored field of control complexity. We argue that the development of complexity theory for control problems is necessary to study and utilize the concept of morphological computation, if it is possible.

Hematopathology II, Abstract 99–107, Posters

Granular computing is to imitate human’s multi-granular computing strategy to problem solving in order to endow computers with the same capability. Its final goal is to reduce the computational complexity. To the end, based on the simplicity principle the problem at hand should be represented as simpler as possible. From structural information theory, it’s known that if a problem is represented at different granularities, the hierarchical description of the problem will be a simpler one. The simpler the representation the lower the computational complexity of problem solving should be. We presented a quotient space theory to multi-granular computing. Based on the theory a problem represented by quotient spaces will have a hierarchical structure. Therefore, the quotient space based multi-granular computing can reduce the computational complexity in problem solving. In the talk, we’ll discuss how the hierarchical representation can reduce the computational complexity in problem solving by using some examples.

With a wide range of applications, single-image super-resolution (SISR) is a very important computer vision task. There are many SISR strategies which are based on the CNNs improvement, such as residual connections, deeper networks, and attention mechanisms. Although such kind of strategies often improve the SISR performance, they increase CNNs’ computational complexity and complex image texture areas still remain hard to be reconstructed. In particular, all the existing loss functions are minimized in the whole dynamic range, e.g. 0-255 in case of 8 bit image, which causes difficulty of CNNs’ learning in texture image regions above all. Developing new loss function provides a promising SISR solution, i.e. one should be beyond the existing regression loss functions, which encounter problem in reconstructing the image texture details. For such goal, this paper proposes a dynamic fine-grained loss function. It consists of both an existing regression loss function and a new pixel classification loss together with a dynamic regression range loss. Extensive experiments conducted on the benchmark SISR show that the proposed method can achieve better results and without extra computational cost.

Computability and computational complexity were first considered over the fields of real and complex numbers and generalized to arbitrary algebraic systems. This article approaches the theory of computational complexity over an arbitrary algebraic system by taking computability over the list extension for a computational model of it. We study the resultant polynomial complexity classes and mention some NP-complete problems.

This paper presents a new method for frame rate up‐conversion using sub‐sampled bilateral motion estimation, motion vector smoothing, simplified bidirectional motion compensated interpolation, and complex motion detection. The proposed method produces improved image quality and requires less computational complexity compared to conventional motion estimation. The average PSNRs of the proposed method for frame rate up‐conversion outperform those of the benchmark frame rate up‐conversion technique by up to 4.39 dB for the test sequences. Furthermore, the motion estimation of the proposed FRUC method reduces the required computational complexity to 12.5% of that required for full‐search block matching motion estimation.

Multicast is an internetwork service that provides efficient delivery of packets from a single source to multiple recipients. When there are large number of members in the group, security and scalability problems arise and an attempt to solve this, gives rise to additional computational complexities at the server. A model is said to be highly efficient if only it has less computational complexity at the server for all membership events and highly secure only when it requires large number of computations to successfully break the multicast model. In this paper, the computational complexities are determined and analyzed for different multicast models. Theoretical evaluation and experimental results prove that for all the membership events, the recently proposed multicast model named LeaSel [3] has computational complexity of O(NSG) when compared to other models which has computational complexity of O(N), where N ≫ NSG. It is also shown that to successfully break LeaSel, the computational complexity is O(SaN) when compared to other models whose computational complexity is O(Sn).

Multi-target detection is an important scientific proposition for millimeter-wave multiple-input multiple-output (MIMO) radar systems. Due to the short wavelength at millimeter-wave bands, multi-channel MIMO arrays are typically realized with very dense element spacing (often on the order of λ/2), which corresponds to only millimeter-level physical separation. It increases near-field interaction and parasitic coupling through the feed or package, making mutual coupling between adjacent channels severe and deteriorating the sensing performance. However, conventional decoupling methods unavoidably increase system hardware complexity, rendering them incompatible with compact sensing platforms. As a result, sparse MIMO radar supposes to be an efficient solution. Meanwhile, the non-uniform virtual co-array generated by sparse configurations typically contains missing lags, defined as holes, leading to incomplete virtual-aperture measurements and may degrade DOA resolution and multi-target separability. Conventional used interpolation-based methods utilize the convex optimization algorithm to fill in the holes, which significantly elevates computational complexity. In this paper, we proposed an enhanced multi-target detection approach using optimal sparse millimeter-wave radar architecture and polynomial interpolation method to reduce mutual coupling among different sensing channels. In this system, a lightweight sparse MIMO radar array configuration is proposed firstly. Then, the polynomial interpolation method is introduced to achieve DOF enhancement with lower computational and hardware complexity than traditional convex-optimization-based methods. Finally, the proposed approach is experimentally validated through multi-target detection scenarios, successfully identifying 8(6) targets using a 5(4) sensing channels. Simulation and experimental results demonstrate the superiority of the proposed sparse MIMO radar architecture over multi-target detection performance and computational complexity, offering a practical solution for diverse MIMO radar sensing platforms.

1 Modal logic: a semantic perspective

This paper examines the pedagogical benefits of introducing the history of algebra at the outset of a Theory of Computation course. By tracing the evolution from the abstraction of numbers and the invention of computational tools to the development of abstract concepts such as imaginary numbers, students are guided to recognize that the essence of computation resides in the abstraction and symbolic representation of human thought. This historical approach provides an intuitive and intellectually rich foundation for understanding core topics, including formal languages, computability, and computational complexity, while also encouraging students to reflect on the origins, development, and future trajectory of computational science.

In this paper we introduce an advanced supervised training method for neural networks. It is based on Jacobian rank deficiency and it is formulated, in some sense, in the spirit of the Gauss-Newton algorithm. The Levenberg-Marquardt algorithm, as a modified Gauss-Newton, has been used successfully in solving nonlinear least squares problems including neural-network training. It outperforms (in terms of training accuracy, convergence properties, overall training time, etc.) the basic backpropagation and its variations with variable learning rate significantly, however, with higher computation and memory complexities within each iteration. The new method developed in this paper is aiming at improving convergence properties, while reducing the memory and computation complexities in supervised training of neural networks. Extensive simulation results are provided to demonstrate the superior performance of the new algorithm over the Levenberg-Marquardt algorithm.

The analysis of high-volume microarray data faces challenges such as limited sample size, computational complexity, and the risk of inappropriate gene selection. The scarcity of samples hampers computational analysis and classification complexity, while reducing the classification's ability to generalize and predict new samples. Moreover, datasets with a high gene-to-sample ratio raise concerns about the selection of relevant genes for accurate predictive models. Interpreting disease-causing genes becomes intricate as only a subset of genes offers a precise biological insight into the disease. To address these issues, a focus on a smaller set of gene expression data is crucial for a more effective understanding of informative genes. Hence, the primary objective in microarray data analysis is to significantly reduce the number of genes through discriminative gene selection, enhancing the precision of information contained in the data. This article conducts gene expression classification on various cancer types, including colon cancer, breast cancer, leukemia, prostate tumors, and DLBCL. Each cancer type is independently evaluated in the feature selection cycle and classified using varying numbers of features. This approach aims to overcome challenges in microarray data analysis and improve the accuracy and interpretability of gene expression classification.

Lyapunov design has never been systematic. In the adaptive control of complex multi-input non-linear systems, physical considerations, such as conservation of energy or entropy increase, represent one of the major tools in building Lyapunov-like functions and providing stability and performance guarantees. In this paper we show that a physically motivated Lyapunov-like function based on the concept of total information can be derived for large classes of non-linear physical systems. We study how this function may be used for designing estimation, adaptation and learning mechanisms for such systems. In the process we revisit familiar notions such as controllability and observability from an information perspective, which in turns allows us to define 'natural' space-time scales at which to observe and control a given complex system. By formulating control problems in algorithmic form, we emphasize the importance of computability and computational complexity for issues of control. Generic control problems are shown to be NP-hard: each additional complication, such as the presence of noise or the absence of complete system identification, moves the control problem further up the polynomial hierarchy of computational complexity. In some cases, requirements of 'optimality' may be unrealistic or irrelevant, since the solution to the problem of finding the optimal algorithm for control is uncomputable.

Spiking neural networks coupled with neuromorphic hardware and event-based sensors are getting increased interest for low-latency and low-power inference at the edge. However, multiple spiking neuron models have been proposed in the literature with different levels of biological plausibility and different computational features and complexities. Consequently, there is a need to define the right level of abstraction from biology in order to get the best performance in accurate, efficient and fast inference in neuromorphic hardware. In this context, we explore the impact of synaptic and membrane leakages in spiking neurons. We confront three neural models with different computational complexities using feedforward and recurrent topologies for event-based visual and auditory pattern recognition. Our results showed that, in terms of accuracy, leakages are important when there are both temporal information in the data and explicit recurrence in the network. Additionally, leakages do not necessarily increase the sparsity of spikes flowing in the network. We also investigated the impact of heterogeneity in the time constant of leakages. The results showed a slight improvement in accuracy when using data with a rich temporal structure, thereby validating similar findings obtained in previous studies. These results advance our understanding of the computational role of the neural leakages and network recurrences, and provide valuable insights for the design of compact and energy-efficient neuromorphic hardware for embedded systems.