Feature Overlapping: Temporal Differential Decoupling for Efficient Spiking Neural Network Training.

Bio-plausible spiking neural networks (SNNs) have gained a great momentum due to its inherent efficiency of processing event-driven information. The dominant computation-matrix bit-wise And-Add operations-in SNN is naturally fit for process-in-memory architecture (PIM). The long input spike train of SNN and the bit-serial processing mechanism of PIM, however, incur considerable latency and frequent analog-to-digital conversion, offsetting the performance gain and energy-efficiency. In this paper, we propose a novel Search-in-Memory (SIM) architecture to accelerate the SNN inference, named SIMSnn. Rather than processing the input bit-by-bit over multiple time steps, SIMSnn can take in a sequence of spikes and search the result by parallel associative matches in the CAM crossbar. As a weight-agnostic SNN accelerator, SIMSnn can adapt to various evolving SNNs without rewriting the crossbar array.

Spiking neural networks (SNNs) are potentially highly efficient models for inference on fully parallel neuromorphic hardware, but existing training methods that convert conventional artificial neural networks (ANNs) into SNNs are unable to exploit these advantages. Although ANN-to-SNN conversion has achieved state-of-the-art accuracy for static image classification tasks, the following subtle but important difference in the way SNNs and ANNs integrate information over time makes the direct application of conversion techniques for sequence processing tasks challenging. Whereas all connections in SNNs have a certain propagation delay larger than zero, ANNs assign different roles to feed-forward connections, which immediately update all neurons within the same time step, and recurrent connections, which have to be rolled out in time and are typically assigned a delay of one time step. Here, we present a novel method to obtain highly accurate SNNs for sequence processing by modifying the ANN training before conversion, such that delays induced by ANN rollouts match the propagation delays in the targeted SNN implementation. Our method builds on the recently introduced framework of streaming rollouts, which aims for fully parallel model execution of ANNs and inherently allows for temporal integration by merging paths of different delays between input and output of the network. The resulting networks achieve state-of-the-art accuracy for multiple event-based benchmark datasets, including N-MNIST, CIFAR10-DVS, N-CARS, and DvsGesture, and through the use of spatio-temporal shortcut connections yield low-latency approximate network responses that improve over time as more of the input sequence is processed. In addition, our converted SNNs are consistently more energy-efficient than their corresponding ANNs.

Hydrological modelling at multiple sub-daily time steps: Model improvement via flux-matching

Thermosolutal convection inevitably occurs during the solidification of alloys owing to the nonuniform distribution of temperature and/or solute concentration, and this can drastically alter the resulting solidification microstructures. In this study, we present a large-scale simulation scheme for the phase-field lattice Boltzmann model, which can express dendrite growth upon considering the solute, heat transport, and liquid flow. A multiple mesh and time step method was employed to reduce computational costs, where different mesh sizes and time steps are used to solve the phase-field equation, the advection–diffusion equations of heat and solute, and the lattice Boltzmann equations for fluid flow. Furthermore, we implemented parallel computations using multiple graphics processing units (GPUs) to accelerate the large-scale simulation. Through the application of the multiple mesh and time step method, the computation was accelerated by approximately one hundred times compared to the case using a constant mesh and time step for all equations. Moreover, we confirmed that the developed parallel-GPU computation combined with the multiple mesh and time step method could achieve good acceleration and scaling through increasing the number of GPUs. We also confirmed that the developed method could simulate multiple dendrite growth with thermosolutal convection.

Drought has become a recurrent phenomenon in Turkey in the last few decades. Significant drought conditions were observed during years of late 1980s and the trend continued in the late 1990s. The country’s agricultural sector and water resources have been under severe constraints from the recurrent droughts. In this study, spatial and temporal dimensions of meteorological droughts in Turkey have been investigated from vulnerability concept. The Standardized Precipitation Index (SPI) method was used to detail geographical variations in the drought vulnerability based on frequency and severity of drought events at multiple time steps. Critical (threshold) rainfall values were derived for each station at multiple-time steps in varying drought categories to determine least amount of rainfall required to avoid from drought initiation. The study found that drought vulnerability portrays a very diverse but consistent picture with varying time steps. At regional scale, south-eastern and eastern Anatolia are characterized with moderate droughts at shorter time steps, while the occurrence of severe droughts at shorter time steps is observed at non-coastal parts of the country. A similar picture was observed with very severe droughts. The critical (threshold) values exhibited rising numbers during the growing season at 3-month step in the South-eastern Anatolia, which might have significant consequences considering presence of large irrigation projects under-development in the region. In general, rainfall amounts required for non-drought conditions decrease from the coastal parts toward the interiors with increasing time steps.

We have empirically tested limits of the magnitude of multiple time steps in molecular dynamics simulations of aqueous systems, and the extent to which these offer a means to shorten computation time. Three different steps were employed, δ0t for calculation of “bonded” forces, δ1t for calculations of short-range (< 6 Å) non-bonded forces, and δ2t for long-range (< 10 Å) non-bonded forces. Each longer step was a multiple of the shortest one. The leap-frog integration algorithm was used with SHAKE for restraint of all bond lengths and water molecules. For a system of SPC water molecules, calculation of short-range non-bonded forces could be done with a time step δ1t = 10 fs, without appreciable change of the average temperature and energy, radial distribution function or diffusion coefficient. These properties were found to be insensitive to the inclusion of long-range non-bonded forces. A multiple-step protocol with δ0t = 2, δ1t = 4 and δ2t = 16 fs has been compared with a single-step procedure with δt 2 fs for small polypeptides in water. The exploration of conformation space, with crossing of low energy barriers, was tested with the glycine dipeptide and was found to proceed at similar rates. Mean, hysteresis and statistical error of the free energy for changing alanine to α-amino butyric acid in the dipeptide, calculated by the slow-growth method, proved independent of the cutoff distance or exact protocol, within 1 kJ/mol. In conclusion, we recommend, instead of use of a single time step of 2 fs at a 10 Å cutoff, use of a time step δt = 4 fs for short-range nonbonded forces and a time step δ2t = 16 fs for long-range nonbonded forces for a 60% reduction of computation time.

We compare several established approaches for propagating wave functions and calculating hopping probabilities within the fewest switches surface hopping (FSSH) algorithm for difficult cases with many electronic states and many trivial crossings. If only a single time step (Δtc) is employed, we find that no published approach can accurately capture the dynamics correctly unless Δtc → 0 (which is not computationally feasible). If multiple time steps are employed, for a fixed classical time step (Δtc), a robust scheme can be found for dynamically choosing quantum time steps (δtq1 and δtq2) and calculating hopping probabilities so that one can systematically reduce all errors and achieve maximally efficient accuracy; scattering calculations confirm that one can choose a fairly large classical time step. The robust scheme presented here uses both the "local diabatic" and adiabatic interpolation and thus borrows elements from both the Granucci/Persico and Meek/Levine algorithms. Our findings should be broadly applicable in the future.

Joint A-SNN: Joint training of artificial and spiking neural networks via self-Distillation and weight factorization

In molecular dynamics (MD) calculations, reduction in calculation time per MD loop is essential. A multiple time-step (MTS) integration algorithm, the RESPA (Tuckerman and Berne, J. Chem. Phys. 1992, 97, 1990-2001), enables reductions in calculation time by decreasing the frequency of time-consuming long-range interaction calculations. However, the RESPA MTS algorithm involves uncertainties in evaluating the atomic interaction-based pressure (i.e., atomic pressure) of systems with and without holonomic constraints. It is not clear which intermediate forces and constraint forces in the MTS integration procedure should be used to calculate the atomic pressure. In this article, we propose a series of equations to evaluate the atomic pressure in the RESPA MTS integration procedure on the basis of its equivalence to the Velocity-Verlet integration procedure with a single time step (STS). The equations guarantee time-reversibility even for the system with holonomic constrants. Furthermore, we generalize the equations to both (i) arbitrary number of inner time steps and (ii) arbitrary number of force components (RESPA levels). The atomic pressure calculated by our equations with the MTS integration shows excellent agreement with the reference value with the STS, whereas pressures calculated using the conventional ad hoc equations deviated from it. Our equations can be extended straightforwardly to the MTS integration algorithm for the isothermal NVT and isothermal-isobaric NPT ensembles. © 2017 Wiley Periodicals, Inc.

Spiking neural networks (SNNs) are considered as a potential candidate to overcome current challenges, such as the high-power consumption encountered by artificial neural networks (ANNs); however, there is still a gap between them with respect to the recognition accuracy on various tasks. A conversion strategy was, thus, introduced recently to bridge this gap by mapping a trained ANN to an SNN. However, it is still unclear that to what extent this obtained SNN can benefit both the accuracy advantage from ANN and high efficiency from the spike-based paradigm of computation. In this article, we propose two new conversion methods, namely TerMapping and AugMapping. The TerMapping is a straightforward extension of a typical threshold-balancing method with a double-threshold scheme, while the AugMapping additionally incorporates a new scheme of augmented spike that employs a spike coefficient to carry the number of typical all-or-nothing spikes occurring at a time step. We examine the performance of our methods based on the MNIST, Fashion-MNIST, and CIFAR10 data sets. The results show that the proposed double-threshold scheme can effectively improve the accuracies of the converted SNNs. More importantly, the proposed AugMapping is more advantageous for constructing accurate, fast, and efficient deep SNNs compared with other state-of-the-art approaches. Our study, therefore, provides new approaches for further integration of advanced techniques in ANNs to improve the performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic computing.

Landscape connectivity plays a key role in determining the persistence of species inhabiting fragmented habitat patches. In dynamic landscapes, most studies measure connectivity at multiple time steps, but pay less attention to explicitly quantifying its temporal dynamics to gain insights into its role in biodiversity patterns, thereby enabling more effective operational outcomes. This article aimed at making an overview of the existing methods for the assessment of the temporal dynamics of connectivity. By analysing their differences and possible applications, we aimed to highlight knowledge gap and future research directions. We conducted a systematic review of literature dealing with the assessment of the temporal dynamics of connectivity and obtained 32 studies. We presented two main approaches based on graph theory and compared them from conceptual and operational perspectives. The first widely used approach, accounting only for the spatial dispersal of organisms, quantifies temporal changes in spatial connectivity. Based on two or multiple time steps in the time series, this approach enables assessment of the sense and magnitude of the temporal changes in spatial connectivity. The second recently developed approach quantifies spatio-temporal connectivity, thus accounting for both spatial and temporal dispersal. So far, this holistic assessment of spatio-temporal connectivity only covers two time steps. Existing methods for the assessment of the temporal dynamics of connectivity provide indicators to advance our understanding of biodiversity patterns, and to be able to implement measures to conserve and restore connectivity. We propose future directions to develop these methods.

S4-KD: A single step spiking SiamFC+ + for object tracking with knowledge distillation.

S[formula omitted]NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks

Convolutional neural networks (CNNs) have been widely used for image recognition and classification in recent years. Low energy consumption is crucial in the circuit design of edge devices, and data reuse is one method of reducing energy consumption. In addition, spiking neural networks (SNNs) are receiving increasing attention due to their low power use. However, the temporal characteristic of SNNs causes repeated data access at different time steps, leading to high energy consumption. In this paper, a spiked-based CNN accelerator that can support various inference time steps and models is proposed. Spatiotemporal parallel data flows are employed to reuse data from different time steps and, convolution operations are used to reduce energy consumption. Furthermore, the accelerator is designed for high sparsity and event driven SNNs. The synthesis achieves power efficiency of 62.45 TOPS/W and area efficiency of 7.58 TOPS/kmm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> .

This paper describes some model improvements developed to allow new possibilities in reservoirs representation with simulators using a finite difference scheme. It includes: possibility of flow between any grid block and from one to many blocks;a multiple time step calculation which allows to solve flow equations with several time step lengths for different group of grid blocks. These facilities provide a flexible and performant way of handling irregular and complex grid patterns with different nested levels of refinements. They are particularly useful to accurately simulate: –unconfined pilots or areas with smaller wells spacing;–faulted reservoirs;–vanishing or discordant geological layers;–reservoirs connected to a general aquifer. The new approach has been tested on a black oil simulator, implicit in pressure, implicit in saturation. The solutions used for some important aspects of model design are outlined. These are the Input/Output data organisation, the inner model grid pattern representation, the matrix solver, the multiple time steps selector and the associated equation solver. Two examples illustrate the use of the modified black oil simulator, and results are presented. The first example concerns the simulation of a repeated five spot injection. This test is made to check the local mesh refinements efficiency. The second example is a full field reservoir study for which using the modified model is suitable. The performance of a produced oil reservoir connected to an aquifer is simulated. In both cases, results obtained with a composite grid and the modified model are compared to those obtained with a coarse grid and then a fine grid used on a conventional model. The following points are outlined: –the use of local grid refinements give results very closed to those obtained with the fine gridded conventional model;–on the computers used, the running time of the modified model simulations is much lower than the time needed for the conventional model. Moreover, it is shown that the modified model results remain unchanged when the multiple time step selector is used. A different time step length is then automatically calculated for the oil reservoir and the aquifer, thus allowing a noticeable extra gain of processing running time.