Open Access- https://doi.org/10.1109/ipdps53621.2022.00096
Task-based Acceleration of Bidirectional Recurrent Neural Networks on Multi-core Architectures
- May 1, 2022
- Robin Kumar Sharma +1 more
This paper proposes a novel parallel execution model for Bidirectional Recurrent Neural Networks (BRNNs), B-Par (Bidirectional-Parallelization), which exploits data and control dependencies for forward and reverse input computations. B-Par divides BRNN workloads across different parallel tasks by defining input and output dependencies for each RNN cell in both forward and reverse orders. B-Par does not require per-layer barriers to synchronize the parallel execution of BRNNs. We evaluate B-Par considering the TIDIGITS speech database and the Wikipedia data-set. Our experiments indicate that B-Par outperforms the state-of-the-art deep learning frameworks TensorFlow-Keras and Pytorch by achieving up to 2.34× and 9.16× speed-ups, respectively, on modern multi-core CPU architectures while preserving accuracy. Moreover, we analyze in detail aspects like task granularity, locality, or parallel efficiency to illustrate the benefits of B-Par.
- # Bidirectional Recurrent Neural Networks
- # Multi-core Architectures
- # Task Granularity
- # Model For Recurrent Neural Networks
- # Parallel Tasks
- # Acceleration Of Neural Networks
- # Reverse Orders
- # Parallel Execution
- # Reverse Computations
- # Forward Input
