Reinforcement Learning Based Relay Selection for Underwater Acoustic Cooperative Networks

This paper investigates the impact of relay selection (RS) on the performance of cooperative non-orthogonal multiple access (NOMA), where relays are capable of working in either full-duplex (FD) or half-duplex (HD) mode. A number of relays (i.e., K relays) are uniformly distributed within the disc. A pair of RS schemes are considered insightfully: 1) single-stage RS (SRS) scheme; and 2) two-stage RS (TRS) scheme. In order to characterize the performance of these two RS schemes, new closed-form expressions for both exact and asymptotic outage probabilities are derived. Based on analytical results, the diversity orders achieved by the pair of RS schemes for FD/HD cooperative NOMA are obtained. Our analytical results reveal that: 1) the FD-based RS schemes obtain a zero diversity order, which is due to the influence of loop interference at the relay; and 2) the HD-based RS schemes are capable of achieving a diversity order of K , which is equal to the number of relays. Finally, simulation results demonstrate that: 1) the FD-based RS schemes have better outage performance than HD-based RS schemes in the low signal-to-noise ratio (SNR) region; 2) as the number of relays increases, the pair of RS schemes considered are capable of achieving the lower outage probability; and 3) the outage behaviors of FD/HD-based NOMA SRS/TRS schemes are superior to that of random RS and orthogonal multiple access based RS schemes.

The achievement of efficient data transmissions via underwater acoustic channels, while dealing with large data packets and real-time data fed by underwater sensors, requires a~high data rate. However, diffraction, refraction, and reflection phenomena, as well as phase and amplitude variations, are common problems experienced in underwater acoustic (UWA) channels. These factors make it difficult to achieve high-speed and long-range underwater acoustic communications. Due to multipath interference caused by surface and ocean floor reflections, the process of modeling acoustic channels under the water's surface is of key importance. This work proposes a~simple geometry-based channel model for underwater communication. The impact that varying numbers of reflections, low water depth values, and distances between the transmitter and the receiver exert on channel impulse response and transmission loss is examined. The high degree of similarity between numerical simulations and actual results demonstrates that the proposed model is suitable for describing shallow underwater acoustic communication environments.

The fast temporal variations, long multipath delay spreads, and severe frequency-dependent attenuations of underwater acoustic communication channels are extremely complex that impedes underwater acoustic (UWA) data transmission. To alleviate this problem, channel coding is indispensable in UWA communication system to increase the reliability. In this paper, performances of LDPC codes with different parameters over different underwater acoustic communication channels are presented, which is to know how to adjust the encoding and decoding parameters according to different underwater acoustic channels. The results indicate that LDPC codes have a good performance of the estimated BER on the order of 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> -10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-5</sup> either in positive sound velocity gradient (PSVG) underwater channel or negative sound velocity gradient (NSVG) one, even invariable sound velocity gradient (ISVG) one. In addition, whatever the encoding or decoding parameters is adopted, performance of the LDPC codes over the NSVG underwater channel is the best, compared with PSVG one and ISVG one. Furthermore, in the desired performance of BER of 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-5</sup> , the encoding and decoding parameters according to the three different underwater channels specifications are summarized.

Under water acoustic (UAC) channel is subjected to large delay spread compared to radio frequency (RF) channels. In order to capture this large delay spread and avoid ISI, the UAC symbol duration needs to be considerably large reducing the transmission data rate. Unlike other RF systems, in Ultra-wideband (UWB) RF systems, small pulse width in the order of nano seconds is used to transmit information. This in turn, gives high resolution multipaths. In a time hopping UWB system, to avoid multipath overlap, the chip duration and the frame duration are kept equal to or larger than the maximum delay spread of the channel. This results in large UWB symbol duration and hence reduces the data rate. In order to increase the data rate, the chip duration, and frame duration can be reduced to less than the maximum delay spread. In addition, to farther increase the data rate, guard interval between consecutive symbols can be reduced/removed allowing consecutive symbols to overlap. This results in intersymbol interference (ISI). Surprisingly, this type of overlap doesn't degrade UWB system performance significantly. In this paper, the same concept is applied to various UAC channels. In order to increase the data rate, consecutive UAC symbols are allowd to overlap. Performance under ISI is then evaluated.

In this paper, the estimation of doubly spread underwater acoustic (UWA) channels is investigated. The UWA channels are characterized by severe delay spread and significant Doppler effects, and can be well modeled as a multi-scale multi-lag (MSML) channel. Furthermore, exploiting the sparsity of UWA channels, MSML channel estimation can be transformed into the estimation of parameter sets (amplitude, Doppler scale factor, time delay). Based on this, the orthogonal matching pursuit (OMP) algorithm has been widely used. However, the estimation accuracy of OMP depends on the size of the dictionary, which is related with both delay spread and Doppler spread. Thus it requires high computational complexity. This paper proposes a new method, called wavelet transform (WT) based algorithm, for the UWA channel estimation. Different from OMP algorithm which needs to search in both time domain and Doppler domain, WT-based algorithm only needs to search in time domain by using the Doppler invariant characteristic of hyperbolic frequency modulation (HFM) signal. The performance of the proposed algorithm is evaluated by computer simulations based on BELLHOP. The simulation results show that WT-based algorithm performs slightly better than OMP algorithm in low signal to noise ratio (SNR) while can greatly reduce computational complexity.

In recent years, the MIMO-based underwater acoustic (UWA) wireless sensor networks have attracted significant research attention because of its wide and critical applications in marine research, oceanography, marine commercial operations, offshore oil industry, and defense systems. The cooperative-communications-based wireless relay networks have been widely cited as a very efficient technique to enhance the transmission and lifespan performances of the wireless networks. However, how to make the energy-efficient relay selection in the MIMO-based UWA wireless sensor networks, particularly under the harsh underwater wireless environments with the stringent power supply constraint, have been not well understood, neither thoroughly studied. To overcome the above problems, in this paper we propose the optimal relay selection scheme to minimize the energy consumption while guaranteeing the quality of service (QoS) of each link over the MIMO-based UWA cooperative wireless sensor networks. We use the Finite-state Markov chain to model the UAW channel. Moreover, we also incorporate MIMO multiplexing-gain and diversity-gain techniques, as well as residual relay energy in the relay selection process. We implement our proposed optimal relay selection scheme by the linear programming (LP) technique. We conduct the simulations to validate and evaluate our proposed scheme, showing its superlatives over the other existing schemes.

In this paper, we consider relay selection based decode-and-forward (DF) cooperative communication over free space optical (FSO) links. A relay selection scheme is considered in which source selects a relay on the basis of highest instantaneous signal-to-noise ratio (SNR) of the source-relay links. We derive the cumulative distribution function (CDF) and probability density function (PDF) of the highest instantaneous SNR of Gamma-Gamma distributed source-relay links. The average error performance of the DF protocol with relay selection over the Gamma-Gamma fading channels is derived in terms of power series. Coding gain and diversity order of the relay selection based DF scheme are analytically obtained. It is inferred by the analysis and simulation that the performance of the DF FSO system can be significantly improved by using the simple relay selection scheme considered in this paper.

The underwater acoustic (UWA) channel causes large propagation delays and reduces the bit error rate (BER) of wireless communication systems. The t-distribution is the optimal distribution to perform UWA noise. In this study, polar-coded differential chaos shift keying (DCSK) and quadrature chaos shift keying (QCSK) communication with UWA noise are considered. First, we have proposed a PDF for the UWA noise channel, and based on this PDF, the theoretical BER is derived. Second, polar coding’s performance is determined to demonstrate the improvement in the BER performance compared to the uncoded UWA system by means of Monte Carlo simulations. The experimental results prove that the nearest model that is applicable to the UWA channel is a t-distribution with five and six degrees of freedom. The BER formulas of the proposed systems are derived and compared with the simulation results. The results confirm the performance improvement of the polar-coded chaotic modulation systems over uncoded systems in UWA channels.

In this study, the authors analyse the outage performance of multirelay decode‐and‐forward cooperative networks subject to two joint practical issues of wireless communications, namely, energy constraints and transceiver hardware impairment (HI). To deal with energy constraints at relay nodes, radio‐frequency (RF) energy harvesting (EH) technique is adopted. Considering the joint impacts of RF EH technique and HI, two relay selection schemes, namely, harvested energy‐based relay selection (HEbS) scheme and channel quality‐base relay selection (CQbS) scheme are proposed. Tight closed‐form approximate expressions for the outage probability of each scheme are derived and corroborated through Monte Carlo simulations. Some representative performance comparisons are carried out and show that the diversity order achieved by the HEbS scheme is 1 and is independent of the number of relays, K , whereas that achieved by the CQbS scheme is K , which is full diversity.

Cooperative communication is regarded as a key technology in wireless networks, including cognitive radio networks (CRNs), which increases the diversity order of the signal to combat the unfavorable effects of the fading channels, by allowing distributed terminals to collaborate through sophisticated signal processing. Underlay CRNs have strict interference constraints towards the secondary users (SUs) active in the frequency band of the primary users (PUs), which limits their transmit power and their coverage area. Relay selection offers a potential solution to the challenges faced by underlay networks, by selecting either single best relay or a subset of potential relay set under different design requirements and assumptions. The best relay selection schemes proposed in the literature for amplify-and-forward (AF) based underlay cognitive relay networks have been very well studied in terms of outage probability (OP) and bit error rate (BER), which is deficient in multiple relay selection schemes. The novelty of this work is to study the outage behavior of multiple relay selection in the underlay CRN and derive the closed-form expressions for the OP and BER through cumulative distribution function (CDF) of the SNR received at the destination. The effectiveness of relay subset selection is shown through simulation results.

In this study, the performance of wirelessly powered relay‐aided non‐orthogonal multiple access networks is investigated in terms of outage probability. Specifically, two relay selection strategies, i.e. two‐stage relay selection (TRS) and maximum energy harvesting relay selection (MEHS), and two energy harvesting scenarios, i.e. time switching (TS) and power splitting (PS) are considered. In each setup, outage probabilities' analytical expressions and their asymptotics are derived. Monte‐Carlo simulations are also carried out to verify the correctness of the analysis. The results show that regardless of relay selection and energy harvesting strategies, increasing transmit power can improve the proposed system performance. However, TRS can achieve a full diversity order, while MEHS has a unit diversity order. Besides, the results recommend parameter selections of PS and TS coefficients for optimal performance in term of outage probability.

In this paper, optimal power allocation and relay selection strategies in energy harvesting cooperative wireless networks are studied. In particular, signal‐to‐noise ratio (SNR)‐maximizing based power allocation and relay selection without and with energy cooperation—via wireless energy transfer—are considered. Moreover, total relay power minimization subject to target end‐to‐end SNR is investigated. The different optimal strategies are formulated as optimization problems, which are non‐convex. Thus, intelligent transformations are applied to transform non‐convex problems into convex ones, and polynomial‐time solution procedures are proposed. Simulation results illustrate that power allocation strategies achieve higher end‐to‐end SNR than relay selection ones. Finally, energy cooperation is shown to be effective in improving end‐to‐end SNR, while total relay power minimization balances end‐to‐end SNR, transmit power consumption, and harvested energy. Copyright © 2016 John Wiley &amp; Sons, Ltd.

Noise in an underwater acoustic (UWA) channel does not necessarily follow Gaussian statistics, especially in a shallow water environment which is dominated by impulsive noise sources. However most of the receivers are designed with the assumption that the channel noise is additive white Gaussian (AWGN). Such receivers may not be the optimum ones to deal with the non-Gaussian UWA noise. Several non-Gaussian statistics have been proposed in previous literature to model UWA noise among which the Generalized Gaussian (GG) model has been very popular due to its flexible parametric form. However, to the best of our knowledge, no analytical error analysis with the assumption of channel noise being generalized Gaussian has yet been reported. In this paper, we derive the analytical expression for probability of error considering the presence of GG noise in the UWA channel. We also try to study the performance of an AWGN receiver in the presence of non-Gaussian noise. We consider three different communication systems using BPSK, QPSK and M-ary PAM constellations, and observe, for all three of them, that if noise kurtosis is greater than the Gaussian kurtosis, the system performance degrades. Thus here is an initial phase of designing UWA system with GG noise assumptions instead of the traditional Gaussian receiver. It is still a matter of argument, due to all possible redesign complexities, whether it is worth taking all the redesign complexities or the performance degradation can be tolerated as per the requirements of system application.

Underwater acoustic (UWA) channel is bandwidth-limited owing to the low sound speed, severe time- and frequency-selective fading. We believe that improving the spectral-efficiency is a decent approach to increase the capacity of UWA systems. In this paper, we introduce a highly spectrum-efficient MIMO-OFDM system under UWA channel that uses blind channel estimation and elimination of phase ambiguity induced in equalization. We employ classic subspace algorithm (SPA) as the blind channel estimation method without the training sequences and pilot tones. To maximize the performance of SPA, a resampling-based solution to mitigate the channel Doppler shift is applied at the receiver. Considering the phase ambiguity introduced inherently in SPA, we use an interleaved symbol mapping schemes to eliminate the ambiguity, which achieves no rotational symmetry in the combined signal constellation. Numerical simulations verified that the phase error could be corrected without any data aiding and the spectrum efficiency of MIMO-OFDM UWA communications is greatly improved compared with pilot-based method.

In this paper, a meta-learning-based underwater acoustic (UWA) orthogonal frequency division multiplexing (OFDM) system is proposed to deal with the environment mismatch in real-world UWA applications, which can effectively drive the model from the given UWA environment to the new UWA environment with a relatively small amount of data. With meta-learning, we consider multiple UWA environments as multi-UWA-tasks, wherein the meta-training strategy is utilized to learn a robust model from previously observed multi-UWA-tasks, and it can be quickly adapted to the unknown UWA environment with only a small number of updates. The experiments with the at-sea-measured WATERMARK dataset and the lake trial indicate that, compared with the traditional UWA-OFDM system and the conventional machine learning-based framework, the proposed method shows better bit error rate performance and stronger learning ability under various UWA scenarios.