SwinJSCC: Taming Swin Transformer for Deep Joint Source-Channel Coding

We investigate joint source channel coding (JSCC) for wireless image transmission over multipath fading channels. Inspired by recent works on deep learning based JSCC and model-based learning methods, we combine an autoencoder with orthogonal frequency division multiplexing (OFDM) to cope with multipath fading. The proposed encoder and decoder use convolutional neural networks (CNNs) and directly map the source images to complex-valued baseband samples for OFDM transmission. The multipath channel and OFDM are represented by non-trainable (deterministic) but differentiable layers so that the system can be trained end-to-end. Furthermore, our JSCC decoder further incorporates explicit channel estimation, equalization, and additional subnets to enhance the performance. The proposed method exhibits 2.5 – 4 dB SNR gain for the equivalent image quality compared to conventional schemes that employ state-of-the-art but separate source and channel coding such as Better Portable Graphics (BPG) and Low-Density Parity-Check (LDPC) schemes. The performance further improves when the system incorporates the channel state information (CSI) feedback. The proposed scheme is robust against OFDM signal clipping and parameter mismatch for the channel model used in training and evaluation.

Lightweight and efficient neural network models for joint source-channel coding (JSCC) are critical for advancing semantic communication. We propose an adaptive JSCC architecture, named Mamba-based adaptive joint source-channel coding (MAJSCC), which is explicitly designed for wireless image transmission tasks. By integrating the Mamba architecture into both the encoder and decoder, the architecture enhances local feature representation and incorporates an adaptive mechanism to flexibly adjust to diverse channel conditions and transmission rates. Furthermore, the proposed network utilizes wavelet convolution to exploit a broader spectrum of signal information during training, thereby improving its ability to capture high-resolution image details. Comprehensive experimental evaluations demonstrate that the proposed MAJSCC achieves comparable or superior performance in large-scale, high-resolution image transmission tasks. Compared with the state-of-the-art BPG + 5G LDPC-coded systems (executed on CPU), it delivers faster end-to-end encoding speeds (accelerated on graphics processing unit), with a compact model design that ensures higher efficiency than traditional CNN-based JSCC methods.

In recent years, channel state information (CSI) in WiFi 802.11n has been increasingly used to collect data pertaining to human activity. Such raw data are then used to enhance human activity recognition. Activities such as lying down, falling, walking, running, sitting down, and standing up can now be detected with the use of information collected through CSI. Human activity recognition has a multitude of applications, such as home monitoring of patients. Four deep learning models are presented in this paper, namely: a convolution neural network (CNN) with a Gated Recurrent Unit (GRU); a CNN with a GRU and attention; a CNN with a GRU and a second CNN, and a CNN with Long Short-Term Memory (LSTM) and a second CNN. Those models were trained to perform Human Activity Recognition (HAR) using CSI amplitude data collected by a CSI tool. Experiments conducted to test the efficacy of these models showed superior results compared with other recent approaches. This enhanced performance of our models may be attributable the ability of our models to make full use of available data and to extract all data features, including high dimensionality and time sequence. The highest average recognition accuracy reached by the proposed models was achieved by the CNN-GRU, and the CNN-GRU with attention models, standing at 99.31% and 99.16%, respectively. In addition, the performance of the models was evaluated for unseen CSI data by training our models using a random split-of-dataset method (70% training and 30% testing). Our models achieved impressive results with accuracies reaching 100% for nearly all activities. For the lying down activity, accuracy obtained from the CNN-GRU model stood at 99.46%; slightly higher than the 99.05% achieved by the CNN-GRU with attention model. This confirmed the robustness of our models against environmental changes.

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In this paper, we present a cross-layer approach for video transmission in wireless LANs that employs joint source and application-layer channel coding, together with rate adaptation at the wireless physical layer (PHY). While the purpose of adopting PHY rate adaptation in modern wireless LANs like the IEEE 802.11a/b is to maximize the throughput, in this paper we exploit this feature to increase the robustness of wireless video. More specifically, we investigate the impact of adapting the PHY transmission rate, thus changing the throughput and packet loss channel characteristics, on the rate-distortion performance of a transmitted video sequence. To evaluate the video quality at the decoder, we develop a cross-layer modeling framework that considers jointly the effect of application-layer joint source-channel coding (JSCC), error concealment, and the PHY transmission rate. The resulting models are used by an optimization algorithm that calculates the optimal JSCC allocation for each video frame, and PHY transmission rate for each outgoing transport packet. The comprehensive simulation results obtained with the H.264/AVC codec demonstrate considerable increase in the PSNR of the decoded video when compared with a system that employs separately JSCC and PHY rate adaptation. Furthermore, our performance analysis indicates that the optimal PHY transmission rate calculated by the proposed algorithm, can be significantly different when compared with rate adaptation algorithms that target throughput improvement. </para>

In this paper, we address the problem of QoS (Quality of Service) provisioning for multi-user video streaming over multiple heterogeneous wireless networks based on the distributed, cross-layer design framework. By jointly considering the rate allocation and the Joint Source-Channel Coding (JSCC), our proposal aims at maximizing the QoS provisioning under the given resource constraint. At first, we develop and evaluate a framework for optimal video rate allocation over multiple networks based on the observed Available Bit Rate (ABR) and the Round Trip Time (RTT) over each access network, as well as the video rate-distortion characteristics. The rate allocation is formulated as a convex optimization problem that minimizes the sum of all video streams expected distortion. Then, we propose an analytical JSCC scheme for error-resilient scalable encoded video, and integrate the JSCC with the specific rate allocation algorithm to improve the constructed video quality by optimally applying the appropriate channel coding rate given the constraints imposed by the transmission rate and the prevailing channel conditions. Objective and subjective simulation results are provided which demonstrate the effectiveness of our proposed joint scheme.

As the technique that determines the position of a target device based on wireless measurements, Wi-Fi localization is attracting increasing attention due to its numerous applications and the widespread deployment of Wi-Fi infrastructure. In this paper, we propose ConFi, the first convolutional neural network (CNN)-based Wi-Fi localization algorithm. Channel state information (CSI), which contains more position related information than traditional received signal strength, is organized into a time-frequency matrix that resembles image and utilized as the feature for localization. The ConFi models localization as a classification problem and addresses it with a five layer CNN that consists of three convolutional layers and two fully connected layers. The ConFi has a training stage and a localization stage. In the training stage, the CSI is collected at a number of reference points (RPs) and used to train the CNN via stochastic gradient descent algorithm. In the localization stage, the CSI of the target device is fed to the CNN and the localization result is calculated as the weighted centroid of the RPs with high output value. Extensive experiments are conducted to select appropriate parameters for the CNN and demonstrate the superior performance of the ConFi over existing methods.

Acquiring accurate channel state information (CSI) is essential for enabling reliable and efficient wireless transmission and reception. However, CSI is inherently stochastic, high-dimensional, and time-varying, which makes its acquisition particularly challenging. Motivated by the success of deep learning (DL) across many data modalities, this thesis explores DL-based solutions for CSI estimation, compression, and prediction.First, we study CSI estimation in full-duplex (FD) multiple-input multiple-output (MIMO) systems, where strong self-interference (SI) complicates channel acquisition. To reduce the pilot and computational burden of estimating both SI and user channels, we propose a pilot-sharing strategy together with a convolutional neural network that jointly estimates these channels.We further introduce a neural mapping that enables CSI acquisition at the transmit chain.Second, we investigate DL–based CSI compression and its limited robustness under distribution shifts. To address this issue, we adopt a full-model fine-tuning while explicitly accounting for model update signaling overhead. Specifically, we employ a spike-and-slab prior to promote sparsity in the model updates and fine-tune the pretrained network using a rate–distortion objective regularized by the update bit rate.Third, we tackle CSI prediction using a diffusion-based generative framework. The method consists of a temporal encoder that extracts latent features from past CSI and a diffusion generator that synthesizes future CSI. We also study a simplified encoder-free design to reduce latency, compare autoregressive and sequence-to-sequence inference, and explore multiple architectures for both temporal encoding and diffusion generation.

Channel state information (CSI) estimation is one of the most fundamental problems in wireless communication systems. Various methods, so far, have been developed to conduct CSI estimation. However, they usually require high computational complexity, which makes them unsuitable for 5G wireless communications due to employing many new techniques (e.g., massive MIMO, OFDM, and millimeter-Wave (mmWave)). In this paper, we propose an efficient online CSI prediction scheme, called OCEAN, for predicting CSI from historical data in 5G wireless communication systems. Specifically, we first identify several important features affecting the CSI of a radio link and a data sample consists of the information of the features and the CSI. We then design a learning framework that is an integration of a CNN (convolutional neural network) and a long short term with memory (LSTM) network. We also further develop an offline-online two-step training mechanism, enabling the prediction results to be more stable when applying it to practical 5G wireless communication systems. To validate OCEAN's efficacy, we consider four typical case studies, and conduct extensive experiments in the four scenarios, i.e., two outdoor and two indoor scenarios. The experiment results show that OCEAN not only obtains the predicted CSI values very quickly but also achieves highly accurate CSI prediction with up to 2.650-3.457 percent average difference ratio (ADR) between the predicted and measured CSI.

Indoor occupancy detection (IOD) via Wi-Fi sensing capitalizes on the varying patterns in CSI (Channel State Information) to estimate the number of people in a given area. However, the precision of such systems heavily depends on the quality of the CSI data, which can be degraded by noise and environmental factors. To address this issue, In this paper, we present a CSI preprocessing method to improve the accuracy of IOD systems using Wi-Fi sensing. Unlike existing preprocessing methods that use computationally complex signal processing or statistical techniques, we expand the dimension of CSI amplitude data into a three-channel vector through nonlinear transformation to amplify subtle differences between CSI data belonging to a different number of people. By drawing clearer boundaries between CSI data distributions belonging to a different number of people in a monitored area, our method improves the people-counting accuracy of a Wi-Fi sensing system. To ensure temporal consistency and improve data quality, we discretize the CSI measurements based on their transmission periods and aggregate consecutive measurements over a given time interval. These samples are then fed into a Convolutional Neural Network (CNN) specifically trained for the IOD task. Experimental results in diverse real-world scenarios verify that compared to the traditional methods, the enhanced feature representation capability of our approach leads to more accurate and robust sensing outcomes even in the most resource-constrained environment, where a commercial off-the-shelf CSI capture machine with only one antenna is used when a Wi-Fi sender with one transmit antenna sends packets periodically to the channel with the smallest Wi-Fi channel bandwidth.

We propose SignFi to recognize sign language gestures using WiFi. SignFi uses Channel State Information (CSI) measured by WiFi packets as the input and a Convolutional Neural Network (CNN) as the classification algorithm. Existing WiFi-based sign gesture recognition technologies are tested on no more than 25 gestures that only involve hand and/or finger gestures. SignFi is able to recognize 276 sign gestures, which involve the head, arm, hand, and finger gestures, with high accuracy. SignFi collects CSI measurements to capture wireless signal characteristics of sign gestures. Raw CSI measurements are pre-processed to remove noises and recover CSI changes over sub-carriers and sampling time. Pre-processed CSI measurements are fed to a 9-layer CNN for sign gesture classification. We collect CSI traces and evaluate SignFi in the lab and home environments. There are 8,280 gesture instances, 5,520 from the lab and 2,760 from the home, for 276 sign gestures in total. For 5-fold cross validation using CSI traces of one user, the average recognition accuracy of SignFi is 98.01%, 98.91%, and 94.81% for the lab, home, and lab+home environment, respectively. We also run tests using CSI traces from 5 different users in the lab environment. The average recognition accuracy of SignFi is 86.66% for 7,500 instances of 150 sign gestures performed by 5 different users.

This article examines the temporal processes of information transmission by response times (RTs) in visual search. Information (hazard) rate functions of RTs were evaluated for a large dataset provided by Wolfe and colleagues. Set size accounted for a minor portion of the variance of these RTs, and the present analysis examined temporal processes involved in the larger variance within each stimulus and task condition. Momentary information rates were measured by (a) response probabilities sequentially dependent on the proportion of longer RTs, (b) with continuously varying visual influence, and (c) evaluated in bits/s. Results include the following: (1) Rates of information transmission from visual input to behavioral output varied systematically with the RTs in each search condition. (2) Information rate functions for “feature” and “conjunction” tasks were highly nonlinear, with distinct phases of accelerating and decelerating visual influence. Roughly constant information rates occurred in only a few conditions in the “spatial configuration” task. (3) Faster and slower observers responded to qualitatively different visual organizations of target-absent displays in all three tasks. (4) Perceptual organization and response control were synchronized parallel processes.

With the rapid development of Location-Based Services (LBSs), research on Indoor Positioning Systems (IPS) has gained extensive attention. Although existing channel state information (CSI)-based positioning algorithms have achieved good positioning accuracy, they still face significant challenges in practical applications due to their high computational complexity and cost. To address this, this paper proposes an indoor positioning method based on the frequency fading characteristics of broadband signals, named the ACCS algorithm. This method introduces the innovative use of the autocorrelation spectrum of broadband signals to transform the indoor positioning problem into an image classification task. After that, to achieve lightweight model design, a convolutional autoencoder (CAE), which is an unsupervised learning model based on convolutional neural networks, is employed to reduce the dimension and compress the input images. Finally, a deep residual network combined with a CS-Attention module, which is composed of a channel feature extraction module and a spatial feature extraction module, is used to efficiently capture image features for more accurate position estimation. The performance of the proposed method was tested using data collected in a conference room. Experimental results show that the autocorrelation processing and attention module in the proposed algorithm significantly improve positioning accuracy, achieving a positioning error of 0.245 m in complex indoor environments. Additionally, comparisons with common positioning algorithms (AOA, W-KNN) demonstrate that our method exhibits significant advantages in positioning accuracy and robustness.

Hybrid beamformer design plays very crucial role in the next generation millimeter-wave (mm-Wave) massive MIMO (multiple-input multiple-output) systems. Previous works assume the perfect channel state information (CSI) which results heavy feedback overhead. To lower complexity, channel statistics can be utilized such that only infrequent update of the channel information is needed. To reduce the complexity and provide robustness, in this work, we propose a deep learning (DL) framework to deal with both hybrid beamforming and channel estimation. For this purpose, we introduce three deep convolutional neural network (CNN) architectures. We assume that the base station (BS) has the channel statistics only and feeds the channel covariance matrix into a CNN to obtain the hybrid precoders. At the receiver, two CNNs are employed. The first one is used for channel estimation purposes and the another is employed to design the hybrid combiners. The proposed DL framework does not require the instantaneous feedback of the CSI at the BS. We have shown that the proposed approach has higher spectral efficiency with comparison to the conventional techniques. The trained CNN structures do not need to be re-trained due to the changes in the propagation environment such as the deviations in the number of received paths and the fluctuations in the received path angles up to 4 degrees. Also, the proposed DL framework exhibits at least 10 times lower computational complexity as compared to the conventional optimization-based approaches.

The deep convolutional neural networks (CNN) have a vast amount of parameters, especially in the fully connected (FC) layers, which have become a bottleneck for real-time applications where processing latency is high due to computational cost. In this paper, we propose to optimize the FC layers in CNN via making it much slimmer. We make analysis of the statistical distribution of the weights in FC layer, and observe each column follows Gaussian distribution. Regression model analysis of the weights of FC layer based on Akaike information criteria and Bayesian information criterion demonstrates that they have Granger causality, which means the columns are correlated and they follow colored Gaussian distribution. Based on this distribution, we derive a CNN design and optimization theorem for FC layers from information theory point of view. The theorem provides two design criteria, rank and singular values. Further, we show that FC layer with weights of colored Gaussian is more efficient than that of white Gaussian. The optimization criteria is singular-values-based, so we apply singular value decomposition to find the maximal singular values and QR to identify the corresponding columns in FC layer. We evaluate our optimization approach to AlexNet and apply the slimmer CNN to ImageNet classification. Simulation results show our approach performs much better than random dropout. Specifically, with only around 28% of weights, the AlexNet could perform as well as the original AlexNet in terms of top one error and top five error.

In massive multiple-input multiple-output (MIMO) systems, the channel state information (CSI) feedback enables performance gain in frequency division duplex networks. However, with the increase in the number of antennas, the feedback overhead of CSI will also enhance. To this end, this study addresses the issue of massive MIMO CSI reconstruction using convolutional neural network (CNN), long short-term memory (LSTM) and attention mechanism, and proposes an efficient network architecture (denoted as CNN-LSTM-A). To achieve a compromise between performance and complexity, the proposed method significantly reduces the number of training parameters by utilising a single-stage network rather than a multiple-stage network. Finally, simulation results show that the authors method can reduce the feedback overhead of CSI effectively, and achieves better performance in terms of CSI compression and recovery accuracy compared with existing state-of-the-art methods.