Contextual Bandits with Delayed Feedback and Semi-supervised Learning (Student Abstract)

Next generation high data rate applications such as High-Definition (HD) video transmission place a heavy load on wireless network resources. To achieve the strict quality of service requirements of HD video in a multi-user network requires efficient resource allocation. Rate-distortion theory provides an appropriate basis for this. The Nash Bargaining Solution (NBS) is proven to generate an optimal solution to a rate-distortion based convex resource allocation problem. Particle Swarm Optimization (PSO) is a meta-heuristic optimizer capable of generating near-optimal solutions. PSO, however, is not constrained to convex optimization problems. The motivation of this work is to demonstrate a PSO implementation capable of achieving an optimal solution to the resource allocation problem. The capability of the PSO approach to reach an optimal solution suggests its potential for resource allocation in a network of heterogeneous applications where convex optimization methods do not apply.

Problem statement. Due to their mobility, flexibility, ease of deployment, and low cost, Autonomous Aerial Vehicle (AAV) play an important role in future wireless networks. However, their pratical implementation faces challenges, including energy constraints, dynamic channel variations, interference management, and the need for efficient resource allocation to ensure seamless connectivity for ground users. Traditional optimization methods often fail to adapt to these complexities in real-time, limiting the effectiveness of AAV-assisted wireless networks. The aim of the work provides a comprehensive review of resource allocation in AAV-assisted wireless networks, focusing on power and bandwidth optimization strategies, as well as the key challenges in ensuring efficient and reliable communication. Methods: used in this study include a systematic review of existing literature, analyzing optimization approaches such as Game Theory, Artificial Intelligence for efficient resource allocation in AAV-assisted wireless networks. Novelty: this study analyzes resource allocation challenges in AAV-assisted network, focusing on the interdependence of power and bandwidth allocation. It explores optimization techniques like Game Theory, and Artificial Intelligence. Results. The analysis in this paper demonstrates that Game Theory and Artificial Intelligence based approaches significantly improve resource allocation efficiency. Additionally, the study identifies key challenges, including heterogeneous density network, security concerns, and complex channel modeling, providing insights for future research. Practical / Theoretical Relevance: This study advences the theoretical understanding of resource allocation in AAV-assisted wireless networks by integrating optimization strategies from Game Theory and Artificial Intelligence. Pratically, it provides insight into enhancing network efficiency, adaptability and security, making AAV-based communication more viable for real-world applications, such as disaster recovery, remote areas coverage, and IoT data collection.

An overview of game-theoretic approaches to energy-efficient resource allocation in wireless networks is presented. Focusing on multiple-access networks, it is demonstrated that game theory can be used as an effective tool to study resource allocation in wireless networks with quality-of-service (QoS) constraints. A family of non-cooperative (distributed) games is presented in which each user seeks to choose a strategy that maximizes its own utility while satisfying its QoS requirements. The utility function considered here measures the number of reliable bits that are transmitted per joule of energy consumed and, hence, is particulary suitable for energy-constrained networks. The actions available to each user in trying to maximize its own utility are at least the choice of the transmit power and, depending on the situation, the user may also be able to choose its transmission rate, modulation, packet size, multiuser receiver, multi-antenna processing algorithm, or carrier allocation strategy. The best-response strategy and Nash equilibrium for each game is presented. Using this game-theoretic framework, the effects of power control, rate control, modulation, temporal and spatial signal processing, carrier allocation strategy and delay QoS constraints on energy efficiency and network capacity are quantified.

Resource allocation and admission control algorithm based on non-cooperation game in wireless mesh networks

Проблематика. Сьогодні мобільний зв'язок зайнятий наданням IP-сервісів та передаванням мультимедійного контенту з одного місця в інше, але вже завтра нове покоління 5G зможе контролювати широкий спектр матеріальних об'єктів у реальному часі із лише незначним втручанням людини. Створюється так званий Інтернет речей «IoT» із безліччю нових привабливих програм. У поданій роботі розглянутий новий підхід до розподілу ресурсів для майбутніх безпроводових локальних мереж у складі мереж 5G.
\nМета досліджень. DIDO (Distributed Input Distributed Output) – нова технологія, яка направлена на надання послуг гнучкої багатокористувацької безпроводової мережі в усьому світі у межах міжнародного узгодження і конвенцій для використовуваних частот серед інших мобільних технологій для інтеграції з 5G. Техніка DIDO націлена на розподіл ресурсів для майбутніх мереж безпроводового зв'язку, які можуть використовувати широкий спектр «нетрадиційних» частот від 3 МГц до 30 ГГц і працювати незалежно одна від одної із повною пропускною спроможністю і високою швидкістю передачі даних на відстанях від 100 м до 1000 км.
\nМетодика реалізації. Був проведений аналіз усіх відомих публикацій, які присвячені новому поколінню мобільного зв'язку 5G. Досліджувались аспекти взаємодії 5G із WLAN, BT, WSN та іншими мікро- и пікорадіомережами. Розглянутий новий підхід до розподілу частотних ресурсів для майбутньої WLAN.
\nРезультати досліджень. Представлений огляд, а також обговорюються основні напрямки розробки майбутніх мереж мобільного радіозв'язку 5G за стандартом IMT 2020. Вони включають, окрім іншого, перспективну технологію DIDO (Distributed Input Distributed Output) для розподілу ресурсів для майбутніх мереж безпроводового зв'язку. Зазначена технологія повинна тільки розширити існуючу інфраструктуру мобільного радіозв'язку GERAN, UTRAN, SAE і IMS попередніх поколінь 2G – 4G за допомогою гнучкої «всесвітньої» WLAN, яка працює з використанням баз даних для доступних діапазонів частот та з доступом до Web-контенту на основі так званих DIDO-центрів (DIDO Data Centers). 
\nВисновки. Эфективні WLAN, у тому числі із використанням техноології DIDO, повинні стати частиною нових систем покоління 5G. У якості нового метода розподілу ресурсів для майбутніх безпроводових локальних мереж, з точки зору деяких провідних компаній, таких як Rearden, Microsoft, Samsung, Huawei, має бути використаний метод DIDO.

In wireless networks many nodes contend for available resources creating a challenge in resource allocation. With shared resources, fairness in allocation is a serious issue. Fairness metrics have been defined to measure the fairness level of resource decisions in allocations. Therefore, fairness metrics significantly influence network and node performances emphasizing the need for due diligence to fairness metrics. It can be seen that fairness metrics in many of the allocation strategies and algorithms in the literature are not analyzed in depth to show the overall fairness of allocations from different perspectives. Hence, we propose a series of fairness metrics for resource allocation in wireless networks, which evaluate individual, system, short and long term fairness. Our metrics are general enough to be adapted in either multiple or single resource sharing scenarios. Algorithms using these metrics for channel allocation problem in peer-to-peer wireless regional area network is proposed and simulated, and the results show that our metrics in conjunction with smart channel allocation strategies guarantee both the fairness and performances.

In this paper, a cross-layer resource allocation mechanism is proposed for wireless multimedia service.In particular, a game theory based on quality of service (QoS) for multimedia users is introduced to deal with the fairness of network resource allocation in wireless networks.Moreover, the channel states of wireless users are additionally regarded under the cross-layer design in WiMAX environment.In details, the bargaining solution is adopted to discover the efficient and fair resource allocation strategy for multimedia service in considering QoS in the peak signal-to-noise ratio (PSNR) and the channel states in the carrier-to-interface ratio (CINR).The proposed mechanism is illustrated and evaluated by simulation results of transmitting video sequences in WiMAX environment.

This paper presents a comprehensive approach to solve the problem of joint dynamic resource allocation (JDRA) in heterogeneous wireless networks using a Hopfield neural network (HNN). A generic formulation for packet services with delay constraints is proposed to decide the optimal bit rate and radio access technology (RAT) allocation. Some illustrative simulations results in a basic scenario are presented to evaluate performance of the proposed algorithm.

Personalization has become a focal point of modern revenue management. However, it is often the case that minimal data are available to appropriately make suggestions tailored to each customer. This has led to many products making use of reinforcement learning-based algorithms to explore sets of offerings to find the best suggestions to improve conversion and revenue. Arguably the most popular of these algorithms are built on the foundation of the multi-arm bandit framework, which has shown great success across a variety of use cases. A general multi-arm bandit algorithm aims to trade-off adaptively exploring available, but under observed, recommendations, with the current known best offering. While much success has been achieved with these relatively understandable procedures, much of the airline industry is losing out on better personalized offers by ignoring the context of the transaction, as is the case in the traditional multi-arm bandit setup. Here, we explore a popular exploration heuristic, Thompson sampling, and note implementation details for multi-arm and contextual bandit variants. While the contextual bandit requires greater computational and technical complexity to include contextual features in the decision process, we illustrate the value it brings by the improvement in overall expected

In this paper, we propose a new contextual bandit problem with two objectives, where one of the objectives dominates the other objective. Unlike single-objective bandit problems in which the learner obtains a random scalar reward for each arm it selects, in the proposed problem, the learner obtains a random reward vector, where each component of the reward vector corresponds to one of the objectives. The goal of the learner is to maximize its total reward in the non-dominant objective while ensuring that it maximizes its reward in the dominant objective. In this case, the optimal arm given a context is the one that maximizes the expected reward in the non-dominant objective among all arms that maximize the expected reward in the dominant objective. For this problem, we propose the multi-objective contextual multi-armed bandit algorithm (MOC-MAB), and prove that it achieves sublinear regret with respect to the optimal context dependent policy. Then, we compare the performance of the proposed algorithm with other state-of-the-art bandit algorithms. The proposed contextual bandit model and the algorithm have a wide range of real-world applications that involve multiple and possibly conflicting objectives ranging from wireless communication to medical diagnosis and recommender systems.

Cross-layer design approaches are critical for efficient utilization of the scarce radio resources with QoS provisioning in the third-generation wireless networks and beyond. Better system performance can be obtained from information exchanges across protocol layers, which may not be available in the traditional layering architecture. This article provides an overview of cross-layer design approaches for resource allocation in 3G CDMA networks, summarizes state-of-the-art research results, and suggests further research issues. In addition, a cross-layer design approach for real-time video over time-varying CDMA channels is proposed, where link layer resource allocation benefits from information in both the application and physical layers. Simulations results are given to demonstrate the effectiveness of the proposed approach.

A novel optimization framework for resource allocation in wireless networks and radar systems is proposed, which merges the methods of maximum block improvement (MBI) and of sequential optimization. A detailed convergence proof is provided, showing that the proposed algorithm is able to monotonically increase the objective value while ensuring that every limit point of the generated variable sequence fulfills the problem first-order optimality conditions under very mild hypothesis. These results extend available convergence results on MBI and sequential optimization, significantly widening the range of applications that can be handled by the proposed framework compared to available approaches. This point is illustrated in detail presenting relevant applications from both the cellular and radar context, which fall under the umbrella of the developed optimization method.

The paper describes a Resource allocation in cellular networks using orthogonal frequency division multiple access (OFDMA) systems. Both center cell and edge cell users using same power, edge cell users suffer from Inter Cell Interference (ICI). Power allocation cell-edge user's power is maximized automatically reduce the inter cell interference under the condition that desirable performance for cell-center users must be maintained. The power optimization is formulated as a Greedy Power allocation method. Simulation results indicate that our proposed scheme can achieve significantly balanced performance improvement between cell-edge and cell-center users in multi-cell networks, and goal of future wireless networks in terms of providing high performance to anyone from anywhere.

— This article presents a time-aware admission control and resource allocation scheme in wireless networks, in the context of a future generation cellular network. The quality levels (and their respective utility) of different connections are specified using discrete resource-utility (R-U) functions. The scheme uses these R-U functions for allocating and reallocating bandwidth to connections, aiming to maximise the accumulated utility of the system. However, different applications react differently to resource reallocations. Therefore at each allocation time point the following factors are taken into account: the age of the connection, a disconnection (drop) penalty and the sensitiveness to reallocation frequency. The evaluation of our approach shows a superior performance compared to a recent adaptive bandwidth allocation scheme (RBBS). In addition we have studied the overhead that performing a reallocation imposes on the infrastructure. To minimise this overhead, we present an algorithm that efficiently reduces the number of reallocations, while remaining within a given utility bound.

Resource allocation in wireless networks, i.e., assigning time and frequency slots over specific terminals under spatio-temporal constraints, is a fundamental and challenging problem. Belief Propagation/message passing (inference) algorithms have been proposed for constraint satisfaction problems (CSP), since they are inherently amenable to distributed implementation. This work compares two message passing algorithms for time and frequency allocation, satisfying signal-to-interference-and-noise-ratio, half-duplex-radio operation and routing constraints. The first method periodically checks whether the constraints are satisfied locally and restarts specific messages, when the local constraints (encoded in corresponding factors) are not satisfied. The second method stochastically perturbs Belief Propagation, using Gibbs sampling. The methods are evaluated, based on how often they fail to converge to a valid (i.e., constraint-satisfying) allocation, coined as outage probability. Numerical results demonstrate that, as the maximum number of iterations increase, both methods decrease the outage probability. However, the restarting method offers faster convergence to a valid CSP solution. Future work will focus on next generation 5/6G wireless networks.