Date on Master's Thesis/Doctoral Dissertation

8-2024

Document Type

Doctoral Dissertation

Degree Name

Ph. D.

Department

Electrical and Computer Engineering

Degree Program

Electrical Engineering, PhD

Committee Chair

Li, Hongxiang

Committee Member

Zurada, Jacek

Committee Member

Inanc, Tamer

Committee Member

Hu, Changbing

Author's Keywords

Aeronautical networks; dynamic routing; dynamic spectrum access; deep reinforcement learning

Abstract

As the airspace is experiencing an increasing number of aircraft, spectrum sharing among Air Vehicles (AVs) and Terrestrial Users (TUs) emerges as a compelling solution to improve spectrum utilization efficiency. I investigated three types of aeronautical communication including the single-hop Air-Air Communication Network (AACN), the multi-hop Air-Air Ad-hoc Network (AAAN), and the Aerial and Terrestrial Hybrid Network (ATHN). I assume a spectrum-limited scenario through all communication networks. Thus, the number of communication pairs is greater than that of the available channels, resulting in co-channel interference due to the reuse of the same channel among communication links. In a single-hop AACN, I formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q-learning-based algorithm is proposed to find the optimal solution. Specifically, I design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach makes it scalable to large networks. Finally, my experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios. In a multi-hop AAAN, I focus on the delay-sensitive packet routing problem, where spectrum access and routing decisions are jointly considered for optimal performance. I propose a Graph Attention (GAT) based deep Q-learning algorithm to find the optimal relay and channel selections that minimize the End-to-End (E2E) delay. In particular, the GAT layer is the bottom layer to produce high-level features by analyzing the sub-graph structure and channel information. Dueling Double Deep Q-Network (D3QN) is adopted to address the overestimation problem caused by Deep Q-Network (DQN). During offline training, the GAT-based learning algorithm gradually converges to the optimal solution through trial and error. During real-time execution, the trained agent is deployed across each node, enabling multiple relay nodes to concurrently determine the next-hop relay and frequency channel. Our experiments across various scenarios conclusively demonstrate the efficiency of the proposed algorithm. In an ATHN, AVs and BSs collaboratively form a multi-hop ad-hoc network to minimize the average E2E packet transmission delay. Meanwhile, the BSs and TUs constitute a terrestrial network aimed at maximizing the uplink and downlink sum capacity. Given the concept of spectrum sharing between aerial and terrestrial users in ATHN, I formulate a joint routing and capacity optimization problem, which is a multi-stage combinatorial problem subject to the curse of dimensionality. To navigate this complexity, I propose a Deep Reinforcement Learning (DRL)-based algorithm. Specifically, the D3QN structure is constructed to learn an optimal policy through trial and error. Extensive simulation results demonstrate the efficacy of my proposed solution.

Download

Share

COinS