Date on Master's Thesis/Doctoral Dissertation

12-2023

Document Type

Doctoral Dissertation

Degree Name

Ph. D.

Department

Electrical and Computer Engineering

Degree Program

Electrical Engineering, PhD

Committee Chair

Inanc, Tamer

Committee Member

Zurada, Jacek M.

Committee Member

Naber, John

Committee Member

Mclntyre, Michael

Committee Member

Richards, Christopher

Author's Keywords

Personalized medicine; anemia management; warfarin; adaptive model predictive control; reinforcement learning; precise dosing

Abstract

Personalized precision medicine aims to develop the appropriate treatments for suitable patients at the right time to obtain optimal results. Personalized medicine is challenging due to inter- and intra-patient variability, narrow therapeutic window, the effect of other medications, comorbidity (more than one disease at a time), nonlinear patient dynamics, and time-varying patient dose response characteristics which include bleeding (internal and external). This research aims to develop a framework for an adaptive personalized modeling and control method with minimum clinical patient specific dose response data for optimal drug dosing. The proposed methodology is applied to anemia and warfarin management. It is challenging in practice to achieve an optimal dosage of erythropoietin (EPO) to maintain Hemoglobin (Hgb) levels between 10-12 $g/dl$ in case of anemia management and the optimal dosage of warfarin to maintain an International Normalized Ratio (INR) between 2.0 to 3.0 in case of warfarin management, based on population-based models due to inter-and intra-variability of the patients. For personalized patient modeling, semi-blind robust system identification incorporates the effect of non-zero initial conditions and uses the minimum number of patient specific clinical data. The model (In)validation technique and Kalman filter are used for adaptation. Furthermore, Adaptive Model Predictive Control (AMPC), Extremum-Seeking Control (ESC), Model-Free Reinforcement Learning (MFRL), and Model-Based Reinforcement Learning (MBRL) control policies are defined for Virtual Chronic Kidney Disease (VCKD) patients. These methods are tested for the events of bleeding and missing dosages. The results conclude that data-driven adaptive control methods, such as AMPC and DQN-RL, can handle serious conditions of bleeding and missing dosage for virtual CKD patients which have a narrow therapeutic window. However, one major drawback of the MFRL methods is the requirement of a high number of patient specific data points to train the agent. This requirement is not suitable for personalized medicine. To reduce the number of patient specific data points required for training the agent, MBRL is introduced. However, MB-DQN-RL faces challenges in providing steady EPO dosages. Therefore, AMPC along with semi-blind robust model identification with Kalman filter provides a complete practical framework to provide personalized optimal dosages.

Download

Share

COinS