Date on Master's Thesis/Doctoral Dissertation

8-2023

Document Type

Master's Thesis

Degree Name

M.S.

Department

Computer Engineering and Computer Science

Degree Program

Computer Science, MS

Committee Chair

Nasraoui, Olfa

Committee Co-Chair (if applicable)

Karem, Andrew

Committee Member

Karem, Andrew

Committee Member

Popa, Dan

Author's Keywords

recommender systems; deep learning; large language models; deep learning; machine learning; conversational recommender systems

Abstract

Recommendation algorithms have become an absolute necessity in the modern world to avoid information overload. However, the interaction between the human and the system is largely superficial and without any real contact. If you are given poor recommendations, you have no choice but to sift through mountains of content on your own until the model learns to accommodate your tastes more. This is bad for business as well as the consumer. Recently, large language models like ChatGPT have seen a significant rise in popularity due to their ease of use and wide range of knowledge. It has now become nearly as easy to ask ChatGPT a question as it is to search for it online yourself. Due to their domain knowledge and transferability, they have frequently been evaluated as possible tools for recommendation in recent years. However, most existing studies exploring LLM as recommendation agents focus only on a single input/output basis; neglecting one of the major benefits that LLM have expanded upon. That is, the ability to converse and respond to feedback dynamically. In this thesis we investigate how effective ChatGPT is as a recommender in its natural use case: as a conversational, direct top-n recommendation system. We build an evaluation pipeline around ChatGPT that allows us to provide iterative feedback throughout the course of a conversation to simulate how a user would actually interact with it. We additionally explore whether popularity bias is present in ChatGPT's recommendations, and how it compares against baseline models. Our findings indicate that reprompting ChatGPT with feedback on its recommendations has a signficant impact on precision. Lastly, we show that popularity bias is present in ChatGPT's recommendations but can be easily mitigated through a combination of prompt engineering and raising the temperature parameter of the model.

Download

Share

COinS