Towards Safe and Trustworthy NLP Models: Understanding Performance Disparities in Computational Social Science

Date
2023
Authors
Lwowski, Brandon Joseph
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

The field of Natural Language Processing (NLP) has seen rapid growth in recent years due to increased access to big data through social media platforms. The ability of NLP models to model societal patterns has enormous potential to positively impact communities. Computational social scientists have already used NLP models for public health surveillance, virus tracking, and event detection, providing evidence of the efficacy of NLP in these applications. However, when applying NLP models in real-world applications that have the potential to impact decision- making in certain populations, researchers must be aware of potential biases and vulnerabilities in the models. Evidence has shown that NLP models can learn systematic and social biases that are present in historical data, leading to disparities in performance and potential harm to certain subpopulations. This multi-essay thesis explores the potential of NLP models to positively impact society, while also highlighting the need to understand and mitigate performance disparities across different subpopulations. The four essays presented aim to further increase the trust and safety of NLP models by demonstrating real-world use cases where the models can accurately impact society, as well as analyzing performance differences across different subpopulations. The overall goal is two-fold: to demonstrate the potential of NLP models to positively impact society, while also highlighting the need to understand and mitigate performance disparities. By presenting real-world examples and conducting in-depth analysis, this thesis hopes to increase trust and confidence in the use of NLP models in decision-making.

Description
This item is available only to currently enrolled UTSA students, faculty or staff.
Keywords
Natural Language Processing, Public health surveillance, Social biases
Citation
Department
Information Systems and Cyber Security