Voice Profiles vs. Text Transcripts, Do Data Modality Layers in Voice Recordings Matter to Alexa Users? The Role of Modality in Voice Data Privacy

This study explores user responses to two distinctive data units /components (i.e., text transcripts and voice profiles) identified within a single data source (i.e., voice recordings). Participants were randomly assigned to one of three scenarios in which Amazon Alexa presented data used for personalization differently (i.e., voice recordings vs. voice profiles + text transcripts vs. text transcripts only). Users’ perceived information sensitivity across different data types was also surveyed. The main findings include that users distinguish voice interaction data from other types of personal information, and those who report higher data sensitivity over voice interaction data assess its privacy risks higher. Furthermore, only using text transcripts (without voice profiles) for personalization alleviated users’ perceived privacy risk. This study informs future privacy implications for data transparency and design to incorporate modality differences in user data collected, stored, and processed towards personalization of voice-enabled CUIs (like OpenAI ChatGPT and Google Gemini) that extend traditional voice assistants (like Amazon Alexa and Apple Siri).