Towards a Rich Format for Closed Captioning

Closed-captioning is an essential part of viewing audio-visual content for many people, including those who are D/deaf and Hard-of-Hearing. Traditional closed-captioning systems generally consist of a single track of timed text that ofers limited options for personalization. Research into extending the capabilities of captioning, such as afective, poetic, and customizable captions has shown a desire among a subset of users for these features, but only in specifc contexts. However, due to the difculty in creating custom stimuli videos utilizing the custom captioning system, comparisons between systems and longitudinal studies have not been pursued. This demo paper introduces Rich Captions, a structured system that allows for a single closed-caption fle to be tagged with additional information that can then be fexibly leveraged to render diferent customizable, creative, and poetic captions from the same fle. Additionally, we introduce the Rich Caption Editor¹, a free, open-source software system designed to author, edit, and render rich captions. The system design was informed by a formative design workshop with closed-captioning researchers and advocates. The current design allows researchers to generate reproducible stimuli for closed-captioning studies. Once the design space and user preferences are better understood, the rich captioning framework could be refned to serve a general audience.