TY - GEN
T1 - Empowering Children to Create AI-Enabled Augmented Reality Experiences
AU - Zhang, Lei
AU - Zhou, Shuyao
AU - Liaqat, Amna
AU - Mak, Tinney
AU - Berengard, Brian
AU - Qian, Emily
AU - Monroy-Hernández, Andrés
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/9/27
Y1 - 2025/9/27
N2 - Despite their potential to enhance children's learning experiences, AI-enabled AR technologies are predominantly used in ways that position children as consumers rather than creators. We introduce Capybara, an AR-based and AI-powered visual programming environment that empowers children to create, customize, and program 3D characters overlaid onto the physical world. Capybara enables children to create virtual characters and accessories using text-to-3D generative AI models, and to animate these characters through auto-rigging and body tracking. In addition, our system employs vision-based AI models to recognize physical objects, allowing children to program interactive behaviors between virtual characters and their physical surroundings. We demonstrate the expressiveness of Capybara through a set of novel AR experiences. We conducted user studies with 20 children in the United States and Argentina. Our findings suggest that Capybara can empower children to harness AI in authoring personalized and engaging AR experiences that seamlessly bridge the virtual and physical worlds.
AB - Despite their potential to enhance children's learning experiences, AI-enabled AR technologies are predominantly used in ways that position children as consumers rather than creators. We introduce Capybara, an AR-based and AI-powered visual programming environment that empowers children to create, customize, and program 3D characters overlaid onto the physical world. Capybara enables children to create virtual characters and accessories using text-to-3D generative AI models, and to animate these characters through auto-rigging and body tracking. In addition, our system employs vision-based AI models to recognize physical objects, allowing children to program interactive behaviors between virtual characters and their physical surroundings. We demonstrate the expressiveness of Capybara through a set of novel AR experiences. We conducted user studies with 20 children in the United States and Argentina. Our findings suggest that Capybara can empower children to harness AI in authoring personalized and engaging AR experiences that seamlessly bridge the virtual and physical worlds.
KW - Augmented Reality
KW - Authoring Tools
KW - Human-AI Interaction
KW - Programming Education
KW - Visual Programming Languages
UR - https://www.scopus.com/pages/publications/105022979560
UR - https://www.scopus.com/pages/publications/105022979560#tab=citedBy
U2 - 10.1145/3746059.3747662
DO - 10.1145/3746059.3747662
M3 - Conference contribution
AN - SCOPUS:105022979560
T3 - UIST 2025 - Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology
BT - UIST 2025 - Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology
A2 - Bianchi, Andrea
A2 - Glassman, Elena L.
A2 - Mackay, Wendy E.
A2 - Zhao, Shengdong
A2 - Oakley, Ian
A2 - Kim, Jeeeun
PB - Association for Computing Machinery, Inc
T2 - 38th Annual ACM Symposium on User Interface Software and Technology, UIST 2025
Y2 - 28 September 2025 through 1 October 2025
ER -