TY - GEN
T1 - MODELING MUSIC AND CODE KNOWLEDGE TO SUPPORT A CO-CREATIVE AI AGENT FOR EDUCATION
AU - Smith, Jason
AU - Truesdell, Erin J.K.
AU - Freeman, Jason
AU - Magerko, Brian
AU - Boyer, Kristy Elizabeth
AU - McKlin, Tom
N1 - Publisher Copyright:
© Jason Smith, Erin J.K. Truesdell, Jason Freeman, Brian Magerko, Kristy Elizabeth Boyer, Tom McKlin.
PY - 2020
Y1 - 2020
N2 - EarSketch is an online environment for learning introductory computing concepts through code-driven, sample-based music production. This paper details the design and implementation of a module to perform code and music analyses on projects on the EarSketch platform. This analysis module combines inputs in the form of symbolic metadata, audio feature analysis, and user code to produce comprehensive models of user projects. The module performs a detailed analysis of the abstract syntax tree of a user’s code to model use of computational concepts. It uses music information retrieval (MIR) and symbolic metadata to analyze users’ musical design choices. These analyses produce a model containing users’ coding and musical decisions, as well as qualities of the algorithmic music created by those decisions. The models produced by this module will support future development of CAI, a Co-creative Artificial Intelligence. CAI is designed to collaborate with learners and promote increased competency and engagement with topics in the EarSketch curriculum. Our module combines code analysis and MIR to further the educational goals of CAI and EarSketch and to explore the application of multimodal analysis tools to education.
AB - EarSketch is an online environment for learning introductory computing concepts through code-driven, sample-based music production. This paper details the design and implementation of a module to perform code and music analyses on projects on the EarSketch platform. This analysis module combines inputs in the form of symbolic metadata, audio feature analysis, and user code to produce comprehensive models of user projects. The module performs a detailed analysis of the abstract syntax tree of a user’s code to model use of computational concepts. It uses music information retrieval (MIR) and symbolic metadata to analyze users’ musical design choices. These analyses produce a model containing users’ coding and musical decisions, as well as qualities of the algorithmic music created by those decisions. The models produced by this module will support future development of CAI, a Co-creative Artificial Intelligence. CAI is designed to collaborate with learners and promote increased competency and engagement with topics in the EarSketch curriculum. Our module combines code analysis and MIR to further the educational goals of CAI and EarSketch and to explore the application of multimodal analysis tools to education.
UR - https://www.scopus.com/pages/publications/85183911738
UR - https://www.scopus.com/pages/publications/85183911738#tab=citedBy
M3 - Conference contribution
AN - SCOPUS:85183911738
T3 - Proceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR 2020
SP - 271
EP - 278
BT - Proceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR 2020
A2 - Cumming, Julie
A2 - Lee, Jin Ha
A2 - McFee, Brian
A2 - Schedl, Markus
A2 - Devaney, Johanna
A2 - Devaney, Johanna
A2 - McKay, Cory
A2 - Zangerle, Eva
A2 - de Reuse, Timothy
PB - International Society for Music Information Retrieval
T2 - 21st International Society for Music Information Retrieval Conference, ISMIR 2020
Y2 - 11 October 2020 through 16 October 2020
ER -