Assessing distance learning student's performance: A natural language processing approach to analyzing class discussion messages

In classes supported by electronic messaging systems, students are required or encouraged to discuss the class topics and share their knowledge by posting text messages and replying to others. When the amount of messages is large, it is difficult for the instructor to read through all messages and evaluate student's performance. We apply natural language processing techniques to analyze the course messages to assess student's performance. Students are evaluated from three aspects: knowledge learned from the class, effort devoted to the class, and the activeness of their participation; three measures - keyword density (KD), message length (ML), and message count (MC), are derived from the text messages for each evaluation aspect respectively. The three measures are then combined to compute an overall performance indicator (PI) score for each student. The experiment shows that there is a high correlation between the PI scores and the actual grades; the rank order of students by the PI scores and that by the actual grades are highly correlated as well.