Automated scoring in context: Rapid assessment for placed students

Andrew Klobucar; Norbert Elliot; Perry Deess; Oleksandr Rudniy; Kamal Joshi

doi:10.1016/j.asw.2012.10.001

Automated scoring in context: Rapid assessment for placed students

Andrew Klobucar, Norbert Elliot, Perry Deess, Oleksandr Rudniy, Kamal Joshi

Humanities and Social Sciences

Research output: Contribution to journal › Article › peer-review

20 Scopus citations

Abstract

This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the Criterion^® Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation, generalization, and consequence. Based on the results of our two-year study with students (N=1,482) at a public technological research university in the United States, we found that Criterion offered a defined writing construct congruent with established models, achieved acceptance among students and instructors, showed no statistically significant differences between ethnicity groups of sufficient sample size, correlated at acceptable levels with other writing measures, performed in a stable fashion, and enabled instructors to identify at-risk students to increase their course success.

Original language	English (US)
Pages (from-to)	62-84
Number of pages	23
Journal	Assessing Writing
Volume	18
Issue number	1
DOIs	https://doi.org/10.1016/j.asw.2012.10.001
State	Published - Jan 2013

All Science Journal Classification (ASJC) codes

Language and Linguistics
Education
Linguistics and Language

Keywords

Automated essay scoring (AES)
Validation methods
Writing assessment
Writing placement

Access to Document

10.1016/j.asw.2012.10.001

Cite this

@article{19b286a1ec3a4756bd7cea86bd89dce3,

title = "Automated scoring in context: Rapid assessment for placed students",

abstract = "This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the Criterion{\textregistered} Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation, generalization, and consequence. Based on the results of our two-year study with students (N=1,482) at a public technological research university in the United States, we found that Criterion offered a defined writing construct congruent with established models, achieved acceptance among students and instructors, showed no statistically significant differences between ethnicity groups of sufficient sample size, correlated at acceptable levels with other writing measures, performed in a stable fashion, and enabled instructors to identify at-risk students to increase their course success.",

keywords = "Automated essay scoring (AES), Validation methods, Writing assessment, Writing placement",

author = "Andrew Klobucar and Norbert Elliot and Perry Deess and Oleksandr Rudniy and Kamal Joshi",

year = "2013",

month = jan,

doi = "10.1016/j.asw.2012.10.001",

language = "English (US)",

volume = "18",

pages = "62--84",

journal = "Assessing Writing",

issn = "1075-2935",

publisher = "Elsevier Limited",

number = "1",

}

TY - JOUR

T1 - Automated scoring in context

T2 - Rapid assessment for placed students

AU - Klobucar, Andrew

AU - Elliot, Norbert

AU - Deess, Perry

AU - Rudniy, Oleksandr

AU - Joshi, Kamal

PY - 2013/1

Y1 - 2013/1

N2 - This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the Criterion® Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation, generalization, and consequence. Based on the results of our two-year study with students (N=1,482) at a public technological research university in the United States, we found that Criterion offered a defined writing construct congruent with established models, achieved acceptance among students and instructors, showed no statistically significant differences between ethnicity groups of sufficient sample size, correlated at acceptable levels with other writing measures, performed in a stable fashion, and enabled instructors to identify at-risk students to increase their course success.

AB - This study investigated the use of automated essay scoring (AES) to identify at-risk students enrolled in a first-year university writing course. An application of AES, the Criterion® Online Writing Evaluation Service was evaluated through a methodology focusing on construct modelling, response processes, disaggregation, extrapolation, generalization, and consequence. Based on the results of our two-year study with students (N=1,482) at a public technological research university in the United States, we found that Criterion offered a defined writing construct congruent with established models, achieved acceptance among students and instructors, showed no statistically significant differences between ethnicity groups of sufficient sample size, correlated at acceptable levels with other writing measures, performed in a stable fashion, and enabled instructors to identify at-risk students to increase their course success.

KW - Automated essay scoring (AES)

KW - Validation methods

KW - Writing assessment

KW - Writing placement

UR - http://www.scopus.com/inward/record.url?scp=84872395238&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84872395238&partnerID=8YFLogxK

U2 - 10.1016/j.asw.2012.10.001

DO - 10.1016/j.asw.2012.10.001

M3 - Article

AN - SCOPUS:84872395238

SN - 1075-2935

VL - 18

SP - 62

EP - 84

JO - Assessing Writing

JF - Assessing Writing

IS - 1

ER -

Automated scoring in context: Rapid assessment for placed students

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this