Web-based portfolio assessment: Validation of an open source platform

Regina Collins, Norbert Elliot, Andrew Klobucar, Fadi P. Deek

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Assessment of educational outcomes through purchased tests is commonplace in the evaluation of individual student ability and of educational programs. Focusing on the assessment of writing performance in a longitudinal study of first-time, full-time students (n = 598), this research describes the design, use, and assessment of an open-source scoring platform. Augmenting usability testing, the research design relies on a framework of inter-reader agreement, inter-reader reliability, and coefficients of determination. The open-source, web-based portfolio assessment system yielded rates of agreement, reliability, and determination superior to the traditional paper-based portfolio assessment method. In addition, the system appears to be ideally suited to assess EPortfolios created to showcase student ability in digital environments: agreement range = 70% to 85%; reliability range = κ = .67 (p < .01) to κ = .85 (p < .01); coefficient of determination = R2 = .95, F(5, 34) = 118.59 (p < .01). This novel and innovative application of an open source platform for outcomes assessment yields the foundation for a sound validity argument, the control of human error, and complete system transparency and flexibility. Future research directions point to the need for the design and assessment of an open-source system designed to capture complex constructs as they emerge in digital environments.

Original languageEnglish (US)
Pages (from-to)5-32
Number of pages28
JournalJournal of Interactive Learning Research
Issue number1
StatePublished - 2013

All Science Journal Classification (ASJC) codes

  • Education
  • Human-Computer Interaction
  • Computer Science Applications


Dive into the research topics of 'Web-based portfolio assessment: Validation of an open source platform'. Together they form a unique fingerprint.

Cite this