Automatic Evaluation of Essays
The need for computer-assisted assessment of learning outcomes is connected to two inter-related factors in today's schooling and education markets. First, teachers need to automate the assessment and evaluation process especially in mass courses. Secondly, a student, particularly when following an online course, wants to assess his or her own learning process already before an examination. Furthermore, various evaluation and assessment schemes are widely used also outside traditional education.
Our Java-based essay grading system is based on Latent Semantic Analysis (LSA). LSA is a corpus-based statistical method, related to neural network models, which provides a means of comparing the semantic similarity between a source and target text. LSA can also be considered as a computational model of human knowledge representation. Research has showed that the method is able to simulate learning and several other psycholinguistic phenomena.
The system (Fig.1) preprocesses the essays by a morphological parser. It allows the user to adjust a set of parameters, for fine-tuning the accuracy. The grade is computed by using both human-graded essays and text materials from a textbook.
Figure 1. Essay Grading System
We have experimented with system consisted of essays and textbook written in the Finnish language. For the experiments we collected essays from an undergraduate course in education. We typed them out and ran experiments with our Java-based system. The test set consisted of 143 essay answers to a essay prompt. The essays were graded by a professor on a scale from 0 to 6. The grading system was trained by using a relevant chapter from the course textbook.
Tuomo Kakkonen, firstname.lastname@example.org
LINKS:Esseetehtävien tietokoneavusteinen arviointi [PDF]
(Tuomo's Master's Thesis, in Finnish)
Computer-assisted Essay Evaluation [Powerpoint-presentation]