edit Reflect


Lichao Li, Greg Darke, Jessica Preston, Mark Assad, Judy Kay
School of Information Technologies, University of Sydney

Contact Person

Judy Kay

Photo of Reflect

Project Description

Reflect began as an aid for students to learn more from reading examples. The inspiration was to help students learn to think like a computer scientist when they studied solutions to homework programming tasks. Essentially, the first versions of Reflect (called Assess) asked students to answer a set of questions about each supplied example answer to a problem: for example, they might be asked if it worked correctly when the input file is empty. This stage aimed to help students think about the code as the teacher did, in terms of the aspects that should be considered.

Since the teacher had coded their own answer to the question, the next stage of interaction was to show the student how their answers to the questions compared with the teachers. At this stage, the student could also see general comments from the teacher, perhaps pointing out why certain features of the code are interesting, or elegant, ....

As the student used the system, it kept track of their answers and provided a display of how they were doing. Initially, this was a very simple indication of performance on each question. In more recent versions of Reflect, Licaho Li added a detailed learner model has been built and this is displayed using tools like SIV.

While the initial vision was purely to support reading of examples, Reflect now has support for student submission of their own answers. If these are programs, there is support for automated grading and feedback.

With the addition of a learner model, it was necessary to provide an authoring environment that enables the teacher to create the set of learning objectives which constitute the components of the learner model. This makes use of our Mecureo automatic ontology builder: the author simply provides a glossary definition of each component and this is analysed to define an ontology for the set of Reflect tasks in the site. This means that Reflect can do ontological inference to give feedback on the learner's progress at a range of granularity levels.

Key Publications

S. Bull and J. Kay. Metacognition and open learner models. In The 3rd Workshop on Meta-Cognition and Self-Regulated Learning in Educational Technologies, at ITS2008, pages 7-20, 2008. [View Details]

J. Kay, L. Li, and A. Fekete. Learner reflection in student self-assessment. In Proceedings of ACE 2007, 9th Australasian Computing Education Conference, pages 89-95, Sydney, 2007. Australian Computer Society. [View Details]

A. Carbone, M. de Raadt, J. Kay, R. Lister, A. Litchfield, R. Raban, P. Roe, D. Santamaria, J. Sheard, J. Shepherd, and A. S. and. The Carrick vision and computing education: Four case studies in multi-institutional collaboration. In Proceedings of ACE '07, Australian Computing Education Conference, volume 66 of CRPIT, pages 3-8, Sydney, Australia, 2007. Australian Computer Society. [View Details]

J. Kay and L. Li. Scrutable learner modelling and learner reflection in student self-assessment. In M. Ikeda, K. Ashley, and T. Chan, editors, Intelligent Tutoring Systems: Proceedings 8th International Conference, ITS 2006, pages 197-206. Springer, 2006. [View Details]

J. Kay and L. Li. The cost of authoring with a knowledge layer. In Online Proceedings of the AIED (Artificial Intelligence in Education) 2005 Workshop on Authoring of Adaptive and Adaptable Educational Hypermedia (A3EH), pages 72-79, 2005. [View Details]

J. Kay and L. Li. Learner reflection in student self-assessment. Technical Report 568, School of Information Technologies, The University of Sydney, March 2005. [View Details]

L. Li and J. Kay. Assess: Promoting learner reflection in student self-assessment. In Editor, editor, Online Proceedings of the AIED (Artificial Intelligence in Education) Workshop on Learner Modelling for Reflection, to Support Learner Control, Metacognition and Improved Communication between Teachers and Learners (LeMoRe05), Series Assess: Promoting Learner Reflection in Student Self-Assessment, 2005. [View Details]

L. Li and J. Kay. Student self-assessment and learning to think like a computer scientist. Technical Report 577, School of IT, The University of Sydney, 2005. [View Details]