When less is more: Students’ experiences of assessment feedback
Note: 'Tutorial' denotes small-group discussion following lecture or relating to a specific task; 'Workshop' denotes activity-based teaching
Table 1: Summary of key attributes of the seven completed case studies
Theoretical orientation and conceptual framework
The design of this research project was influenced by our socio-constructivist theoretical orientation. Our basic premise is that student learning processes evolve and are bounded by socially-constructed norms of behaviour and value systems. 'Learning' is situated not only physically in the classroom, institution, and geographic region, but also structurally in the relations between students and tutors, and in the academic norms of the discipline. Whilst situated socio-constructivism orientation does not necessarily preclude or deny the role of individual agency, that agency is seen as significantly constrained in ways unlikely to be visible to the individual. Whilst it is not appropriate in this paper to provide a lengthier explanation, our theoretical orientation is elaborated in other papers (Handley et al., 2007; Rust et al., 2005).
To provide a theoretical focus for our analysis, we developed a conceptual framework highlighting specific areas of interest: in particular, the structural influences embedded in the 'context', the interaction between students and assessors, and the temporal dimension through which student and staff experiences (and styles of engagement) are shaped by succeeding assessment/feedback episodes.
The conceptual framework is developed in figures 1 to 3. Figure 1 presents the three artefacts central to most feedback methods: the assignment brief; the final assignment completed by the student(s); and the feedback.
Figure 2 adds the interaction of staff and student(s) and portrays a traditional assessment/feedback method where the assessor writes feedback which is then read - independently - by the student.
Figure 2: Interaction between student(s) and staff in assessment/feedback methods
Of course, there are many possible variations and options, such as the provision of audio feedback, or staged feedback, or dialogue between staff and student. Furthermore, the member of staff writing the assignment brief may not be the person who marks the final assignment. For simplicity, however, only the essential elements are depicted here. Figure 3 develops the framework by including structural and processual elements to assessment/feedback methods. In doing so, Figure 3 reflects a more realistic picture because it includes a temporal dimension.
Figure 3: Contextual and temporal aspects of assessment/feedback methods
This means that before the assessor even begins to write an assignment brief, he or she is influenced by contextual factors such as the traditions of the academic discipline (e.g. science vs. humanities); by institutional policies; by socio-cultural norms, academic discourses and so on. Students are also influenced by contextual factors, but not necessarily by the same ones, nor in the same way. The conceptual framework also shows that any assessment/feedback episode has a response outcome for both student (e.g. satisfaction, confusion, and increase in self-efficacy, or disillusionment) and staff (e.g. new assumptions about student progress; or disillusionment about student failure to collect marked assignments). Responses may be immediate or longer-term; for example, a student's immediate reaction may be disappointment, followed later by a willingness to re-read the feedback and reflect on it. These immediate and long-term responses influence student and staff styles of engagement with the assessment/feedback process, and with their education experience as a whole.
In this paper we focus on two cases to illustrate a key theme from our research: that formative feedback is often more effective in supporting student learning if given on drafts rather than on final coursework. We illustrate the underlying problem in Figure 4. In this archetypal situation, the assignment feedback is given to students late in the module's duration, and the question for students and staff is whether that feedback really has any relevance to future modules or skills/knowledge development.
Figure 4: The problem of the [lack of] feedback utility across modules
As Nicol and Macfarlane-Dick comment:
In HE, most students have little opportunity to use directly the feedback they receive to close the gap, especially in the case of planned assignments. Invariably they move on to the next assessment task soon after feedback is received. (2004, p.10).
The scenario is particularly problematic if marked coursework is available only after the module ends: for some modules, the number of students bothered to collect this work and feedback is desperately low. Whatever the quality of feedback, students cannot learn from it in these cases.
The cases looked specifically at providing feedback on draft assignments. The setting and findings for each case are discussed in turn.
Module and assessment/feedback approach: Case 1 involved the module Business in Context which explores the complex contemporary environment in which businesses operate. Students are expected to think creatively and to engage with a wide range of sources and activities as they develop an understanding of this subject. The module has a large cohort of second and mainly third year students (111 students at the time of the case study) of whom a substantial proportion are international students. A teaching team approach is used, with one module leader and four seminar leaders.
The assessment/feedback method was designed to allow students to be 'active learners' who could apply the insights gained from feedback (please see Figure 5). In this way, students could bridge the 'learning gap' identified by Sadler (1998). The key design feature was to enable students to re-write and re-submit part of their individual coursework assignment after receiving feedback on their own work. The feedback was received in two ways: verbally, by seminar leaders giving generic feedback to their groups; and verbally, by the module leader giving specific feedback to each student based on the seminar leaders' feedback written on assignment scripts. The individual feedback was given by the module leader over the course of one day in 5-minute appointments with 109 students (2 did not attend). Students were asked: what mark are you expecting? what went well? what could you improve? and what should be re-written and re-submitted? This dialogue which ensued enabled students to identify and understand how they could improve on the draft assignment. Students were given one week to re-write and re-submit their work, and additional feedback was given. On re-submitting, students were given extra marks of up to 5%.
Figure 5: Assessment/feedback approach used in the Business in Context module
Research methods: Student perspectives on this feedback method were collected in two ways: module evaluation forms analysed by the module leader (n78); and semi-structured interviews (n5) conducted by a research associate in the Teaching and Learning Department.
In the module evaluation forms (‘MEF’), students were asked three open questions and given the opportunity to explain their comments. The findings from the MEF are presented in tabular format in Table 2. Comments which illustrate recurring ideas are set out in column 1; numbers and percentages of students making similar comments are in columns 2 and 3. The interpretation of student comments and their allocation to thematic categories was done by the module leader in discussion with another member of staff. Both researchers are within the Teaching and Learning Department and have a shared theoretical and practical interest in pedagogy and student feedback.
Student interview candidates were identified at random from those to came for the verbal feedback; all candidates were willing to be interviewed. The student interviews were taped, transcribed and entered into a qualitative software package (QSR NVivo) by the Research Associate before being analysed. Analysis involved reading and re-reading the transcripts using an open-coding process which requires that each significant section of data is 'labelled' using a code which encapsulates the broad topic (e.g. student experiences of feedback) or an analytical interpretation (e.g. student need for reassurance). Coding-on is the process by which sections of the interview are analysed in increasingly more sensitive ways as the nuances of student experiences are clarified by the researcher.
Table 2: Analysis of student comments on Module Evaluation Forms for the Business in Context module
Data from the MEF questionnaire was supplemented with the lengthier comments from the five interviewees. Students appeared to value the feedback approach for a variety of reasons, including the chance to have face-to-face contact with the module leader and the chance to discuss their work. This attitude is illustrated in comments from Student C, who said that she would have come to the feedback appointment whether or not she was likely to gain an additional 5% marks. Student E favoured the ‘feedback-on-draft’ approach, and was dismissive about traditional feedback given on final assignments:
‘…it doesn’t really stick, because you have got your mark and that’s it - because you have to move on and do the next piece of work.’
Other students said they valued the opportunity to get support, reassurance and confidence from talking about their work. However, some students commented on their occasional reluctance to ask questions about feedback. For example, Student A said she would not normally talk to staff about feedback, because they were usually ‘unapproachable’. She added that in spite of this general impression, she liked having the discussion with the module leader because it gave her reassurance that she was doing the ‘right thing’; she added that she could now see how the feedback could be useful elsewhere. This student’s general diffidence about seeking feedback is significant, and also relevant to our interpretation of the next case.
Negative interview comments about the feedback concerned problems of handwritten feedback, and timing. Student E talked of the ‘constant gripe’ about handwritten scribbles. In the MEF, some students (n16) said they’d have liked longer talking to the module leader, although this theme contrasted with the comments of Student D who thought the approach was too time-consuming: ‘I didn’t like having to wait 40 minutes for my appointment…[it] is kind of inevitable but it is quite a long time just to sit and wait for a 5 minute appointment’. Nevertheless, Student D recommended that the feedback approach should be used in all modules so she could ‘benefit more from it’.
Module and assessment/feedback approach: Case 2 involved a 1st year undergraduate module called Personal Professional and Academic Development in Tourism (‘PPAD’) which is similar to other Personal Development Planning initiatives elsewhere in Higher Education Institutions at the moment. 81 students were initially registered on the module: 22 from the UK; 50 from Eastern Europe; and 9 international students. 74 remained on the module after some fall-out in the first few weeks. The module is usually delivered using 2-hour workshops, but with occasional lectures. Students are introduced to the concept of transferable skills, and are encouraged to demonstrate how those skills can be valuable within the tourism sector.
The assessment/feedback approach is represented in Figure 6. In week 5, students submit a reflective essay in which they identify and discuss perceived strengths and weaknesses with reference to the skills required to complete a degree in tourism. The essay is worth 25% of overall marks. In week 8, students receive their marked assignment with feedback. In week 13, students submit a portfolio in which they demonstrate the application of a range of academic and study skills used during their first semester. Students also describe their strategies for developing those skills during the remainder of the degree programme. Both assignments relate to study skills: the first is an essay on the skills they bring to University from previous educational experiences; and the second is a portfolio about skills acquired during their first experiences at University.
Figure 6: Assessment/feedback approach used in the PPAD module
The key feature for this paper, however, is that students are offered the option of receiving feedback on a draft of their assignments, particularly on the draft portfolio.
Research methods: The principle data collection methods were semi-structured interviews and students’ own coursework. The latter were used to analyse the effectiveness of the feedback given for the reflective essay in improving the quality of the portfolios. However, for this paper, we are interested specifically on students’ responses to the offer of receiving feedback on the draft versions of the assignments, and we therefore rely on our analysis of the interview data.
Sixteen candidates were selected for interview using a randomising process. Of the sixteen selected for interview, five agreed to be interviewed. Interviews were conducted by a researcher at the University with a particular interest in PDPs. The module leader was interviewed by the research associate who was also involved in Case 1. All interviews were taped (except one due to mechanical problems) and then fully transcribed and analysed using a process of thematic analysis.
Findings: The findings described in this paper relate specifically to students’ responses to the offer of feedback on their draft assignments. Unexpectedly, and unlike the experience in Case 1, the number of students submitting drafts for feedback was disappointingly low (n= 3 out of 74). This was inspite of the fact that students had been openly encouraged to submit drafts at several points during the module. However, submission was not compulsory; the draft was not pre-marked, but it was annotated with feedback, and there was no option of an additional 5% as in Case 1.
During interviews, it was apparent that most students were aware of the option to receive feedback, but there was no single reason why that option was rarely taken up. Instead, there seemed to be a combination of reasons which compounded to lead students to feel that feedback was unnecessary or untimely.
For example, one student said she had already gained an adequate understanding of the task by reading the carefully-written assignment briefing, and had stopped the module leader in the corridor when she had specific questions (Student 5). During the interview with the module leader, it became apparent that many other students also used this strategy: they would ask, for example, ‘is it ok to include a, b, c?’; and, ‘do I write in the first or third person?’. These questions were raised not in the module leader’s office, but instead in informal settings such as the stairs and the toilets! The module leader recalled her dismay when a student asked if she could talk to her through the cubicle door. When she asked another student who had also approached her in the toilets and not in the office ‘why?’, the response was ‘we are more on the same level there’.
A number of interpretations and implications can be drawn of these experiences. Firstly, students may have concerns about their physical and psychological ‘space’ in which they are willing to seek and receive feedback: talking to a tutor in the corridor is less threatening than the prospect of an academic office. Secondly, the students have control in the sense that they initiate the interaction, although the setting is not conducive to a reflective dialogue. Thirdly, perhaps as a consequence of snatching spontaneous moments, the student questions are brief and appear at first glance to be relatively superficial.
Another specific issue was that the ‘submission of drafts’ option was largely sabotaged by the timing of such opportunity - just before the Christmas break when the majority of students were focused returning to their homes abroad (and sometimes before Christmas closure), and drafts not being in a ready state for submission. This hypothesis was supported by some interviewed students; for example, two (Student 4/ Student 5) admitted to leaving insufficient time to submit a draft. Indeed one of the three students who did submit a draft did so too late for it to be read let alone discussed with the student. ‘Leaving it to the last minute’ was a strong impression gained by the module leader.
Another reason for non-submission of drafts was that students seemed to misunderstand the nature of feedback. Students commented that the assignment brief was very clear (e.g. Student 5 and other students whom the module leader spoke to informally). These students were pleased that the brief was very straightforward and detailed, and therefore saw no need for further guidance. These 1st year students did not fully appreciate that feedback is about giving guidance on the way in which students respond to the assignment - not about their understanding of the assignment brief. This finding has contributed to the University developing a component in the Personal Development Planning modules which guides students into the range of uses of feedback. It is possible that if – as in Case 1 – the draft had been marked in addition to being annotated with feedback beforehand, students would have been motivated to meet the module leader to seek clarification of the feedback (and in doing so, discuss their draft). The mere offer of a discussion about the draft was not, it seems sufficiently attractive.
Some students made the related point that they did not know how to construct a dialogue (a point also raised by some members of staff). The implication was that students preferred the apparent informality and non-threatening nature of a spontaneous exchange where no ‘deep’ questions would be expected. One student commented:
A lot of the students here are [Eastern] European students and if they want to ask something, they don’t know how to ask it. They are confused, like, ‘how should I go and ask the teacher about this?’, because I have many friends here who are like – ‘I don’t know how to ask her about this’. (Student 3)
Underlying this comment is a deeper issue: that on starting their University experience, many students lack confidence or even fear asking questions, in case this gives the impression of being critical of the lecturer or of appearing unintelligent. Paradoxically, some students who did ask questions or seek feedback felt that they were receiving some sort of ‘personal favour’.
Implications for practice
The strategy of giving feedback on drafts - and allowing students to apply their learning to the same or later assignment - has great potential to enhance student engagement and development. However, the two modules - each offering students feedback on draft assignments - attracted significantly different student responses. In this final section, we offer some suggestions to explain this difference and we also propose implications for practice.
Psychological safety: Student comments from Case 2 indicate that students’ sense of their psychological safety is closely linked to their perceptions of the physical, social and mental learning space. Students in Case 2 were reluctant to take the initiative to meet the module leader in her own office to discuss feedback, and preferred to ask brief questions when a fortuitous occasion arose. Whilst this finding might suggest that offices should be re-located to the comfortable environment of cafes - that is not necessarily the answer. Conversations between students and staff which taking place in 'informal' spaces - defined in terms of physical locations and/or psychologically safe spaces - may not facilitate effective academic dialogue about students’ work and how to improve it.
Nevertheless, there is clearly an issue about encouraging students to feel that they are members of an academic community where the norm is to discuss one's work in a relatively open, informed and scholarly manner. It is interesting to note that one conclusion arising from Case 2 is that students need more guidance during their first semester at University to help them understand what feedback is, and how they can use it. This initiative can be seen as a way of enabling students to become what Lave and Wenger (1991) would call legitimate peripheral members of the academic community of a University. It seems that whilst the Case 1 students overwhelmingly approved of the opportunity for dialogue after they had experienced it as a quasi-compulsory element of the module, the Case 2 students were reluctant to avail themselves voluntarily of a similar opportunity.
Developing skills to ask questions: Linked to the above point, many University students - especially in their first year - struggle with the simple task of asking ‘appropriate’ questions. An apparently banal and simplistic question from students such as ‘How do I complete the task’ may not indicate that the questioner is a loafer hoping for an easy solution. Instead, the student may be struggling to perceive the underlying structure of the assignment and the process of completing it, or even how to tackle problems in general. In these situations, the tutor's response to the surface question may be insufficient; what may be far more useful is a dialogue to help the student understand the assignment question, the criteria and so on. Of course, there is also a possibility that staff lack sufficient skills to ask developmental questions of their students. The ability to ask pertinent questions is, after all, one of the hallmarks of an accomplished academic practitioner.
Benefits of providing feedback on drafts: The main premise of both case studies was that students would benefit from receiving feedback on drafts because they could then apply that feedback and learn from the process of doing so. The findings from Case 1 strongly supported that assumption, whilst the findings from Case 2 highlighted the difficulties of implementing an optional ‘feedback-on-drafts’ strategy. Nevertheless, we hope that our detailed discussion of the problems faced in Case 2 show that the constraints (e.g. timing) were not insurmountable, but simply reflect the real constraints on student life which need to be considered when designing feedback strategies. Our conclusion from these two cases is that ‘feedback-on-drafts’ - especially where the feedback recommends the re-writing of specific areas of weakness - is beneficial for many students, but that the structure of the assignment needs to build in appropriately-timed and psychological-safe spaces in which feedback is given and received. We also conclude that students are more receptive to the offer of additional feedback on drafts where the draft has been (at least partially) marked and annotated with comments. In this situation, feedback then clarifies the marking and comments and may facilitate students' self-assessment of their work. This approach has implication for workload planning, which we discuss next.
When less is more: finding time to give feedback-on-drafts: Academic staff face considerable workloads in the current Higher Education environment, fuelled by increasing student numbers and a lack of resources. These HE trends have conspired to give staff less time to mark student assignments let along give good quality feedback. Given these constraints, it is particular important to focus effort on the points where students have most to gain. This paper began with the suggestion that many students fail to engage with feedback received at the end of a module because they do not perceive its relevant for future modules. Feedback-on-drafts overcomes this problem by providing feedback at the point where the lessons can be applied. Time spent giving formative feedback on drafts is, we argue, usually more effective than feedback given on the final version. This suggests that staff time should be shifted towards the draft, with only a summative mark or grade being given on the final version. Furthermore, in some cases a combination of written notes + short meetings for dialogue may be less time-consuming that the writing up of detailed feedback (especially if the student cannot read handwritten comments). Another option is to use class time to discuss drafts and given feedback, using alternatives such as peer reviews or student-tutor sessions.
Key points raised in this paper are illustrated in our conceptual framework representing draft-plus-rework methods (see Figure 7). The key issue, it seems, is to make feedback more effective by motivating students to engage with it, and at the same time making the giving of feedback more efficient. Less may be more if the effort in giving feedback is shifted to the point at which students are still working on their assignments.
Figure 7: Staff and student experiences of the draft-plus-rework method
Askew, S. & Lodge, C. (2000) 'Gifts, ping-pong and loops - linking feedback and learning', in S. Askew (ed.) Feedback for learning, London: Routledge.
Chinn, C. A. and Brewer, W. F. (1993) 'The role of anomalous data in knowledge acquisition: a theoretical framework and implications for science instruction', Review of Educational Research, vol. 63, no. 1, pp. 1-49.
Fenwick, T. J. (2000), 'Expanding conceptions of experiential learning: a review of the five contemporary perspectives on cognition', Adult Education Quarterly, vol. 50, no. 4, pp. 243-272.
Gibbs, G. & Simpson, C. (2002) 'Does your assessment support your students’ learning?' Available at http://www.brookes.ac.uk/services/ocsd/1_ocsld/lunchtime_gibbs.html [Accessed 15 May 2006]
Handley, K., Clark, T., Fincham, R. and Sturdy, A. (2007) 'Researching situated learning: participation, identity and practices in client-consultant relationships', Management Learning, vol. 38, no. 2, 173-191.
Higgins, R., Hartley, P. and Skelton, A. (2002) 'The conscientious consumer: reconsidering the role of assessment feedback in student learning', Studies in Higher Education, vol. 27, no. 1, pp. 53-64
Hyland, P. (2000) 'Learning from feedback on assessment'. In P. Hyland and A. Booth (eds.) The Practice of University History Teaching. Manchester University Press
Laurillard, D. (1993) Rethinking university teaching: a framework for the effective use of educational technology (1st ed.). New York: Routledge.
Lillis, T. and Turner, J. (2001) ‘Student writing in higher education: contemporary confusion, traditional concerns’, Teaching in Higher Education, vol. 6, no. 1, pp. 57 – 68.
O’Donovan, B., Price, M. and Rust, C. (2004) 'Know what I mean? enhancing student understanding of assessment standards and criteria', Teaching in Higher Education, vol. 9, pp. 325-335
Rust, C., O’Donovan, B., Price, M. (2005) 'A social constructivist assessment process model: how the research literature shows us this could be best practice', Assessment and Evaluation in Higher Education, vol. 30, no. 3, pp. 231-240.
Sadler D.R. (1998) 'Formative Assessment: Revisiting the Territory', Assessment in Education: Principles, Policy and Practice, vol. 5, no. 1, pp. 77-85.
Shepherd, J. (2006) ‘Courses deliver, but feedback falls short’, Times Higher Education Supplement, 25th August 2006
1 * The corresponding author is Karen Handley at email@example.com. Authors (a), (b), (d) and (e) are at Oxford Brookes University. Author (c) is at the University of Bedfordshire.
2 'FDTL' is a reference to the Fund for the Development of Teaching and Learning, under the auspices for the Higher Education Academy http://www.heacademy.ac.uk/1046.htm
Handley et al. 'Students' experiences of assessment feedback' - HEA Conference 2007
База данных защищена авторским правом ©shkola.of.by 2016
звярнуцца да адміністрацыі