Assessing reflection is not always easy, and the problems that it can throw into the path of the assessor are the subject of a PhD for someone (it may already have been written, poor them): these are rendered even more complex by variables such as subject, course, institution, context, focus, purpose, task etc etc. I have been mulling this one over for a very long time and I can’t quite make peace with the idea of enforced levels of reflection, a vexing subject which arises as predictably as Christmas or a tax bill. I can see how stages in critical or analytic thinking as part of the reflective process can be perceived to map onto models such as Biggs’ SOLO Taxonomy or Jenny Moon’s stages of reflective writing, but the need to establish level as a principal benchmark of quality and for students to produce evidence of this to order, potentially in line with a pre-existing formula, still makes me uneasy. Just recently the question was posed during a revalidation meeting (it often is, the speaker prefacing it with frowning, pencil waving and the words “I am a bit worried about level”) and justification required as to how reflection will differ at the ascending levels of study – baby steps for first year undergrads, something a bit more meaty for year twos, and then suitably deep and nuanced elucidations from final year students?

A preoccupation with level has infiltrated our systems in tandem with a belief that evidence of reflective capacity has to be corralled into linear, progressive and sequentially developed tiers. Thus particularly hot topics of conversation include the ‘generic reflective outcome’ – which invites evidence of personal and professional development within the context of a particular unit across more than one academic stage. This is feared by some as sloppy pedagogy, even when accompanied by criteria, format and guides on content, if there is no specific reference to level.

My angst is not entirely pure, as formally assessed reflection can obviously benefit from some kind of explanation or contextualisation (such as kinds of topics to be touched on, particular angles or lines of enquiry). I can also see from the thousands of student reflections I have read over the years that some of these may very exhibit signs of falling into any one of  Moon’s categories 1- 5. I myself have also used my own different examples of reflective writing – good, bad and ugly – as indicators of quality and insight. However I believe that the expression of personally meaningful and authentic reflection has blurrier edges to it than a rigid framework of categories allows.  However I think it is the idea of a fixed framework that someone has to live up to and evidence, rather than meet more organically that I think sits counter to our struggles to convince students that becomign critially reflective practitioners is a worthwhile endeavour.

In considering this notion of reflection ‘for level’ I find Jan McArthur‘s concepts of ‘virtuous mess and wicked clarity’ wonderfully relevant, even though she applied these to higher education research and I am hijacking them for interpretations of reflective capacity. It is so tempting to assume that setting out expectations of depth, complexity, sophistication, polysyllaby or whatever, is naturally bound to make a reflective record more appropriate and suitably academic, but does not take into consideration lucidity, honesty, authenticity, insight etc, all of which might be very simply expressed at any level.  I wonder, too, if the focus on level is to do with our expectation that summative, assessed reflection will be in a written format, where evidence of level can be more easily identified?

The ‘wickedness’ of the linear approach also assumes a homogeneity of understanding of and engagement in reflective practice at different levels of study that simply does not exist. Far better to find out from your students where they are at in reflective terms and work from there, as students in high school or further education may have sharper understandings of how and why reflection might usefully aid their learning and personal development than a doctoral student who is highly resistant to any notion of value. There are so many factors that influence our understandings of reflection beyond the linear categorisation of level.

I mentioned Biggs earlier, however I’m not even going to get into the whole “learning outcomes: illuminating protocol or confusing quagmire” debate.  I don’t have a problem with clear and well-structured ones, although producing these seems to be something of a struggle on occasion. I do think, though, that they are part of the reason that we get exercised about what we should be measuring, and how, when it comes to reflection. (This might be exacerbated in situations such as mine where we have been combining level-oriented learning outcomes with a generic one for reflection; this has been working fine, however tends to be called into question when encountered by visitors new to the combination). I think we mislead ourselves when we try to impose a level-oriented measurement to grade someone’s personal evaluation of their learning experiences using the same criteria as we use for levels of learning a subject. This may work ok if you are studying psychology or neuroscience or anything involving how the brain works, but to assess how someone has analysed their learning on a course I think it is unreliable.

The reason I think this is because we measure by level in terms of grasp of content and ability to do things in increasingly complex and sophisticated ways. However, when we ask students to reflect on their learning we ask them to review in terms of how well things have gone, what they might do differently, and a whole plethora of other things ( see our list of 14 occasions which will engender reflective thinking in Chapter 1). The language we use to do this and our evaluation of our progress may be insightful and genuine and purposeful, but it can be all of these things without it being at the same level as the way we are engaging with our subject. Reflection is about how someone looks critically at their experiences to try and make better sense of them and this is where trying to impose protocols which have been designed to judge someone’s competence in a discipline or grasp of a subject can be misleading or unnatural. When I wrote my PhD, my theorising of identity and construction in relation to someone’s life and art was in keeping with the criteria of doctoral study, however the reflective journal I kept while I was researching and writing it was another creature entirely. This did not lessen the validity of either. There is also a subjective element to reflection that we don’t necessarily have in our study of content, although perhaps this is more of a blurred line in the arts than in other disciplines. There is also,more often than not, scope for informality in the way this reflection is recorded, although this too will vary greatly: I have seen personal evaluations from other universities which are required to be written in the style of a hybrid academic report with extensive theoretical references, however this is particular to one set of preferences and contexts. As an approach it does not appeal to me, as I find it an unnatural academicising of a personal appraisal. If we go back to SOLO ( and I am not being anti Biggs but he provides a useful model against which to make my case) we can see level indicated by a choice of verb – name, describe, analyze, hypothesize – just to take a sample form the four higher stages. Like all good reflective facilitators, I urge my students not to got bogged down in description and listing, but to delve, explore and question. However, even at the highest levels of study the simplest activities or kinds of engagement may still be relevant at any point in reflection because it is an iterative, messy, non linear mode of thinking and contemplating. Just take confidence, for example. This might be sky high when a child leaves primary school and rock bottom by the time they go to university, on all sorts of counts, which have nothing to do with academic or intellectual or subject oriented capability.  Specifying level by ‘verbs of engagement’ is one example of the issues I have with how we measure a student’s reflection and word count is equally problematic. Yes, we can request higher word counts at each stage of study, however we have all marked hefty pieces of work which are verbose and mediocre, and short simply expressed ones which have a profundity and insight to them that the former missed through trying to sound suitably academic.

I wonder if part of the reason that the issue of level in reflection recurs so often is that parts of our educational community have not reassured themselves as to the value or purpose of it, which makes them doubly concerned to establish its credibility in academic terms that all can understand. So we insist on higher word counts, or references to theory, or specific topics so our concerns can be assuaged as to the space it occupies in our curriculum.  While we seek to differentiate for our students between styles of reflective and academic writing I wonder if we then confuse ourselves by using the same criteria of level to judge both.  I think we also fixate on level when we don’t know how to solve some of the problems that are sometimes visible in student reflection, such as over descriptive text, or confessional outpourings, diarising or a lack of engagement with it for any number of reasons. So we insist on level, perhaps as a means of forcing students to realise that it is a legitimate activity. We also use notions of level to benchmark achievement – because that is what learning outcomes do. So by the time of submission we expect to see a reflective evaluation that hits specified markers or covers certain kinds of content. But we should not confuse quality and level. Going back to virtuous mess and wicked clarity, I believe that it is not always possible or authentic to tidy up reflective outcomes honestly to fit these kinds of markers. Discussing this issue with two colleagues yesterday we agreed that a factor as simple as age might come into play: not that we are too doddery to make sense of our experiences the older we get, but that we get less dogmatic about things.  One of my favourite postcards of all time was pinned up on a fridge and read: “Teenagers! Leave home now! While you still know everything” – and I think it sums up the surety that we may demonstrate in knowing at one time in our lives, compared to others (as well as the state of that household).

And I have ploughed on for days writing this and have not even touched on the most important aspect for our sphere of interest  – where do creativity and imagination come into all this?  Part of the reason that I remain unconvinced by an over-emphasis on level statements (even though these may useful for formative support in deepening reflection, when used with caution) is I find them antithetical to creative modes of reflection.  We want students to engage willingly in reflection but to do this we need to give free rein to their preferred ways of doing this, and allow them to prioritise the most significant aspects of learning for themselves.  What we can most helpfully do is appreciate that  insight, development, self-appraisal, rate of progress and so on are not always consistent, often traverse iterative cycles and are not biddable at will.  I’ll be exploring ways of assessing reflection creatively, and with a view to developing the quality of an individual’s reflection in tandem with them, in future posts.