Setting boundaries.

In 2017, the GCSEs grade standards in English language, English literature and mathematics will be calculated using a combination of norm- and criterion-referenced approach. This is for the first year of the GCSE only. In subsequent years, a predominantly criterion-referenced approach will be taken. All other GCSEs, being awarded for the first time in 2018, will follow the same pattern.

Before I get into the details, I would like to emphasise that I am blogging this as an English teacher, for English teachers. Many people will already be aware of this. It’s nothing new. Many people will know that this (partly) already happens. However, many do not.

You may already understand this and the implications for you, your team and your kids. I have been a head of English for 9 years now, across 2 different schools. Over that period, I have always worked with at least one team member who ‘can’t do data’ – somebody who get flustered or worried by numbers and terms that sound vaguely mathematical. After having it explained to me recently, I then had the task of explaining this to others. It occurred to me that others out in the wild blue yonder may also benefit from a breakdown. This is important and all teachers need to take it on board, so if you’re not sure then read on. I’ll keep it short.

All of this is written for the context of explaining the GCSE grade standards for 2017. I appreciate that I have simplified and generalised on some points. I welcome feedback and suggestions of how to make this more accurate and clearer (for my intended audience, that is).

What are criterion- and norm- referenced approaches?

I’ll start with the criterion-referenced approach. This is really simple; it’s the system that we already work with, more or less. That is,

                A system of assessment where grades are awarded against pre-determined criteria.

So, up until this year, every student who achieved a certain mark by reaching a certain point on the mark scheme would have achieved a specific grade. Simple. The examiners used the mark-scheme to allocate marks. I’m bending the truth slightly here, because an element of our current system is norm-referenced, but according to OfQUAL and the documentation from the boards, examiner judgement trumps all.

The norm-referenced approach is slightly different. This approach is:

A system where the student is graded against the rest of the population doing the test.

Wikipedia puts it like this: “A norm-referenced test (NRT) is a type of test, assessment, or evaluation which yields an estimate of the position of the tested individual in a predefined population, with respect to the trait being measured. The estimate is derived from the analysis of test scores and possibly other relevant data from a sample drawn from the population” (https://en.wikipedia.org/wiki/Norm-referenced_test accessed on 6/12/16 at 21:45)

This means that there will be an allocation of students achieving certain grades. Criteria will be used to establish what is essentially a national rank order. It’s impossible to predict grade boundaries unless you know exactly how all the other students across the country are performing. 

It means, that for the first year of the new GCSEs, a pre-determined number of students will achieve certain grades (grade 1, grade 4 and grade 7).

A combination of the two approaches

Both of these are valid approaches, normally used for specific purposes to achieve certain ends.

This year (2016-2017), OfQUAL have chosen to use both approaches when deciding the grade boundaries. They are going to use a norm-referenced approach to set grades 1, 4 and 7. These will be based on the same proportion of students who achieved the G, C and A respectively. The 2, 3, 5 and 6 will be calculated mathematically, with an even spacing between the neighbouring grades. This is also an important piece of information:

“Grade 5 will be positioned in the top third of the marks for a current grade C and the bottom third of the marks for a current grade B.” (OfQUAL 2014)

For the top grades, around 20% of those awarded a grade 7 or above will be awarded a grade 9. It is important to note that the 20% is across all subjects – not just the top 20% of 7+ within one qualification. The grade 8 will be placed with equal space between the 7 and 9.

Here’s how they’ll do it and here is where I got much of this information.

grade-boundariesWhat does this mean for teachers?

Essentially, it means everything and it means nothing at the same time. Schrödinger’s grade boundaries.

It means everything because it’s the manner in which our students’ grades are being calculated and you have a right, a responsibility and (hopefully) a desire to know this. It means that subject-specialist examiners don’t have as much influence over the allocation of grades as you’re used to. It means that there’s very little point in trying to work out a mark-scheme. It explains why the PiXL mark-scheme looks so lenient (it’s based on extensive data from a sample of Y10 students; evidently they didn’t perform as well and therefore the proportions of each grade fall lower down the ladder).

It means nothing because it’s happening, it’s out of our hands and many people believe that it’s the right thing – the fair thing – to do. This is the end product of a long consultation on the best way to make the transition to the new GCSEs. It’s also a political strategy, which went hand-in-hand with getting the new systems of assessment approved in parliament.

Your comments are always welcome. If you are going to tell me that I have got something wrong, then I definitely want to hear from you and I’ll edit accordingly, although I may be coming to you for editorial advice on the updated post.

Many thanks.

 

Advertisements