Over the last few months many schools have been obsessing about progress 8 scores and I’ve watched a few schools get worked-up if the value of progress 8 was one decimal point above or below a specific value.
That’s why it was reassuring to see advice from David Blow in Schoolsweek, Leading Learner and from Data Educator that any calculation of P8 was inherently flawed: all calculations are currently based on the 2015 attainment 8 estimates. The clue is within that phrase. ‘2015’ refers to the exam result year the A8 averages are based on – so the averages we use are from last year’s cohort, not this year’s results. That’s why they’re estimates and will always remain so until the 2016 figures are released. Only then will schools have a meaningful way of calculating the figure.
So why are so many educationalists and MIS suppliers (this blog included) offering ways to calculate school and individual pupil P8 scores?
Because it’s still useful to see the relative P8 performance of different groups of pupils within school. So that we can look for within school gaps in progress between girls and boys, between pupil premium and non pupil premium, between ethnic groups or SEN pupils.
A well constructed SIMS marksheet really is a great place to analyse this data: summary rows provide you with the average for the group under analysis, group filters allow you to identify the pupils in the relevant vulnerable groups and each pupil name provides a data rich drill-down so that you can quickly go from the whole-school-big-picture, right down to the needs of individual pupils.
To many schools the biggest advantage of using SIMS is simply this: that SIMS is always the most up to date source of assessment and pupil information on school. Of course you can export to Excel and third party programs, but pupil data changes so quickly that no matter how good the spreadsheet, the data is always going to be somewhat ‘stale’.
Using A8 and P8 in this way is very valid for schools and makes a lot of statistical sense. What makes less sense is expecting the average P8 value for the whole cohort to be an accurate indication of the schools final P8 value as calculated by the DfE when the performance tables for 2016 are released.
Why? Because there are too many unknowns. This year’s results have been very variable and we know that schools will be trying to game the system slightly by encouraging pupils to qualifications that will improve their score. There are already report that schools have considered entering pupils for two English qualifications even if a poor grade is expected, so that at least the nest result will count double.
Can SIMS calculate P8?
I hear this question a lot. P8 seems so complex that some people assume SIMS can’t cope with the maths.
Let’s deal with that first: SIMS Assessment Manager is easily powerful enough to cope – it has a full set of mathematical and logic functions. Although it doesn’t have all the fancy features of Excel, I have yet to encounter an assessment calculation that can’t be replicated in Assessment Manager. I’ve already outlined a method for calculating A8/P8 in a SIMS marksheet and I’ve published an online training course that also shows SIMS users how to calculate the new scores. Plus we now have Capita’s KS4 Resources which will also do the calculation for you.
[et_bloom_inline optin_id=optin_1]
Another objection I sometimes hear is that SIMS can’t do the discounting, or that a particular school offers obscure non standard GCSEs that marksheets can’t cope with. Again, it’s the same answer: if you use the full set of formulas and techniques available in Sims there no reason why you can’t include the most obscure BTEC or City and Guilds courses. With discounting sometimes it’s the simplest solutions that work best: just editing or deleting the discounted grade often works perfectly well.
I’ve blogged before about the dangers of outsourcing the analysis of data to third-party solutions and school data managers still complain to me that their school is effectively becoming ‘deskilled’ and unable to properly analyse its own data.
Other sources to help you calculate P8 values include:
- Your local SIMS support team
- ASCL, who run regular P8 roadshows
- Capita’s own documentation for the KS4 resources.
- Follow Duncan Baldwin on twitter
- Follow Peter Atherton on twitter
With all the caveats applied about P8 only being an estimate for any current cohort, because the estimates table comes from the previous cohort’s exam results, it is beneficial to have some idea of the P8 score for a cohort along with other measures of progress and attainment.
With the changes to points allocations for grades moving from 1-8 for all subjects to 1-9 for English & Maths and 1-8.5 for unreformed GCSEs, this renders the 2015 estimates table unusable as the values are calculated on 1-8 points for a grade.
Have you any thoughts on dealing with this? We want to move our calculations to the 1-8.5 system to give a reflection of what our A8 and P8 scores will be for this Y11 cohort (rather than perhaps roll back the points to continue using the previous points methodology.)
Hi Andy, better statistical brains than me recommend that we only use P8/A8 calculations to look for relative differences between subject and pupil groups in a school. Until the next set of A8 estimates are released (later this year?) the 2015 averages are all we have. Maybe some readers of this blog would like to comment?