What are the main problems school leaders can face in dealing with the ‘data deluge’?
By Kristian Shanks @HistoryKss | Seneca CPD
Last December, Seneca hosted a webinar about how leaders can better use assessment & progress data in school.
You can watch the recording on our YouTube channel here.
Below, it’s the article written by Kristian Shanks, our keynote speaker, summarising his ideas.
Don’t forget to check our other CPD opportunities on this link.
There is not a school leader in the country who does not find themselves confronted with a ‘data deluge’. Whether you are senior or middle leader, academic or pastoral, you will find that dealing with data forms a significant part of your role. However, given the limited time that we all have, it is important to try to filter the good data from the bad.
As a serving middle leader, and with previous senior leadership experience, I have significant experience in navigating the aforementioned data deluge. In turn, I have found some common problems in the roles that I have held.
Problem 1: The flaws of departmental-level data
By departmental-level data, I am referring to both data produced by a department, and data produced about a department. In the former case, this often takes the form of internal assessment scores or grades, as well as predicted grades about future performance on, for example, formal examinations. Potential problems here include the fact that departmental colleagues may have an over-generous view of student performance, perhaps informed by assessments that do not accurately reflect the demand of the curriculum being studied.
An issue I’ve personally encountered is where significant assessments were not properly sampled from the entire domain of knowledge being taught. Instead, students were told in advance the topics that were to come up on their History mock exam. Students performed well on said mock exams. But of course, they were only being tested on a tiny sample of the vast domain of knowledge that students are required to know for a GCSE subject. When the real exams rolled around, it was a different story and student underperformance was significant.
In terms of problems produced about departments, the issue here is that often it can be very noisy. As a Head of Department, you might find yourself being tasked to analyse the performance of tiny numbers of students within certain high profile pupil ‘groups’ such as disadvantaged or SEND, but there’s often not a lot you can meaningfully learn from this. Ultimately, the larger the data set, the more likely you are to be able to say something reliable about it. Much of the data that you encounter probably have confidence levels attached to it, but these might not always be supplied by the person giving you the data!
Problem 2: Our data focuses on product not process