The BBC has, for months, joined in with the united – and completely misleading – media chorus about events at Stafford hospital. Every bulletin on the BBC News channel and every article on its website has stated, as fact, that there were
hundreds of needless deaths at Stafford hospital
in one variant or another.
Yet, in May 2010 – just after the coalition government took office – Radio Four’s ‘More or Less‘ dedicated a large section of its programme to the issue of HSMRs, the mortality statistics that fuelled the furore around the hospital, and specifically to the question of whether the statistics really did justify the lurid headlines.
The programme is remarkable for its balance, which is in itself a grim judgment on the quality of the BBC’s portrayal of health issues. You can listen to the whole thing by clicking here and listening from the 3-minute mark to the 14m mark. But in case you don’t wish to do that, here are a few highlights.
Presenter Tim Harford first notes that the claim that HSMRs showed 400-1200 avoidable deaths at Stafford is not mentioned anywhere in the HCC’s report – it had been removed because the authors of the report knew that the statistical nuances and uncertainties would be ignored and great misunderstanding would result, but was leaked later.
He then goes on to question Richard Hamblin, Director of Intelligence at the Healthcare Commission (HCC):
TH: Does it show that 400-1200 people died as a result of poor care?
RH: I think the answer is no.
Hamblin is then asked how he felt when the figures leaked and the headlines began:
Personally I was gutted.. I was really annoyed.. because we were very very clear.. that this is an inappropriate figure..because we were absolutely clear that we had not demonstrated that poor care led to..excess deaths.
Doesn’t seem much room for doubt, there, does there?
We then hear from David Spiegelhalter, Winton Professor of Public understanding of risk at Cambridge:
Half of all hospitals will have excess deaths – half of all hospitals are below average.. Hospitals can be extreme for a lot of reasons.. There is always variability between hospitals that isn’t taken into account.. not least of which is that we assume that factors such as age, and ethnicity, deprivation have exactly the same effect in every hospital in the country.. You can’t claim that excess mortality is due to poor-quality care.
So, even if the statistics were reliable (which they absolutely were not), you still couldn’t use them to support a figure for avoidable deaths or deaths linked to poor care. And even if the statistics were correct (which they absolutely were not), ‘rebasing’ them to 100 to portray an average figure every year will automatically make half of all hospitals appear to substandard.
For balance, we hear at the end from Prof. Brian Jarman, the creator of the HSMR system. Professor Jarman has recently claimed that his statistics show that 20,000 people died unnecessarily in the NHS, just as it supposedly showed 400-1200 deaths at Mid Staffs. Yet in 2010, his story was a little different:
We don’t claim ‘this is the number of people that were killed’, as it were’.
Quite a different tune. Mr Harford then challenges Prof. Jarman that his organisation is just assuming numbers of avoidable deaths from statistics that don’t show any such thing.
Predictably – since he has done so at every opportunity even though the flaws in the system (especially the data going into it but also with the system itself, which counts some deaths twice, for example) – Prof. Jarman defends the usefulness of his ‘baby’.
But he makes once incredible admission:
about 24% is random variation.
Throughout the period in which it had high HSMRs, Stafford’s figures only once went more than 24% above average – and that was shown to be down to a change in methodology that Stafford’s coders didn’t know about.
24% potential random variation is huge. Yet these statistics have been used to state, as if it were incontestible fact, that Stafford hospital was causing ‘hundreds of needless deaths’.
The more I look at the Stafford situation and at HSMRs in general, the clearer the flaws and weaknesses of the system and its results become – and the more obvious, blatant and culpable the media distortions around them. And it’s absolutely clear that my sceptical view of them is far from a new one.
A balanced, nuanced BBC view in 2010, when the Tory-led coalition had only just taken office – and a shamelessly emphatic statement as fact in 2012 and 2013 of a myth that is incredibly damaging to the NHS and to public affection for it.
I wonder what that could have been caused by..
Reblogged this on hdgough's Blog.
The statistics have clearly been horribly misused, both by the government and media. Clearly any attempt to measure death rate is fiendishly complicated and a scary headline will always take priority over a reasoned discussion.
This is clearly part of the orchestrated public sector bashing going on. Demonise nurses, teachers, social workers, police officers et al and their conditions can be continuously eroded.
More than any government in my memory the coalition sets one section of society against another; rich against poor, old against young, public sector against private aided and abetted by the media. Very disappointing to see BBC doing this, maybe they are running scared.
Saddest thing is that there has been some poor performance in public sector but this is primarily caused by constant management changes, a culture of finance above all else and arrogant management. The resulting lay demoralised staff are then pilloried.
Finally more or less is probably the best programme on radio 4, debunking all sorts of dodgy stats!
Yes and certainly another look by that programme would be timely. Your analysis as to the real causes of poor performance are right – sadly these are an-going and re-enforced (ie it’s only going to get worse). I have said in this blog previously and recurrently that putting quality first is good for patients (or passengers, or pupils …) AND usually ends up cheaper, whereas putting finance first fails on all counts. This is the sadness of HMG’s approach. If you need a clear account of this go to:
http://www.theconsultantjournal.co.uk/journals/issue17/index.html
and go to page 28.
And the reason this (above) is so important is precisely because it is counter-intuitive i.e. if you want to save money in a public service (and it may well apply to the economy more generally) you actually need to focus primarily on quality (ie getting it right/doing it well) and NOT primarily on finance. Odd but true. There is probably quite a lot on the web about this but the reference given above is a good exposition and relevant to the NHS.
I suspect this is more down to the BBC’s scale rather than any overt desire to ignore what one part of it has found out. Quite often people in one part of the BBC are completely ignorant of what’s being done elsewhere. That said, I do believe (as do many BBC journalists) there is a culture of cautiousness across the corporation which blunts the edges of much of its reporting – for example, caving in on reporting ‘cuts’ as ‘savings’ and recently on the phrase ‘bedroom tax’. This isn’t the case in every part of the BBC (for the same reasons), but still worth noting.
Your points are very well made – More or Less is an Oasis of reason in a desert of dross on the BBC. I assume that you at least can see the email addresses – in which case you will guess who I am. If you’d like to converse more on this topic, I may be able to give you some insights.