I don’t think there could be a much clearer example of the concerted, deliberate and wantonly inaccurate media campaign to undermine the NHS and achieve ulterior motives than today’s BBC News headline about the Leeds Children’s Heart Unit (CHU). It’s also perfectly in line with the track record of media and certain health interests in sullying the reputation of Leeds’ crucial and much-loved unit – a continued demonstration of the determination in certain quarters to see Leeds close even if there is no justification for it. There are too many articles on various facets of that story to link them all here, but if you want to see more a search of the blog for ‘Leeds’ will yield plenty of reading material.
The BBC long ago abandoned any pretence of impartiality on the NHS, instead contributing with gusto to the disinformation on issues like the A&E crisis, or even virtually ignoring major stories like the almost unanimous condemnation of the then-NHS bill (now Health and Social Care Act) or the astonishing march of 51,000 people through the streets of Stafford in support of their wrongly-maligned and now-threatened hospital.
And so it continued with Friday’s ridiculous article on children’s heart surgery at Leeds General Infirmary (LGI), which quoted Professor Brian Jarman:
The headline couldn’t be more emphatic. The only small problem is that it’s completely untrue.
Even according to the same BBC article.
The damning verdict begins
A leading expert on hospital mortality data has told the BBC he would not send his child to the Leeds children’s heart unit because of its death rates.
Prof Sir Brian Jarman of the Dr Foster research group at Imperial College said mortality rates for planned operations at Leeds were the worst in England.
But further down (far enough to avoid the notice of casual readers), it contradicts itself:
The only other hospital with a worse-than-expected death rate was Guys’ and St Thomas’ in London where five children died from its 442 admissions compared with an expected figure of 4.10 (a relative risk ratio of 122.04).
Last time I looked, London was in England. Let’s just double-check:
Yup, there it is.
So, the BBC puts up a strident, absolute headline – and then shows itself beyond doubt to be a liar a few sentences later.
Except it’s still lying.
The article draws on Prof Jarman’s ‘Hospital Standardised Mortality Ratios’ for all England’s CHUs, measuring statistically-adjusted (always to be treated cautiously!) death-rates in children’s cardiac surgery from 2009 to 2012.
Prof Jarman, not long ago, send me a link to a file he had created to depict HSMRs in English CHUs. The file included the graph below, which shows the mortality rates from 2009 to the beginning of 2013 – if you can’t make it out clearly, click on the image to see a larger version:
No, you’re not misreading. Not only is Leeds not the highest, there is not only Guys & St Thomas that is as high or higher. The mortality rate at Alder Hey in Liverpool is not only higher than at Leeds – it’s fully 50% higher.
Yet we don’t have a screaming headline that Alder Hey has mortality rates far, far higher than any other hospital in England, nor do we have Prof Jarman saying that he wouldn’t send his children there.
Incredible, yes?
Well, it ought to be. But it makes perfect sense if there is a campaign at the top to make sure that the thwarted attempt to close Leeds against all the evidence and against common sense succeeds at last.
Prof Jarman’s ‘wouldn’t send my children’ claim is uncannily similar to claims by Roger Boyle, then head of NICOR (the body which styles itself as the ultimate authority on children’s heart surgery statistics) in April.
Boyle was forced to resign for his statement, which was as unprofessional as it was unfounded: the original, detailed report on LGI’s CHU concluded that:
the review team found no reason for on-going closure of the Unit.
It seems that, whoever wants Leeds closed and whatever their reasons, their determination to see it done no matter what is matched only by their lack of imagination and creativity.
So why would Professor Jarman misrepresent the situation in this way?
I’ve had some interactions with Sir Brian on Twitter. Even though we disagree vehemently about his HSMR method and its use/abuse regarding Stafford hospital, he strikes me as a decent man. As pointed out, he has provided me with useful information when I’ve asked for it.
Prof Jarman’s HSMRs were used to falsely smear Mid Staffs NHS Trust as having killed 400-1200 patients. That claim that has now been thoroughly debunked not only on this blog but by the ‘Straight Statistics‘ group, Computer Weekly and even one tiny BBC News web article – and by an important scholarly article which concluded flatly:
The findings of the Mid-Staffordshire Inquiry do not uphold the use of hospital standardized mortality ratios as a screening test for ‘bad’ hospitals
HSMRs were revealed as flawed and unreliable, even aside from the risks that apply to any statistical mortality measurement because of their dependence on data that can be wildly inaccurate and inconsistent (and it must be noted that NICOR knows and has put on record that there are serious problems with its ‘CCAD’ cardiac data, so even the best statistical system will produce wrong results).
But even in ideal circumstances, HSMRs are intrinsically weak because of gaps in the data sets on which they are based. Because of this, they were replaced by a new measure – SHMIs – as the official mortality statistics of the NHS.
Professor Jarman’s credibility as a medical statistician is tied to his system – and his system is widely considered flawed. His job may also be tied to it. The professor is based at the Doctor Foster Unit (DFU) at Imperial College, London – and his unit provides statistical analysis to a commercial company, Doctor Foster Intelligence (DFI).
In his testimony to the Francis public inquiry, Professor Jarman made a crucial admission. The following day, when examining Roger Taylor of DFI, counsel to the inquiry Tom Kark phrased it thus:
When Professor Jarman gave evidence yesterday, he was talking not just about the HSMR, but also about the real-time monitoring tool and all the information that Dr Foster overall can provide. But he seemed very, very unsure about whether the government wanted all of the information that DFI could provide, and went so far as saying that without DFI the government would have shut the unit, Dr Foster Unit, down.
Without DFI’s commercial use of the statistics, the government would have closed down Prof Jarman’s unit.
DFI took DFU’s statistical analyses, used them to create a league table of hospital death rates – and then sold its monitoring system to hospitals for £35,000 a year to help them improve their position in the league table.
Roger Taylor, astonishingly, claimed that this did not represent a conflict of interest – and even more staggeringly, this was accepted by the inquiry. But it obviously is one, nonetheless.
Prof Jarman therefore finds himself in a very difficult position. I believe him to be an essentially honest man – but he is facing great pressure, and an inevitable conflict of interest, to vindicate his system and restore its status as a credible measure of hospital mortality.
This has led him onto shaky ground before. Again to the BBC, to a TV interviewer as well as on the BBC News website, Prof Jarman claimed that there were 20,000 avoidable deaths in the NHS – even though he himself agreed with me on Twitter that HSMRs cannot be used to calculate numbers of avoidable deaths.
A professor with a deep-seated need to justify his magnum opus, NHS executives that appear determined to close the unit at Leeds in spite of judicial rulings against it, misrepresented reports and flawed data. However these factors combined, it’s unmistakably clear that the media portrayal of surgical safety at LGI is deeply unreliable and based on hidden agendas – and that it fits perfectly with an overall strategy to undermine and demonise the NHS in the public perception.
LGI’s CHU is too important to the people who depend on it for us to fall for the ruse. And that’s even more true of the wider NHS.
Don’t be fooled.
Reblogged this on Vox Political and commented:
Latest Skwawkbox blog – Steve does the NHS much better than I do. Get your anti-disinformation pills (figuratively) here!
If only the BBC and other major TV news outlets had the wit and intelligence to provide their viewers with a clear and rational analysis of NHS statistics like this one, Steve, we’d have a much better informed public. Of course, the government shudder at the thought.
There’s another aspect to this as well. Statistics have their place but they don’t generally come with context. Any large hospital performing high risk heart surgery on sick children is going to have a lot of those patients coming through its doors. Some of these children are going to be a lot sicker than others as a result of their various heart conditions and its inevitable that some will be too sick to survive the surgery. The bigger hospitals will therefore be more likely to have more of these deaths. Its not necessarily a measure of their performance at all. And it doesn’t surprise me that Alderhey has the biggest death rate because like Leeds its serves the North of England where there’s more deprivation and ill health. I would expect these death rate figures across all hospital departments to increase more in the North as austerity progresses. And I also expect this morally bankrupt government will capitalise on this to drive their pernicious agenda of hospital closures to make way for the sharks who lurk waiting to make a killing.
Exactly. An attempt is made to ‘standardise’ the varying circumstances and case mixes out, but it can never be perfect and even a small margin of error can make massive differences in the results.
The problem with statistics of any sort is that it is very easy to mislead people who don’t know how to interpret them. In the graph shown I assume the t-bars are the confidence interval of the statistic for each location. If that’s the case I can draw a straight line through all of them bar Alder Hey implying within the accuracy of measurement they all could have the same performance. Except Alder Hey. As you have pointed out this could be because of the hospital’s coverage but there is no way to tell from such a graph. Also since these are statistically adjusted and Steve has pointed out the unreliability of the data input I would have to conclude that the data set suffers from GIGO (Garbage in Garbage Out). In normal statistical analysis the Alder Hey result would not be included because it is anomalous to the rest of the data. All I would conclude from the result is that there is no valid data from which to draw any conclusion about any of the units, but especially it is impossible to single out one of the units as being any different from any other.
Another point made elsewhere is that the expected mortality was 4.2 when talking about integer data. This shows emphatically that the statistical method is simply wrong. This is another example of not just misusing numbers that many of the Parliamentarians do but also using bad statistical analysis to prove ‘what they want’. This is even more disingenuous because it is much more difficult for people to analyse. After all the people providing the analysis are ‘experts’.
Great work again by Steve.
Repiosted at http://jaynelinney.wordpress.com/2013/06/29/jarman-leeds-worst-chu-in-england-except-for-some-others/
Reblogged this on Leeds North West Constituency Labour Party and commented:
Read this to see how determined some people are to close Leeds Children’s Heart Unit – in spite of the evidence.
Reblogged this on Adel & Wharfedale Branch Labour Party and commented:
Read this – it shows just how determined some people are to close Leeds Children’s Heart Unit
Guy’s – “5 children died instead of the expected 4.2”.
Does this sentence have any meaning at all??!!
Not in the slightest!
Yet they have used it to calculate a mortality ratio, from which they conclude that another unit is unsafe! Are they barking mad (or if I was getting really heated – wicked).
i heard a good analogy – take one leg off a beetle, and it can’t walk straight. so take another off to even it up. now it can’t walk at all. if anyone’s in any doubt about the ideology behind the NHS privatisation, and how it is being acted out, i strongly recommend taking the time to watch this video.
it spells it out ckearly, something that makes sense of what i witnessed while i was working for the NHS from 2000 – 2007.
http://www.youtube.com/watch?feature=player_embedded&v=OkTnCtg_Omk
http://paul8ar.wordpress.com/2013/05/28/the-rise-of-the-shadow-state-the-truth-about-outsourcing/
Brilliant info mate, thanks!
Reblogged this on paul8ar.
The BBC article quotes the Leeds’ SMR as 158.93 and Evelina’s as 122.04, so Leeds is greater by those figures (I imagine the data has been updated since Jarmann produced that graph.) You seem not to have provided a link to the article, so I will:
http://www.bbc.co.uk/news/uk-england-leeds-23086203
The article also states that at Evelina “five children died from its 442 admissions compared with an expected figure of 4.10.” The eagle eyed amongst you will have noticed that IRL child deaths tend to occur in integer numbers, so I wouldn’t get to excited about the death rate at Evelina.
The BBC article quotes the Leeds’ SMR as 158.93 and Evelina’s as 122.04, so Leeds is greater by those figures (I imagine the data has been updated since Jarmann produced that graph.) You seem not to have provided a link to the article, so I will:
h ttp://www.bbc.co.uk/news/uk-england-leeds-23086203
The article also states that at Evelina “five children died from its 442 admissions compared with an expected figure of 4.10.” The eagle eyed amongst you will have noticed that IRL child deaths tend to occur in integer numbers, so I wouldn’t get to excited about the death rate at Evelina.
No, remember they’re referring to 2009-12, so the Feb 13 nos are the same or more current.
Well clearly something’s changed, because the numbers quoted in the article are different from those in your graph. It quotes Leeds’ SMR as 158.93, and Guys’ as 122.04, whereas your graph has them both at roughly 100. We’ll have to wait until the full programme is broadcast to find the details. Also, a new batch of data was released in April after the Leeds closure, so it might related to that:
http://www.england.nhs.uk/wp-content/uploads/2013/04/finl-rep-mort-paed-card-surg-2009-12.pdf
(IIRC Dr Foster doesn’t have direct access to the data, and has to get it second hand, so that might explain why the earlier data they used might have been incomplete).
Just to be clear Guys’ SMR was described as “worse-than-expected” (albeit by a fraction of a full number) not worse than Leeds.
Surely any measure that can put you in a different category as a result of one extra death in 400 cases has to be used with EXTREME caution, or better still not interpreted by anyone to mean anything in particular.
But never mind all that, maybe the good people of Leeds are quite happy to accept an additional fraction of a % mortality for having the service accessible for the majority (especially if they appreciate that such a small difference, which could well be the other way round next year, is meaningless). A lot of this stuff is hard for the public to grasp when they are fed such ridiculous interpretations but what they do understand is whether they can reasonably access services. Increasingly they are realising that they can’t.
It’s probably worth waiting for the full interview to be broadcast to see if it elaborates on the methodology but it would seem that some measure of statistical significance was used:
“Prof Jarman acknowledged that the number of deaths was small but said the data was checked against controls to ensure there was only a small chance of random findings.
“The figures are not very high but if your child is one of those it does not matter,” he said.””
” (especially if they appreciate that such a small difference, which could well be the other way round next year, is meaningless)”
The LGI came in the top three for mortality in all of the three previous triennia, as well, and of the three units that came above it at one time or another, two had their SMRs revised down because some procedures had been misclassified, and the other has since been shut down due to high mortality:
http://www.leeds.gov.uk/docs/2011-01-11%20JCPCT%205%20reports%201.pdf
“But never mind all that, maybe the good people of Leeds are quite happy to accept an additional fraction of a % mortality for having the service accessible for the majority.”
Maybe, but that has to be an informed decision. Roger Boyle and Brian Jarman would take a different view, which I find understandable in the circumstances, particularly as two of Leeds’ next nearest units have mortality rates in the bottom three. It’s worth noting that Leeds isn’t being shut down (if it is) because of these issues, but as part of a wider review due to the fact that the wider way that children’s heart surgery is being delivered has been judged to be unsafely spread out.
Anyway the point that seems to be passing people by is that the bulk of Steve’s argument (which he puts in typically strident terms- describing people as liars) is based on what could most charitably be considered a remarkable lapse in reading comprehension- see my previous comments.
If, as you say, the LGI has been high for mortality for some time then presumably the DoH and experts with lower mortality have been in to look at possible reasons and make sure they are put right by use of relevant assistance. That is what a collaborative system would do, wouldn’t it?
or
are you suggesting that either
a) the differences were too small to warrant such intervention
or
b) there is, in fact, a competitive market at work here where the other units have a vested interest in the LGI doing worse and thus being an easier target to close (which would be unacceptable surely?)
I only buy the volume argument for surgical procedures up to a threshold point – which is usually surprisingly low. Thereafter it’s got to be procedural differences in either the surgery or the supportive care (ie all differences are remediable and should not be used to remove services from populated locations on a random basis).
You mean the review that was judged to have been illegal and which would have left huge tracts of England with hours, and 100s of miles, to travel to centres that offered no statistically significant advantages? Move the surgeons, if you really think Newcastle or another unit is that much better – not the population.
As for stridency or calling the BBC a liar, what else do you call making a headline claim that the BBC’s own article contradicted? Calling Prof J a decent man holding a wrong position is hardly overly aggressive.
Sadly typical contribution.