In the immediate aftermath of the release of the Francis Report into events at Mid Staffordshire NHS Foundation Trust, I identified that David Cameron’s crocodile-tears and apparent humility were just a feint that would quickly turn into an attack on the NHS nationally (yet another front in their all-out war on it), using Mid Staffs as a template for attacking other hospitals – and Labour.
This morning’s headlines – covered by the BBC (website and news channel) and the right-wing press – about Professor Sir Brian Jarman’s claim that 20,000 NHS deaths could have been prevented come on the back of a 2-week long assault by Health Secretary Jeremy Hunt on Labour’s supposed failings leading to Mid Staffs.
The claims are utter nonsense – but they are being used by the press and the government as
A sledgehammer to smash the NHS – and Labour
I’ve already shown, at considerable length, that the headlines you will have seen in the media about 400-1200 ‘needless deaths’ at Stafford are utterly unfounded. Prof Jarman’s latest claim takes all the same errors and distortions that the media and government have shamelessly stated as fact – and then compounds them into even greater distortion and error.
Why it’s all nonsense
Prof Jarman’s errs on so many fronts that it’s almost unbelievable that a clearly intelligent man can make such fundamental mistakes. That a statistician can so plainly ignore obvious statistical truths suggests a deep-rooted personal agenda – one which is shamelessly being exploited by the Tory-friendly media to foster the government’s aim of destroying the NHS as a national, social institution.
Those errors are both logical and factual.
The ‘100’ error and the danger of the average
As I explained in my post on the real story of Mid Staffs, Prof Jarman’s HSMR (hospital standardised mortality ratio) system rates English hospitals according to where they sit according to the average death rate for the nation. A hospital hitting exactly the average rate would receive a ‘score’ of 100. A hospital doing better than average would have an HSMR below 100, and one doing worse would be over 100.
So far, so clear – I hope. But here’s the key fact: every year, the system is ‘rebased‘ – the averages are re-measured and ‘100’ is re-calibrated to the new average.
This leads to 3 key problems:
- Because of how averages work, unless by some miracle every single hospital in the country got exactly the same score, you will always have some hospitals above 100 and some below. This does not mean the ‘extra’ deaths in those hospitals were avoidable – it just means that somebody has to be above the line because it’s an average.
- Because the ‘100-line’ moves every year, a hospital can maintain exactly the same standard in one year as it achieved in the previous year – and yet can score below 100 one year and above 100 the next. The performance of the hospital did not get worse. The line just moved. It’s not only wrong but ridiculous to extrapolate ‘extra’ or ‘avoidable’ deaths from a position above or below a line that moves every year.
- Leading on from number 2 – and it’s impossible to overstate this – there is no ‘standard’ rate of deaths from a particular illness. No expert clinicians are sitting down together and saying ‘Yes, we agree that out of every 100 patients with an intracranial bleed, this many are going to die’. No. All that happens is that the average for the previous year becomes the re-calibrated ‘expected death’ figure for the following year. This means that HSMRs are measuring the success/failure in achieving/beating/failing to bear a target that moves every year – and has no basis in clinical expertise. It’s just a number.
In his testimony to the Francis inquiry, Prof Jarman claimed that he had to present the figures this way because the English are ‘simple-minded‘ – but in doing so he has committed a fundamental error of logic worthy of a simpleton.
These fundamental logical errors mean that even if everything else was perfect, HSMR scores over 100 cannot be used to calculate avoidable deaths.
But everything is not all perfect with HSMRs
Prof Jarman’s claims are based on his repeating his assumptions about Mid Staffs to arrive at an even larger death figure for the 14 hospitals that are under investigation for having similar HSMR scores to that hospital. But if the assumptions don’t stand up for Mid Staffs, then they are meaningless for the larger set of hospitals. And they are meaningless.
I won’t repeat everything I wrote in my earlier post, as this one would become unreadably long. I’ll summarise a few key points, and if you want to check the details, please refer to the full post.
But I also have additional information.
Earlier this week, I had the opportunity to meet with Sandra Haynes Kirkbright – the coding manager that was brought in by Mid Staffs to address the problems they knew they had with their coding. She had read my first post on the Mid Staffs HSMRs and told me
I thought, ‘Someone gets it!’ I thought ‘How did he get inside my head?!”
She therefore let me know that she wanted to meet me and give me more information crucial to a proper understanding of what happened at Mid Staffs – and why the claims of press and politicians are so deeply misleading.
The information I got from Ms Kirkbright (who had begged to be allowed to testify to the Francis inquiry in person but been forbidden by the Trust’s lawyers) sheds even more light on the Mid Staffs HSMRs – and that light shows them (and the headlines and assumptions that have been based on them) to be even more full of holes than I already knew.
I’m going to write a separate post on the full discussions with Ms Kirkbright, as they ranged more widely than Mid Staffs. However, while the claims in the media about 20,000 needless deaths are absolutely risible, they are also potentially catastrophic for the NHS if they are widely believed.
Because of the urgency of getting good information out into circulation to counteract the invidious nonsense, for this post I’m going to pick out some of the key points about Mid Staffs and its HSMRs that have not, as far as I’m aware, been covered at all by the mainstream media.
First the earlier post, summarised:
Only one ‘excess’ death at Mid Staffs
HSMRs are a statistical device. If you want to be sure whether deaths were avoidable or not, you need to look in detail at the case notes for each patient. The doctor in charge of the Independent Case Note Review (INCR) was asked by the inquiry how many ‘excess’ deaths he had discovered among all of the cases for which families asked for a review. His answer was telling – but has been almost completely ignored:
Perhaps one such death.
Rubbish in, rubbish out
Prof Jarman’s HSMR scores are based on comparing ‘expected’ deaths with ‘observed’ deaths. Each condition has its own typical death rate, so if more people die than would be expected for any recorded condition, an HSMR of over 100 will result.
But the conditions generating Mid Staffs’ HSMRs often did not reflect reality. For example:
- HSMRs are based on the first diagnosed condition, even if a much more serious condition is subsequently diagnosed.
- HSMRs can be adjusted by taking into account ‘co-morbidities – conditions that existed alongside the main diagnosis. But Mid Staffs coding manager was absent on long-term sick leave, and co-morbidities were not entered. Co-morbidities increase the ‘expected’ death rate – so leaving them out will make the ‘expected’ figure too small and lead to an ‘excess’ in the observed figures.
- Doctors were not aware of the significance and consequences of poorly-recorded or missing diagnoses and co-morbidities. Under pressure because of short-staffing, many gave only minimal effort to coding and recorded onlyone diagnosis – often an inappropriate one.
- Mid Staffs was not recording ‘zero-length stays’ (people who came in, got treated and went home the same day or the next day). Since those people would, by definition, not have died in the hospital, including them would lower the overall death rates and bring down HSMRs. Mid Staffs’ HSMRs were therefore inflated by not including them.
- ‘Palliative care’ is treatment given to patients that are going to die no matter what. A code, ‘Z51.5‘, was introduced for palliative care so that people dying from incurable diseases would not worsen the HSMR score. Mid Staffs was not using this code until 2008.
There are many more contributory factors, whose details you can find in the earlier post. The cumulative effect of all these factors was a huge inflation of Mid Staffs HSMRs.
88
When Ms Kirkbright arrived at Mid Staffs, she carried out a re-coding exercise on past deaths. This re-coding corrected the absent Z51.5 code and used the case notes to add in the co-morbidities (what is known in the jargon as ‘depth of coding’) that was missing. This brought down Mid Staffs’ HSMR to 88 – well below the national average death rate.
This fact has been completely ignored by almost all the mainstream media.
The new information
The above facts on their own are enough to show that Mid Staffs was not ‘killing hundreds of patients‘. And if the ‘excess deaths’ there never happened – then the figures at other hospitals are similarly meaningless in terms of identifying ‘excess’ deaths.
But the new information I received from Sandra Haynes Kirkbright makes the case even more watertight.
The external audit
It’s important to know that the statistics used by Prof Jarman’s system are not entered specifically for the purpose of recording mortality rates. HSMRs are drawn from codes that have to be entered by hospitals on the treatments they provide and the outcomes they achieve in order for them to be paid for their work under the ‘payment by results‘ (PBR) scheme.
In his testimony to the Francis inquiry, Prof Jarman implied that Mid Staffs had ‘gamed‘ the system to bring down their HSMR score – basically, that they were fiddling the figures in order to improve their HSMRs.
But because coding could be used to increase a hospital’s income, the government’s (now-defunct) Audit Commission (AC) carried out audits of Trusts, to make sure that their figures are legitimate.
Following the re-coding exercise, the AC arrived and audited Mid Staffs’ coding – and awarded an unprecedented score of 97-98% for accuracy. Under no circumstances could Prof Jarman’s allegations be correct.
More ‘rubbish in, rubbish out’
Ms Kirkbright told me of a Trust she knew of that was routinely ‘gaming’ its codes by 15-20% ‘so as not to appear outrageous’. This was not for HSMR purposes, but to increase its income – but it would also improve the HSMR score.
Similarly, Royal Bolton Hospital stands accused of inflating its septicaemia coding to increase its income – but again, this would impact on HSMRs.
If some hospitals in the country are playing the system because of funding pressures, then the significance of the higher HSMRs of the 14 hospitals under investigation is just as likely to mean that they are more honest as it is to mean more patients are dying avoidably. In fact, in the context of everything else, it’s more likely.
‘Receptionist triage’ – another myth
The misleading articles on Mid Staffs made much of the claim that patients were so thirsty that ‘many’ had to drink from flower vases to assuage their thirst. Flower vases have been banned at Stafford and Cannock since the mid-90s – so the story is either completely unfounded or based on one exception (most likely a confused patient).
Similarly, it has been frequently claimed that the Accident and Emergency unit (A&E) at Stafford was so badly run that triage (the initial assessment of patients) was carried out by receptionists.
This is simply untrue. To admit a patient, receptionists had to complete a set of information on the hospitals IT system. The system was configured so that users could only go to the next input page after they had completed the preceding one – including entering a name for the member of staff who had assessed the patient.
Receptionists (who thought ‘triage’ was pronounced ‘tree-ar-gee’!) often did not know or were unable to spell the name of the doctors who assessed patients, so – to save time and because they couldn’t go further without entering a name – they routinely entered their own.
That this scenario has been so distorted in reporting on the Mid Staffs situation is a very good indicator of the (abysmally low) level of reporting and investigation behind the prurient headlines.
Unqualified coders ‘drowning’
Clinical coding is a skill with its own qualification. It takes up to two years to learn the necessary skills, including procedures and conventions of coding and detailed anatomical study to ensure that notes are properly understood and coded. Ms Kirkbright was a qualified coder in the US – and then had to study again to gain the UK qualification.
For a qualified, full-time coder, the average coding rate is around 7,500 cases per year. Mid Staffs treats around 60,000 patients a year – so needed 8 full-time, qualified coders.
In the absence of the coding manager, the coders trying to keep up with the caseload at Mid Staffs numbered 5 – part-time and unqualified.
In this context, it’s no surprise that coding was inaccurate and lacked ‘depth’ (co-morbidities) – massively pushing up HSMRs. To get on top of the coding, Ms Kirkbright and 11 others were eventually employed to cover the hours necessary to manage the workload properly.
If Mid Staffs was unable for years to employ and train sufficient coders to code properly , it’s certain that there are many other Trusts in a similar situation (many do not even have a clinical coding policy) – making the whole HSMR ranking system meaningless.
No training
Doctor Fosters Intelligence (DFI), who publish HSMR tables in their ‘Good Hospital Guide‘, and which finances Prof Jarman’s Doctor Fosters Unit, provides no training on how to code to ensure accurate HSMRs.
It does provide training on how to use its Real Time Monitoring (RTM) tool which allows Trusts to use and monitor HSMRs. It charges £35,000 a year for this service – and Trusts with poor HSMR scores are more likely to want to purchase it.
Can you say ‘conflict of interest’?
More ‘first (mis)diagnosis’ examples
As already stated, HSMRs are based on ‘first diagnosis’. My earlier post gave an example of a diagnosis of a broken shinbone that was later discovered to be caused by bone cancer. Broken legs have very low expected mortality rates. Bone cancer does not. But the HSMR programme would measure that death against the broken leg, not the cancer.
Ms Kirkbright provided me with other examples of similar issues. For example, if a patient arrives unconscious and the ambulance driver reports ‘He fainted’, fainting would be the first diagnosis. If that patient is then discovered to have had a serious stroke and dies, that death will be recorded against ‘fainting’ – a condition with a very low death-rate – resulting in an inflated HSMR.
Similarly, Ms Kirkbright once challenged the NHSIC about the fact that post-mortem results are not included in the coding. She was told:
We don’t do death coding – we do morbidity (disease) coding
to which she responded:
Do you think he developed congestive heart failure after he died?!
The data on which HSMRs rely must inevitably be missing the real cause of death in many cases – meaning that deaths are allocated to the wrong diagnoses and making the HSMR tables meaningless.
DOA or, It’s worse than that – he’s dead, Jim!
Because of a misunderstanding of procedure, A&E staff at Stafford were admitting patients who were already dead on arrival at the unit. The correct procedure would have been to send such a patient to the recovery unit to see if they could be revived, and then admit them if they were resuscitated. But dead patients were routinely admitted as patients, declared dead – and then form part of the hospital mortality statistics even though they were already dead when they got there.
Again, this would add to the cumulative error in the statistics – and if it was happening at Stafford, it’s almost certain to be happening elsewhere.
Rocking horse droppings (or A&E coding)
As mentioned above, hospitals are not paid for ‘zero-length stays’ – people who come into A&E but are discharged on the same day or next day. A&E treatments are funded on a simple 3-tier system according to severity rather than on the specific ailment or injury. As a result Stafford’s finance manager was not bothering to send the codes forward for inclusion in the PBR data – not realising that it would have a massive impact on the HSMR scores.
Because of the low priority allocated to it, A&E had only 2 coders – neither of whom had any training, and who just entered whatever was ticked on the back of the Casualty Card, even if it made no sense, or made a ‘best guess’ if it was unclear.
This meant that A&E coding was ‘all over the place’ – and therefore generated completely unreliable HSMRs.
Measures were taken at Stafford to address all these problems, with the result that the Trust’s HSMR scores came down dramatically. But if these and similar issues remain unaddressed at other Trusts, then the data that the whole HSMR system relies upon is close to meaningles.
And it certainly isn’t solid enough to support the wild and emphatic claims made by Prof Jarman and the media that are exploiting him.
Motives
Detective shows often make a play of the ‘tripod’ that has to be in place before a suspect can be charged with a crime: means, opportunity and motive. Prof Jarman and the media and politicians have access to the HSMR data. They have the means to publicise their claims. But what about motive?
For the Tories and their media allies, the motive is clear enough. The massive distortions around events at Mid Staffs present a perfect opportunity to attack that Trust – and to use it as a basis for attacking others. The right has hated the NHS since its inception – protestations that they love it are for public consumption only – and the current crop is on record as wanting to end it.
As for Professor Jarman – who knows? But he testified to the Francis inquiry that he felt Labour had not treated his system with the seriousness he believes it deserves – and that the Tory-led government has been much ‘friendlier’ towards it.
Could it be revenge on Andy Burnham for the perceived slight when Burnham – with complete justification as it turns out – rejected the supposed importance of HSMRs and used other methods to assess hospitals and address their issues?
Could it be that Prof Jarman’s testimony to Francis and his subsequent comments reveal a man who is so deeply attached to his system that he can’t accept its flaws and will go to great lengths to vindicate it?
You’ll have to consider and decide for yourself. But what’s absolutely clear is that, whether knowingly or not, Prof Jarman has positioned himself perfectly to be exploited by the ideological enemies of the NHS as a tool for undermining it – and for attacking Labour’s entirely justified public perception as the founders and supporters healthcare ‘free to all at the point of use’.
I don’t doubt that Prof Jarman wants his system to work for the good of patients, but taking flawed results from one hospital and multiplying those across 14 – and then trumpeting a claim of 20,000 avoidable deaths to the media – is reckless in the extreme and has played right into the Tories’ grasping hands.
The threat is serious, because a lot of people will believe what’s being said and written just because it’s in the media and has apparent statistical/scientific support. Very few will look beyond the soundbites to see whether they stand up to scrutiny.
So please, if you’ve read this article and agree with its premise, spread the word. We need good information out their to counteract the absolute tripe that’s being force-fed to the British public.
And please also consider supporting CCGWatch, which is aimed at combating another of the key Tory attacks on the NHS: stealth-privatisation.