In the immediate aftermath of the release of the Francis Report into events at Mid Staffordshire NHS Foundation Trust, I identified that David Cameron’s crocodile-tears and apparent humility were just a feint that would quickly turn into an attack on the NHS nationally (yet another front in their all-out war on it), using Mid Staffs as a template for attacking other hospitals – and Labour.
This morning’s headlines – covered by the BBC (website and news channel) and the right-wing press – about Professor Sir Brian Jarman’s claim that 20,000 NHS deaths could have been prevented come on the back of a 2-week long assault by Health Secretary Jeremy Hunt on Labour’s supposed failings leading to Mid Staffs.
The claims are utter nonsense – but they are being used by the press and the government as
A sledgehammer to smash the NHS – and Labour
I’ve already shown, at considerable length, that the headlines you will have seen in the media about 400-1200 ‘needless deaths’ at Stafford are utterly unfounded. Prof Jarman’s latest claim takes all the same errors and distortions that the media and government have shamelessly stated as fact – and then compounds them into even greater distortion and error.
Why it’s all nonsense
Prof Jarman’s errs on so many fronts that it’s almost unbelievable that a clearly intelligent man can make such fundamental mistakes. That a statistician can so plainly ignore obvious statistical truths suggests a deep-rooted personal agenda – one which is shamelessly being exploited by the Tory-friendly media to foster the government’s aim of destroying the NHS as a national, social institution.
Those errors are both logical and factual.
The ‘100’ error and the danger of the average
As I explained in my post on the real story of Mid Staffs, Prof Jarman’s HSMR (hospital standardised mortality ratio) system rates English hospitals according to where they sit according to the average death rate for the nation. A hospital hitting exactly the average rate would receive a ‘score’ of 100. A hospital doing better than average would have an HSMR below 100, and one doing worse would be over 100.
So far, so clear – I hope. But here’s the key fact: every year, the system is ‘rebased‘ – the averages are re-measured and ‘100’ is re-calibrated to the new average.
This leads to 3 key problems:
- Because of how averages work, unless by some miracle every single hospital in the country got exactly the same score, you will always have some hospitals above 100 and some below. This does not mean the ‘extra’ deaths in those hospitals were avoidable – it just means that somebody has to be above the line because it’s an average.
- Because the ‘100-line’ moves every year, a hospital can maintain exactly the same standard in one year as it achieved in the previous year – and yet can score below 100 one year and above 100 the next. The performance of the hospital did not get worse. The line just moved. It’s not only wrong but ridiculous to extrapolate ‘extra’ or ‘avoidable’ deaths from a position above or below a line that moves every year.
- Leading on from number 2 – and it’s impossible to overstate this – there is no ‘standard’ rate of deaths from a particular illness. No expert clinicians are sitting down together and saying ‘Yes, we agree that out of every 100 patients with an intracranial bleed, this many are going to die’. No. All that happens is that the average for the previous year becomes the re-calibrated ‘expected death’ figure for the following year. This means that HSMRs are measuring the success/failure in achieving/beating/failing to bear a target that moves every year – and has no basis in clinical expertise. It’s just a number.
In his testimony to the Francis inquiry, Prof Jarman claimed that he had to present the figures this way because the English are ‘simple-minded‘ – but in doing so he has committed a fundamental error of logic worthy of a simpleton.
These fundamental logical errors mean that even if everything else was perfect, HSMR scores over 100 cannot be used to calculate avoidable deaths.
But everything is not all perfect with HSMRs
Prof Jarman’s claims are based on his repeating his assumptions about Mid Staffs to arrive at an even larger death figure for the 14 hospitals that are under investigation for having similar HSMR scores to that hospital. But if the assumptions don’t stand up for Mid Staffs, then they are meaningless for the larger set of hospitals. And they are meaningless.
I won’t repeat everything I wrote in my earlier post, as this one would become unreadably long. I’ll summarise a few key points, and if you want to check the details, please refer to the full post.
But I also have additional information.
Earlier this week, I had the opportunity to meet with Sandra Haynes Kirkbright – the coding manager that was brought in by Mid Staffs to address the problems they knew they had with their coding. She had read my first post on the Mid Staffs HSMRs and told me
I thought, ‘Someone gets it!’ I thought ‘How did he get inside my head?!”
She therefore let me know that she wanted to meet me and give me more information crucial to a proper understanding of what happened at Mid Staffs – and why the claims of press and politicians are so deeply misleading.
The information I got from Ms Kirkbright (who had begged to be allowed to testify to the Francis inquiry in person but been forbidden by the Trust’s lawyers) sheds even more light on the Mid Staffs HSMRs – and that light shows them (and the headlines and assumptions that have been based on them) to be even more full of holes than I already knew.
I’m going to write a separate post on the full discussions with Ms Kirkbright, as they ranged more widely than Mid Staffs. However, while the claims in the media about 20,000 needless deaths are absolutely risible, they are also potentially catastrophic for the NHS if they are widely believed.
Because of the urgency of getting good information out into circulation to counteract the invidious nonsense, for this post I’m going to pick out some of the key points about Mid Staffs and its HSMRs that have not, as far as I’m aware, been covered at all by the mainstream media.
First the earlier post, summarised:
Only one ‘excess’ death at Mid Staffs
HSMRs are a statistical device. If you want to be sure whether deaths were avoidable or not, you need to look in detail at the case notes for each patient. The doctor in charge of the Independent Case Note Review (INCR) was asked by the inquiry how many ‘excess’ deaths he had discovered among all of the cases for which families asked for a review. His answer was telling – but has been almost completely ignored:
Perhaps one such death.
Rubbish in, rubbish out
Prof Jarman’s HSMR scores are based on comparing ‘expected’ deaths with ‘observed’ deaths. Each condition has its own typical death rate, so if more people die than would be expected for any recorded condition, an HSMR of over 100 will result.
But the conditions generating Mid Staffs’ HSMRs often did not reflect reality. For example:
- HSMRs are based on the first diagnosed condition, even if a much more serious condition is subsequently diagnosed.
- HSMRs can be adjusted by taking into account ‘co-morbidities – conditions that existed alongside the main diagnosis. But Mid Staffs coding manager was absent on long-term sick leave, and co-morbidities were not entered. Co-morbidities increase the ‘expected’ death rate – so leaving them out will make the ‘expected’ figure too small and lead to an ‘excess’ in the observed figures.
- Doctors were not aware of the significance and consequences of poorly-recorded or missing diagnoses and co-morbidities. Under pressure because of short-staffing, many gave only minimal effort to coding and recorded onlyone diagnosis – often an inappropriate one.
- Mid Staffs was not recording ‘zero-length stays’ (people who came in, got treated and went home the same day or the next day). Since those people would, by definition, not have died in the hospital, including them would lower the overall death rates and bring down HSMRs. Mid Staffs’ HSMRs were therefore inflated by not including them.
- ‘Palliative care’ is treatment given to patients that are going to die no matter what. A code, ‘Z51.5‘, was introduced for palliative care so that people dying from incurable diseases would not worsen the HSMR score. Mid Staffs was not using this code until 2008.
There are many more contributory factors, whose details you can find in the earlier post. The cumulative effect of all these factors was a huge inflation of Mid Staffs HSMRs.
88
When Ms Kirkbright arrived at Mid Staffs, she carried out a re-coding exercise on past deaths. This re-coding corrected the absent Z51.5 code and used the case notes to add in the co-morbidities (what is known in the jargon as ‘depth of coding’) that was missing. This brought down Mid Staffs’ HSMR to 88 – well below the national average death rate.
This fact has been completely ignored by almost all the mainstream media.
The new information
The above facts on their own are enough to show that Mid Staffs was not ‘killing hundreds of patients‘. And if the ‘excess deaths’ there never happened – then the figures at other hospitals are similarly meaningless in terms of identifying ‘excess’ deaths.
But the new information I received from Sandra Haynes Kirkbright makes the case even more watertight.
The external audit
It’s important to know that the statistics used by Prof Jarman’s system are not entered specifically for the purpose of recording mortality rates. HSMRs are drawn from codes that have to be entered by hospitals on the treatments they provide and the outcomes they achieve in order for them to be paid for their work under the ‘payment by results‘ (PBR) scheme.
In his testimony to the Francis inquiry, Prof Jarman implied that Mid Staffs had ‘gamed‘ the system to bring down their HSMR score – basically, that they were fiddling the figures in order to improve their HSMRs.
But because coding could be used to increase a hospital’s income, the government’s (now-defunct) Audit Commission (AC) carried out audits of Trusts, to make sure that their figures are legitimate.
Following the re-coding exercise, the AC arrived and audited Mid Staffs’ coding – and awarded an unprecedented score of 97-98% for accuracy. Under no circumstances could Prof Jarman’s allegations be correct.
More ‘rubbish in, rubbish out’
Ms Kirkbright told me of a Trust she knew of that was routinely ‘gaming’ its codes by 15-20% ‘so as not to appear outrageous’. This was not for HSMR purposes, but to increase its income – but it would also improve the HSMR score.
Similarly, Royal Bolton Hospital stands accused of inflating its septicaemia coding to increase its income – but again, this would impact on HSMRs.
If some hospitals in the country are playing the system because of funding pressures, then the significance of the higher HSMRs of the 14 hospitals under investigation is just as likely to mean that they are more honest as it is to mean more patients are dying avoidably. In fact, in the context of everything else, it’s more likely.
‘Receptionist triage’ – another myth
The misleading articles on Mid Staffs made much of the claim that patients were so thirsty that ‘many’ had to drink from flower vases to assuage their thirst. Flower vases have been banned at Stafford and Cannock since the mid-90s – so the story is either completely unfounded or based on one exception (most likely a confused patient).
Similarly, it has been frequently claimed that the Accident and Emergency unit (A&E) at Stafford was so badly run that triage (the initial assessment of patients) was carried out by receptionists.
This is simply untrue. To admit a patient, receptionists had to complete a set of information on the hospitals IT system. The system was configured so that users could only go to the next input page after they had completed the preceding one – including entering a name for the member of staff who had assessed the patient.
Receptionists (who thought ‘triage’ was pronounced ‘tree-ar-gee’!) often did not know or were unable to spell the name of the doctors who assessed patients, so – to save time and because they couldn’t go further without entering a name – they routinely entered their own.
That this scenario has been so distorted in reporting on the Mid Staffs situation is a very good indicator of the (abysmally low) level of reporting and investigation behind the prurient headlines.
Unqualified coders ‘drowning’
Clinical coding is a skill with its own qualification. It takes up to two years to learn the necessary skills, including procedures and conventions of coding and detailed anatomical study to ensure that notes are properly understood and coded. Ms Kirkbright was a qualified coder in the US – and then had to study again to gain the UK qualification.
For a qualified, full-time coder, the average coding rate is around 7,500 cases per year. Mid Staffs treats around 60,000 patients a year – so needed 8 full-time, qualified coders.
In the absence of the coding manager, the coders trying to keep up with the caseload at Mid Staffs numbered 5 – part-time and unqualified.
In this context, it’s no surprise that coding was inaccurate and lacked ‘depth’ (co-morbidities) – massively pushing up HSMRs. To get on top of the coding, Ms Kirkbright and 11 others were eventually employed to cover the hours necessary to manage the workload properly.
If Mid Staffs was unable for years to employ and train sufficient coders to code properly , it’s certain that there are many other Trusts in a similar situation (many do not even have a clinical coding policy) – making the whole HSMR ranking system meaningless.
No training
Doctor Fosters Intelligence (DFI), who publish HSMR tables in their ‘Good Hospital Guide‘, and which finances Prof Jarman’s Doctor Fosters Unit, provides no training on how to code to ensure accurate HSMRs.
It does provide training on how to use its Real Time Monitoring (RTM) tool which allows Trusts to use and monitor HSMRs. It charges £35,000 a year for this service – and Trusts with poor HSMR scores are more likely to want to purchase it.
Can you say ‘conflict of interest’?
More ‘first (mis)diagnosis’ examples
As already stated, HSMRs are based on ‘first diagnosis’. My earlier post gave an example of a diagnosis of a broken shinbone that was later discovered to be caused by bone cancer. Broken legs have very low expected mortality rates. Bone cancer does not. But the HSMR programme would measure that death against the broken leg, not the cancer.
Ms Kirkbright provided me with other examples of similar issues. For example, if a patient arrives unconscious and the ambulance driver reports ‘He fainted’, fainting would be the first diagnosis. If that patient is then discovered to have had a serious stroke and dies, that death will be recorded against ‘fainting’ – a condition with a very low death-rate – resulting in an inflated HSMR.
Similarly, Ms Kirkbright once challenged the NHSIC about the fact that post-mortem results are not included in the coding. She was told:
We don’t do death coding – we do morbidity (disease) coding
to which she responded:
Do you think he developed congestive heart failure after he died?!
The data on which HSMRs rely must inevitably be missing the real cause of death in many cases – meaning that deaths are allocated to the wrong diagnoses and making the HSMR tables meaningless.
DOA or, It’s worse than that – he’s dead, Jim!
Because of a misunderstanding of procedure, A&E staff at Stafford were admitting patients who were already dead on arrival at the unit. The correct procedure would have been to send such a patient to the recovery unit to see if they could be revived, and then admit them if they were resuscitated. But dead patients were routinely admitted as patients, declared dead – and then form part of the hospital mortality statistics even though they were already dead when they got there.
Again, this would add to the cumulative error in the statistics – and if it was happening at Stafford, it’s almost certain to be happening elsewhere.
Rocking horse droppings (or A&E coding)
As mentioned above, hospitals are not paid for ‘zero-length stays’ – people who come into A&E but are discharged on the same day or next day. A&E treatments are funded on a simple 3-tier system according to severity rather than on the specific ailment or injury. As a result Stafford’s finance manager was not bothering to send the codes forward for inclusion in the PBR data – not realising that it would have a massive impact on the HSMR scores.
Because of the low priority allocated to it, A&E had only 2 coders – neither of whom had any training, and who just entered whatever was ticked on the back of the Casualty Card, even if it made no sense, or made a ‘best guess’ if it was unclear.
This meant that A&E coding was ‘all over the place’ – and therefore generated completely unreliable HSMRs.
Measures were taken at Stafford to address all these problems, with the result that the Trust’s HSMR scores came down dramatically. But if these and similar issues remain unaddressed at other Trusts, then the data that the whole HSMR system relies upon is close to meaningles.
And it certainly isn’t solid enough to support the wild and emphatic claims made by Prof Jarman and the media that are exploiting him.
Motives
Detective shows often make a play of the ‘tripod’ that has to be in place before a suspect can be charged with a crime: means, opportunity and motive. Prof Jarman and the media and politicians have access to the HSMR data. They have the means to publicise their claims. But what about motive?
For the Tories and their media allies, the motive is clear enough. The massive distortions around events at Mid Staffs present a perfect opportunity to attack that Trust – and to use it as a basis for attacking others. The right has hated the NHS since its inception – protestations that they love it are for public consumption only – and the current crop is on record as wanting to end it.
As for Professor Jarman – who knows? But he testified to the Francis inquiry that he felt Labour had not treated his system with the seriousness he believes it deserves – and that the Tory-led government has been much ‘friendlier’ towards it.
Could it be revenge on Andy Burnham for the perceived slight when Burnham – with complete justification as it turns out – rejected the supposed importance of HSMRs and used other methods to assess hospitals and address their issues?
Could it be that Prof Jarman’s testimony to Francis and his subsequent comments reveal a man who is so deeply attached to his system that he can’t accept its flaws and will go to great lengths to vindicate it?
You’ll have to consider and decide for yourself. But what’s absolutely clear is that, whether knowingly or not, Prof Jarman has positioned himself perfectly to be exploited by the ideological enemies of the NHS as a tool for undermining it – and for attacking Labour’s entirely justified public perception as the founders and supporters healthcare ‘free to all at the point of use’.
I don’t doubt that Prof Jarman wants his system to work for the good of patients, but taking flawed results from one hospital and multiplying those across 14 – and then trumpeting a claim of 20,000 avoidable deaths to the media – is reckless in the extreme and has played right into the Tories’ grasping hands.
The threat is serious, because a lot of people will believe what’s being said and written just because it’s in the media and has apparent statistical/scientific support. Very few will look beyond the soundbites to see whether they stand up to scrutiny.
So please, if you’ve read this article and agree with its premise, spread the word. We need good information out their to counteract the absolute tripe that’s being force-fed to the British public.
And please also consider supporting CCGWatch, which is aimed at combating another of the key Tory attacks on the NHS: stealth-privatisation.
First you decide on your desired outcome, then you invent a pseudoscence to justify your actions getting to that outcome to the rubes. That’s what governments do to get the results they want and sneak it past the public. We’re seeing it here just as we see it in the Atos testing and the use of the perverted biopsychosocial model.
Fascinating, I was just reading the M&M section in the Francis report as I was curious to see how the conclusions had been reached, The problem with the Executive summary is that the abstracted summary looses the detail in the other 2 volumes. I am sure most will only read the summary, and many only scan that.
You are drawing attention to some really important issues. I left the NHS some years ago and am working abroad, but have sent this on to all my contacts back there as I still care very much about it.
Keep digging, and good luck, I hope this gets picked up in the mainstream media.
Its a masterpiece. You are right brave Jedi. I support you totally. When I said all this, everyone ignored me.
Thank you! I haven’t ignored you… 🙂
Another very well argued piece. Did Mid Staffs suffer a double whammy? If coding was done poorly and this also led to underpayment for work done, this would explain a number of other things.
Firstly why there was a perceived need to save more money before becoming an FT even though the Trust had generally come in on budget (and often had money taken away before the year end to prop up other Trusts or the SHA – hence the push to be an FT, ironically egged on by the SHA who hadn’t got many applicants). IE if the Trust had only realised these things early on, they would have had more money in the kitty (more nurses, better care etc) and a lower HSMR! Secondly why the PCT(s) did not jump up and down about quality – after all, they were getting a lot of stuff very cheaply (even if they didn’t understand why).
Apparently there are a whole lot of other artefacts of the PBR system such as a disincentive to do two procedures for a patient on the same day (convenient and less anxiety inducing for them) as you might only get paid for one (you can probably negotiate with your PCT but you have top realise this first). As late as 2011 the Trust brought someone in from (?)London to look at this – the additional income was £5m annually! If PBR came in around 2004, that’s £30m less investment. If coding itself was losing the Trust a similar amount of money between 2004 and 2008 that’s another £20m (making a cool £50m the DoH hasn’t paid MSFT for the work done). Interesting that, given the figures related to the current “financial problems” (which would be obliterated if the money owed was paid now, even without interest) – that are so severe as to warrant the closure of all acute service in the area! [This is not a “holocaust denial” – some patients experienced poor care and of course nobody, least of all the staff on the front line, wanted that – supreme efforts have been made to address this – to the extent that many other Trusts are learning from Mid Staffs – not that they would feel able to say so publicly].
Reblogged this on Oh, Matron! and commented:
READ THIS. Really.
The other one that worries me about Brian Jarman, is the constant banging on about I wrote to the Minister in 2010 and he did not act quickly enough. He wrote to Burnham in march of 2010, at the point when everyone knew the election was in May, Parliament was being wound up in April and so Burnham and all other Labour Ministers were out of a job in 2-3 weeks. There had been lots more information earlier but Jarman keeps bringing up the final march 2010 letter. Realistically what was he expecting to happen in those couple of weeks.
Great point!
I’ve been wondering about these figures derived from statistics. If a high HSMR means excess deaths (I don’t agree but stay with me) then the opposite must also be true i.e. – a low HSMR must mean patients are surviving unexpectedly. In a Trust with a really very low HSMR, say for example Stafford currently, a large number of these miracles must be happening. If you think that’s implausible then the opposite (ie large numbers dying unexpectedly) is equally implausible in most cases – as borne out by coroner’s inquests, independent notes reviews etc that look at the actual cases. Of course the staff are seeking to learn from all cases, however well (or otherwise) they do but that’s routine audit in routine practice. Professor Jarman can’t have it both ways surely?
Agreed! But he tries to by accusing Stafford of gaming. That’s shown to be nonsense by NAO/Capita PBR audits rating Staffs at 97-98% under Ms Kirkbright.
I suppose Prof Jarman is a believer in his method etc so rather like the chap who believed that MMR vaccine caused autism, it’s difficult to think outside the box. What’s inexcusable is to aggressively put forward such a view (again, similarities to the MMR/autism story) when the inevitable consequence is a good deal of worry in various parts of the country about the safety of the local hospital (which may well be groundless) and, when the real agenda is revealed, an additional worry over whether this will lead to the closure of the local hospital leaving no services at all. Don’t the public deserve a bit of stability and reassurance? Ironically if you extend my previous argument as above, because it is based on averages, then the excess deaths in one area (if that’s true) would be balanced by the miraculous survivals in another ie the NET effect in the NHS as a whole must be that there are no excess deaths. Are you beginning to wonder whether this whole thing is a bit of a nonsense as suggested by the 2010 articles in the BMJ I have referenced in previous comments on another of your blogs.
Indeed. For HSMRs to be meaningful and useful, need at least 3 things:
1) Compulsory & standardised training across the whole country.
2) Strict and thorough auditing on a regular basis.
3) To use the system as they do in other countries – benchmark a national ‘100’ once, and then award reducing/increasing scores relative to that point, which remains fixed every year. Then the scores can mean something.
Alas, Prof Jarman considers the English too ‘simple-minded’ to handle that.
So true – vindicates my view from long ago: clinical coding matters. It must be better taught for ALL healthcare professionals who are responsible for adding data to clinical record. Professor Jarman clearly is not a neutral observer. $35k per contract for a system which cannot be accurate while clinical coding so dire is a terrible waste of public money. BTW, a GP working 8 sessions/week will also have more than 7000 consultations to code properly every year. Much and more of the hospital data on co-morbidities (and death) is in the hands of primary care. Why aren’t GPs included in the data collection and coding issues? Bet we would not cost so much for each of our hospitals!
Exactly – massive weakness. There IS a coding manual that can form the basis of training, but rarely used as such. The irony is that HSMRs could be a really useful tool if the data could be made consistent across the board – I know of at least one surgeon who no longer operates because a coder picked up that lots of his patients were dying because of failed sutures.
Clinical coding in General Practice used be solely means to keep notes tidy and allow doctors to keep track of illness episodes and long term conditions. Well and good – motivation to code properly for me used to be simply to leave a lasting record of my engagement with my patients and demonstrate decent quality healthcare.
Now this is changed with accountants’ eye on details and productivity. The motivation to code correctly is to get paid properly for work done. This is no longer just the case for private providers securing money from insurers, but also universally the case in hospital practice and primary heatlh care.
Is it any wonder that coding “properly” means ensuring adequate payments? Is it any wonder “coding” is not accurate when providers are judged to fail or pass according to a Snomed/ICD-10/READ code?
Yes. The pitfalls and perverse incentives of PBR and HSMR have got to be plain as day to any halfway objective observer.
Sadly it’s not a view that is currently replicated within the top levels of management for whom the mantra appears to be ‘cheaper not better’. In the last few years so many people, trained in the importance of accurate coding/inputting, have disappeared and have been replaced (if they’ve been replaced at all) with an endless procession of agency temps.
Yes, this all fits exactly with what I’ve uncovered while investigating HSMRs.
Reblogged this on Broken_Heart Blog and commented:
Brilliant essay illustrating importance of clinical coding in managed healthcare system
Dear Skwawkbox, How do I send you a private email please? (If that’s possible.) I’ve had a comment from statistician friend and I’d be interested in your response to it. I can’t (yet) post it here because I don’t have permission (it was a personal email). He might agree to my passing on his comments to you, and something useful might come out of it. Thanks. Brian.
This all seemed convincing until you decided to blame the Tories where it all turns to baseless speculation. The ‘scandal’ broke under Labour and they seemed quite keen to heap maximum opprobrium on the trust. There certainly seems to be political games being played but saying Labour = good, Tories = evil, is a gross simplification which subtracts credibility from the rest of the article.
You didn’t read carefully enough. Firstly, my original article on Stafford’s HSMRs acknowledges that the problem began under Labour.
Secondly, my issue with the Tories has nothing to do with them being responsible for Mid Staffs. It has everything to do with the fact that they’re trying to paint Mid Staffs as an issue with a ‘target culture’ when even Francis says it was caused by short-staffing – and the Tories are multiplying the problem of short-staffing enormously.
They’re using different means to do it, but the end effects will be the same – danger to patients – and they’re trying to obscure that fact.
Tim Fenton has also been covering this and now I have too FYI http://paulocanning.blogspot.co.uk/2013/03/you-could-make-it-up.html – hopefully reposted at Harry’s Place later. Brilliant job Steve!
Can’t see an email address so communicate this way (don’t post comment!).
My post is being reposted tomorrow on Harry’s Place.
I can completely relate to what you commented to me, I have had similar experiences.
Have you tried the left websites Left Foot Forward or Liberal Conspiracy or Political Scrapbook? Or even Labour List? They all have an interest in defending the NHS, obviously. Plus your story, although lengthy, and needing it to explain properly, is so well reported and damning.
I seem to recall that one B Jarman devised an Index of Deprivation that was used for a while but it proved less accurate than either the Townsend Index or the Carstairs Index. Just a thought which may not be relevant!
Further to my comment above, I’ve heard from my medical colleague, who doesn’t want to get involved in this blog (he’s a very busy man, whereas I’m retired) but doesn’t mind if I quote what he wrote. His email was sent not only to me, to others to whom I’d also sent the link to this blog item.
A disclaimer first: I am no statistician and have no informed view about Steve’s blog, so wont be getting into any disputes. I’m merely the messenger, anxious to help get at the truth.
QUOTE
Wouldn’t it be wonderful if the whole Mid Staffs nightmare could be magicked away as a right-wing conspiracy fuelled by an elementary mathematical error made by a naïve academic in an entrepreneurial University! That hits all the buttons – it’s simple (no hard work understanding things), it contains a conspiracy, there’s a bit of workerist anti-intellectualism, and civilised values get corroded by commerce (a comforting moral tale). Overall it is a fab narrative.
Sadly, it’s not that simple (booo!), but there is a useful insight into the complicated story of HSMRs and their critics at http://www.straightstatistics.org/article/coding-maze-mortality-ratios-and-real-life , with some explanation of deep coding effects and probable gaming using palliative care codes. Jarman makes the point that HSMRs correlate closely with on-site inspection reports by regulators, but not with self-report (which has been the main form of data gathering for light-touch regulation).
Of course the Tory press will not miss an opportunity to attack the NHS, but that does not make Jarman, Dr Foster and HSMRs either wrong or guilty by association. When the press are looking elsewhere we will still need to know how to assess performance in large, complex, public organisations. END QUOTE
Hi Brian. Do I assume correctly that your contact hasn’t actually read the articles but you’ve talked to him about it? I ask because I link to the Straight Statistics article myself in my article – and at no point have I ever said there wasn’t poor care.
Your colleague/friend also mentions ‘gaming’ – but Stafford was externally audited for coding accuracy once the recoding exercise was done, and scored an unprecedented 97/98% in successive years. So gaming at Stafford is not on the table.
If there is gaming at other hospitals (and it appears Bolton is such a case if the allegations bear up to scrutiny, though for cash reasons rather than to change the mortality indicator), then that only strengthens the case I’ve presented. Artificially low HSMRs at ‘gamer’ hospitals would make honest hospitals look worse.
HSMRs do not, in fact, appear to correlate that closely with on-site inspections. Stafford had good inspection reports at various points, even though there was poor care in *some* areas of the hospital. Similarly another hospital at the time that I know of had good inspections and a poor HSMR while another had great HSMR but was hammered in its inspection report.
I do believe that DFI has an inevitable conflict of interest because they publish a ‘Good Hospital Guide’ that inevitably pressures hospitals lower in the table to work their way up – and then DFI sells them a system to help them do so. This is not to say that DFI are deliberately manufacturing/exploiting the opportunity (nor that they’re not!), but the whole point of a conflict of interest is that they could, not that they are. And usual rules of procurement etc bar any conflicted parties from participation.
As for Prof Jarman, I don’t think he’s guilty of any more than overzealousness in defending his ‘baby’ – but in doing that, he plays right into the hands of those who want to ‘bash’ the NHS and lower its status in public affection for their own purposes.
We do need a way to assess performance, of course – but HSMRs are not it, not unless we spend a lot of money at the input end to ensure consistency and accuracy across the whole country. The newer SHMIs are better, but still imperfect.
‘Back to the drawing board’ is not a disaster. The disaster is that Prof Jarman/DFI’s figures have been (mis)treated by the media and politicians to undermine the NHS as a whole – and Prof J’s insistence on the correctness of his system instead of publicly acknowledging the holes in it has provided a very big, hard mallet to hammer the NHS with.
My political position is very clear. But this is not a black & white tale. The poor care that did happen in parts of Mid Staffs happened under Labour’s watch, as I’ve been careful to acknowledge. Labour under Blair was guilty of embracing the Tory ‘Foundation Trust’ idea that should have been binned the moment they took office, and the pressure to become/remain an FT created the underfunding/short-staffing situation at Stafford that led to the poor care.
But the underfunding and shortstaffing there was nothing compared to what’s being imposed on all parts of the NHS by the current government. The methods and motivations of doing so are different but the result is the same – and must be creating poor care in lots of places.
That’s why the Tories are so keen to make Mid Staffs about a ‘target culture’ instead of about underfunding and shortstaffing – if they admitted the truth, it would shine a massive spotlight on what they’re doing.
Please pass this to your friend and ask him to read the articles (again if he has in fact done so once). I hope it helps.
I don’t know, Steve! We’ve never, to my knowledge, met personally — I’m a member of the Medical Practitioners’ Union (MPU, medical branch of Unite) and so is he, and we’ve only exchanged emails following my link to your article. As I said, he doesn’t want to get involved in this — tho very active in other ways. Cheers. Brian
Of course if the IT systems code ‘on the fly’ then the issue of not having enough coders becomes less of a problem. Clinician picks a diagnosis from a coded set (but doesn’t see the code). They pick the diagnosis, the system records the code. Then trained coders are able to do assurance rather than being bogged down trying to translate poorly recorded issues on paper.
Yes. But not all hospitals are that well equipped on the IT side yet, and few would consider they have the funds to do it when funding is so squeezed and there are so many other priorities.
I agree it would help, though – but you’d still have the problem of inexperienced doctors allocating diagnoses that later turned out to be wrong or incomplete.
Steve, FYI http://hurryupharry.org/2013/03/19/you-could-make-it-up/
I saw – thank you very much!
Yep. I’ve worked extensively with HSMRs and I agree with that whole article. They’re more a reflection of coding practices than anything to do with clinical quality. Why else do you think that UCL (where Jarman’s unit is based) always have such a ridiculously outlying low score?
I’ve heard some people suggest that Jarman is something of a self-publicist. I couldn’t possibly comment…
Reblogged this on The SKWAWKBOX Blog and commented:
Just noticed that the Daily Fail (http://www.dailymail.co.uk/news/article-2364765/20-000-extra-NHS-deaths-Labours-watch-amid-calls-site-inspectors-struggling-hospital.html) re-ran Brian Jarman’s old ’20k needless deaths’ claim early on Tuesday morning, ahead of the publication of the Keogh report. There’s no question from the content of their article that the intent was to turn up the heat on Andy Burnham and to try even harder to dislodge Labour as the party of the NHS.
Well, that’s egg, sausage and chips all over their faces now, since Bruce Keogh completely disavowed the 13k deaths claim in a private email (http://skwalker1964.wordpress.com/2013/07/15/national-media-keogh-says-13k-nhs-deaths-keogh-er-no-i-dont/) as well as most emphatically in his report.
The lurid death-headlines, from Mid Staffs to Basildon and Thurrock and every point in between, are now thoroughly discredited – but I’d lay money it’s not the last we’ll see of them. The aims of the government and the right-wing media, and Prof Jarman’s increasingly desperate attempts to restore his dysfunctional statistical system to what he considers its rightful place, fit too well together for them to let little matters like obvious truth stop them trying.
Don’t hold your breath waiting for the Fail to apologise or publicly retract its claims. Do watch out for more rehashed ‘news’ (and no doubt lots of pictures of the equally-dysfunctional Julie Bailey and her Cure mafia), as the media attempts not to let the facts get in the way of a ‘good’ (from their point of view) story.
Since the 20k death myth appears to be persisting even though its as thoroughly nonsensical as the 13k deaths claim, here’s a reblog of why it’s pure, propagandist, nonsense.
Reblogged this on HUMAN RIGHTS & POLITICAL JOURNAL.
Waiting with baited breath now for secret lion tamer to start to roar at Telegraph like hu-mane Prof he really must be:
https://twitter.com/WikiPickie/statuses/358657200958353410
Me too!
Reblogged this on thepositivevoice.