In the most recent issue of Private Eye magaine, the Medicine Balls columnist shows a graph of figures concerning the use of the Z51.5 palliative care code for deaths at a number of Midlands hospitals. He or she claims of this graph:
Here is a pretty picture. It shows how Trusts can disguise high death rates by recoding those who die as palliative care (code Z51.5). They are then “expected to die” and disappear from the published death rates.
Here is the graph the Eye showed:
This chart, which was taken from written evidence submitted by Prof Brian Jarman of Doctor Foster, shows 3 West Midlands Trusts, plus Medway Trust in Kent, and an average for all the other acute Trusts in England minus the 4 aforementioned Trusts.
Looks damning, doesn’t it? But does it show what we’re meant to think it shows?
Specialist subject: the bleedin’ obvious
Professor Jarman – and the Eye article – assert that the graph demonstrates how Trusts can ‘disguise high death rates’ by using the palliative care code. Prof Jarman told the inquiry that he believed the Trusts were ‘gaming’ his system to make themselves look better.
But look at the green line in the graph, the one which represents Walsall Trust.
Walsall’s line jumps from almost zero up to the end of 2006 to around 9% at the beginning of 2008 – to almost 80% by the 4th quarter of 2008. Can anyone seriously believe that Walsall Trust was trying to fool people that 4 out of every 5 people who died at its hospital were being treated for terminal illnesses?
Of course not. So that’s the ‘deliberate gaming’ theory shot down in flames just from a first glance.
What was happening at Walsall during the period where the line shoots up sharply was a misunderstanding among its coders about what constitutes palliative care.
Palliative care constitutes a defined set of treatments that are given in a defined set of circumstances, aimed at making someone comfortable during the very final stages of his or her life. The Liverpool Care Pathway, about which I’ve written previously, is one example. Administering drugs to sedate someone who is panicking because they can’t get their breath at the end of their life is palliative care – but administering similar treatment to someone who is not imminently dying is not.
The confusion at Walsall was between palliative care and what they were calling ‘TLC’ – similar care given to someone who was not imminently dying, but neither were they going to get better. Making someone comfortable, easing breathing and so on – with a view to sending them home (or to a care home etc) once they were settled, is ‘TLC’.
But Walsall’s coders were coding it as palliative care – hence the huge spike in the green line, which then drops sharply once the error was identified and corrected.
Now let’s strip out some of the extraneous information. We know Walsall’s line was very wrong, and why, so let’s remove that from the graph. Since Walsall’s figures must have a drastic effect on the average line for the 3 West Mids Trusts, we’ll remove that too. I’ve edited those out as cleanly as I can with my limited graphics skills, and here’s the resulting graph:
That’s clearer, isn’t it? Without so many lines you can see more clearly. As I wrote in my first post on Mid Staffs’ HSMR mortality rates, when the palliative care code was first introduced to make the coding across the whole country reflect more accurately what was actually happening, the Trust’s coding department did not make the switch to using it.
This is borne out by the graph, which shows Stafford’s line at or very near zero until the end of 2007, while the line representing national use of the code is climbing steadily from 2004. The graph also shows that the Medway Trust in Kent failed to realise the change at first, but it caught on earlier, and its line rises from the beginning of 2007 rather than the end.
So far so good. But the lines for Stafford (and George Eliot and Medway) still look significantly higher than the national average. So surely that’s a problem, isn’t it?
Well, no – but let’s see why.
A jury of its peers?
Prof Jarman’s Doctor Foster’s, along with the Healthcare Commission (HCC) had grouped Stafford into a ‘peer group’ of other Trusts. When asked by Jonathan Pugh, Information Manager at Mid Staffs, what the basis for this grouping was, the HCC didn’t know and Doctor Foster’s didn’t say. But as Mr Pugh told the Francis inquiry in his written statement:
Mid Staffs always argued that this group was not representative of the Trust and we put forward our own peer group.
So, Mid Staffs felt that the peer group was chosen to make them look bad, but even so, as Mr Pugh testified to the inquiry:
There is always someone at the top and a graduated reduction as you go down the graph. Without Medway and Walsall you could have easily included the next 8 in the graph.
In other words, Mid Staffs was put into a group for the purposes of the graph that reflected badly on it – but even in that group, if the others in the group had been included on the graph, we would have seen a spread of lines across the graph.
Seeing an even spread of lines would show that there was considerable variation among Trusts in their use of the Z51.5 palliative code – just as there was considerable variation in the handling of all data relevant to HSMRs. So much variation, in fact, that it made the HSMR results meaningless.
It almost looks as if the candidates chosen for the graph were deliberately selected to make Mid Staffs and other West Midlands Trusts look bad – look like they were faking the figures. Could this be true?
Back to Mr Pugh. In his statement to the inquiry, he points out that
Airedale NHS Foundation Trust, in quarter three of 2006, had a peak of 3.5 times the national average for coding of deaths with palliative care.
Wow. Mid Staffs’ palliative coding – at its worst – was about 3.8 times the average at the time (and less than twice the average in 2010). Surely Airedale was identified as a ‘gaming’ Trust and targeted for auditing and possible sanctions?
No. As Mr Pugh points out, Airedale had been
‘Small Trust of the Year’ in Dr Foster’s annual hospital guide on numerous occasions.
So, Mid Staffs is accused of fiddling the system while Airedale, with very similar use of the Z51.5 code, wins awards. Draw your own conclusions.
The big picture
We’ve already noted that Mid Staffs was allocated a ‘peer group’ by Doctor Foster’s – a group which the Trust disagreed with vehemently on the grounds that it made the Trust look bad. But even so, Mr Pugh’s evidence to the Francis inquiry included a table that showed a comparison of palliative care episodes for each of the peer-group Trusts, including Stafford.
That table bears close examination. Here it is:
This table tells a very clear story – but because it’s a lot of figures, it’s quite easy to miss it. So, at great expense (well, an hour’s worth of me typing the figures into a spreadsheet and creating a chart), here’s how those figures look when put into a graph:
What this chart shows is Mid Staffs (the red line) starting at almost zero in 07/08 and then climbing sharply as use of Z51.5 kicks in. But it is then rapidly overtaken by most of its peers and finishes 3rd-lowest out of the 7 peer Trusts (although there’s hardly anything separating most of them. So Stafford’s use of the palliative code is entirely in line with other hospitals considered its peers.
And remember, Stafford’s codes were audited by the Audit Commission – and found to be 97-98% correct, an unprecedented score. So there’s no chance it’s ‘gamed’.
This peer group was chosen by Dr Foster’s – yet the only one they chose to show on the chart presented to the Stafford inquiry was Mid Staffs, making Mid Staffs and its neighbouring Trusts look like ‘outliers‘ – like the odd ones out, the ones that ‘must’ be up to no good, because they’re so far above the national line.
And don’t forget, the chart you’re seeing above still gives a false impression of a lot of coding – until you notice that the top of the ‘Y’ axis showing the percentages denotes only 1.2% of all the codes entered by those hospitals in those years.
If I make the top of the ‘Y’ axis 100%, here’s what it looks like:
All the lines are so low – fractions of a percent – that you can’t even separate them.
So much for Mid Staffs ‘gaming’ the system.
The even bigger picture
You might now be thinking, ‘That’s all well and good – but Mid Staffs Z51.5 codes are still quite a bit higher than the national average. That’s still suspicious.’
I understand why you might think that – but you’d be wrong. What you can’t see from any of the tables or charts presented is what’s going on outside the figures that still affects them – and there are a factors that easily account for the difference.
What you don’t know might hurt you..
Prof Jarman presented his graph contrasting the West Midlands Trusts (plus Medway) against a line for the national average. We’ve already seen that a number of other Trusts – the ‘peer group’ that was selected, but then ignored, by Dr Foster’s – had very similar patterns of coding for palliative care. If those had been shown on the graph, Mid Staffs and its neighbouring Trusts wouldn’t have looked anything like so unusual.
But they weren’t. And if they weren’t, then it has to be asked: how many other similar Trusts were omitted which distorted the picture even further?
And at the bottom end of the range of figures lie a similarly-unknown number of Trusts with very low use of Z51.5 – and they must exist to bring down the ‘national average’ line low.
Perhaps a lot of Trusts are still not using the code as much as they should, just like Stafford and others did at the outset but later corrected. Any Trusts not using the code much – rightly or wrongly – will drag down the average and make other Trusts look worse on a graph like that presented by Prof Jarman.
And a lot will be using the code a lot less, for perfectly legitimate reasons:
The hospice factor
As I showed in my original HSMR article, HSMRs only measure people who die in hospital. Fair enough, you might think – but there is one big factor that can affect the HSMR scores among hospitals in a completely unseen way: local hospice care provision.
Some hospitals are situated in areas with a number of local hospices. My own area is one such, with a number of hospices situated within the catchment area of my local hospital. This means that people in the final stages of their life and who need palliative care are far less likely to receive that care in hospital – and if they don’t get it in hospital, it never shows up in HSMRs.
By contrast, some Trusts are in areas with little or no hospice provision. Their terminally ill patients are going to die either at home or, far more usually, in the hospital – and if the coding department is doing its job properly, the Z51.5 code will be used a lot.
This means we should expect a huge variation between hospitals with great hospice availability and those with poor or even none.
And Stafford? As one local resident put it, Stafford has
a piddling little hospice.
So, with very modest hospice provision you’d expect Stafford to be high on the chart for use of Z51.5 – but because some hospitals will have even less access to hospice care, not right at the very top.
Which, strangely enough, is exactly what we see in the charts – both mine and Prof Jarman’s.
Whether deliberately or not, the chart showed by Professor Jarman and his Dr Foster’s group present an extremely misleading picture – one that happens to make their HSMR system look useful while making Mid Staffs look like it must be ‘gaming’.
Unfortunately, Private Eye appears to have bought into the myth in the most emphatically wholesale fashion – and its article therefore merely adds to distortion and misunderstanding that is being foisted on the British public, to the detriment of not just Stafford, but of the NHS as a whole.
It’s taken me a few hours to look more closely at the facts and see that the tale presented by the Eye isn’t worth the paper it’s printed on. What a pity they didn’t do the same before they printed it, and avoid stoking the fires of suspicion an anti-NHS sentiment that the Tory-led government are intent on spreading for its own purposes.
Still, I had to pay £1.50 to buy the magazine and get a clear scan of its chart – so maybe for them swallowing a line is good business.