« Gone, gone, gone | Main | Sentences to ponder »
Health Data Now!
It's well known that medical spending is highly variable but so are medical outcomes. Here is Begley and Interlandi in Newsweek:
After we interviewed dozens of oncologists, pored over published papers, and obtained outcomes data that cancer centers have never before made public, it became clear that for these cancers there are indeed significant outcome differences depending where you are treated.
Five years after surgery for prostate cancer, for instance, 72 percent of men treated at leading hospitals are alive, compared with 62 percent of those treated elsewhere. Scrutinizing data from specific cancer centers reveals even greater gaps. Five-year survival for stage IV prostate cancer is 71 percent at Fox Chase, for instance, but 38 percent nationally. For stage IV breast cancer, the respective figures are 28 percent and 19 percent—an almost 50 percent edge. For stage IV cervical cancer, five-year survival is 33 percent at the Cleveland Clinic vs. 16 percent nationally.
Some of this is probably due to differences in patient characteristics but it could go either way - the better hospitals often get the hardest to treat cases.
Many hospitals hide this data (or "fail" to collect it which amounts to much the same thing) but there are some good rules of thumb such as looking for hospitals that specialize in certain procedures and thus perform many of them (there are large economies of scale in quality). Patients can also find information about which hospitals closely follow best practices (kudos to Medicare for this data and see here for a mashup with Google maps) although the measures used are probably the ones that are easiest to collect and not the ones that correlate best with mortality.
Nevertheless, providing information does seem to drive change if only from the shame that a hospital receives when it is found not to be following best practices. It's true that report cards can cause problems when the drive to get a better score causes hospitals to be more reluctant to treat sicker patients but better data on patient characteristics (stage of cancer etc.) and better process/treatment information can alleviate this problem. In fact, all hospitals should be required to provide standardized information for all patients on patient characteristics, treatments and outcomes. Only by making outcome information public will hospitals have the incentive and researchers have the ability to develop more accurate report cards. In short, I cannot think of a simpler change that would improve health care to as great an extent as freeing the data.
Posted by Alex Tabarrok on October 23, 2009 at 07:11 AM in Data Source, Economics, Medicine | Permalink
Comments
Another huge benefit is that the data can be used to identify best practices that can then be disseminated. Unfortunately, it's rarely simply a case of "freeing the data". It's more often a case of forcibly capturing it and making it speak.
This means requiring the data to be collected, standardized and reported. As you point out, hospitals don't necessarily want to share this information so the free market won't generate it without a great deal of help and prodding. The VA and Medicare are out in front on this, and improvements in IT will make it much easier and more effective to do this universally.
Posted by: Erik Brynjolfsson at Oct 23, 2009 7:50:38 AM
But Alex misses a big part of the Obama plan, they save costs by reducing access to specialists to save money. But specialists give you the best outcomes.
Also they want to base physician reimbursement on utilization of resources and if you are above normal you see a reduction in reimbursements. You have a huge incentive to churn procedures, refuse complex referrals, and avoid teaching hospitals for employment.
Posted by: DanC at Oct 23, 2009 9:01:11 AM
If you follow the argument that physicians do not follow best practices what you are really claiming is that medical schools are failing in their mission.
Do we need a Federal takeover of medical schools?
The best practices argument is overstated. While the average physician has a hard time keeping up with the knowledge in every field, that is not because physicians refuse to share information. It is just hard to do. Of course the Obama plan is to find super human physicians who are generalist who know everything, or at least will do what a centralized bureaucracy dictates.
Medicine is a science with a lot of variation. Look at medicaid reimbursements for drugs. The state has decided that research shows that one drug is very effective in a treatment of a disease. You have a patient that doesn't respond. Sometimes your DNA will make one drug more effective then another. Your physician wants to try a different drug because sometimes a small subset of the population responds. Then prepare to argue with a state employee on the other end of the phone. Because they will tell you that you may not use the other drug because general trials have not shown it to be cost effective on the general population or testing has not been done on the subset of the population you treat. i.e. it is not cost effective to treat people because outcome studies show that most people don't respond. Bad luck that you are not like most people. And the state things your physician is unable to make an objective decision. I guess most doctors are just poorly trained over paid plumbers
Posted by: DanC at Oct 23, 2009 9:21:38 AM
how do you know this isn't selective-bias and the leading hospitals aren't just selecting candidates more likely to survive or spot the cancer earlier? - http://intqhc.oxfordjournals.org/cgi/content/abstract/mzp039v1
Posted by: anon at Oct 23, 2009 9:40:23 AM
BTW
On the link that showed Chicago hospitals, I think you will find a strong correlation with the outcomes and the quality of the nursing staff. Hospitals that have a hard time recruiting and retaining nursing staff are the same hospitals that have the lowest scores.
Some of those low scoring hospitals serve poor communities. Communities that most nurses are reluctant to drive through never mind work in. (And pay isn't always that different. I think Cook County and University of Chicago have the highest paid nursing staff.)
In addition some of the other hospitals while not in poor communities have, I am told, difficulty with their nursing staff
Based on that link it would seem that nursing staff has a much bigger impact on outcomes at hospitals then physicians. (Although top physicians also tend to avoid these hospitals.)
Posted by: DanC at Oct 23, 2009 9:50:10 AM
I haven't looked through any of the links, but I would approach this kind of data with considerable caution. There's presumably lots of room for variation in initial patient condition and patient ability to faithfully follow treatment regimes.
Data can be a good thing, but having and using bad data can potentially be worse than no data at all.
It's not only the things you don't know, but the things you know that just ain't so.
Posted by: Phil Steinmeyer at Oct 23, 2009 10:02:22 AM
Another BTW
Their is a smaller community hospital about 60 miles from the Cleveland Clinic. They brag about the outstanding outcomes of their cardiac unit. What they fail to mention is that complex cases are routinely referred the the Cleveland Clinic that has one of the better cardiac units in the world.
The data from the community hospital probably looks like more effective low cost care, but they are free riding on the talent at another hospital.
Posted by: DanC at Oct 23, 2009 10:05:27 AM
As a physician, I generally agree with greater transparency in hospital and practioner outcomes. But I do see one potential downside, depending on how outcomes are measured.
You need to take into account the population you are treating, not only for educational/poverty trends, but more importantly, for risk. If you're a physician that takes high risk patients when no one else does, and you are penalized by such outcome measures, odds are you will not take the chance to treat someone who might otherwise not survive. For that 10-20% of patients who do 'recover' if treated, that's a death sentence.
For example, I did my internal medicine residency at an Ivy League school, which is all the rage currently for reforming medicine. We had a surgical section that was very big on minimizing adverse outcomes. Often, we had to beg, berate, shame, etc., the surgical staff to take on risky cases. First off, the patients would 'not be sick enough' for surgery for the surgeons. But then at some point they'd become 'too sick' to take to the operating room. This helped their outcome measures, as the patients would end up dying on our service. Unconsciously, the surgeons sold out the minority of patients who could benefit while the majority obviously, would not. I believe that's a bad trend for medicine.
Posted by: DrYes at Oct 23, 2009 10:55:46 AM
If you stand still, others will change. It is interesting to watch those who opposed parts of the healthcare bill which fund studies of what is effective now realize that this can lead to better outcomes.
Indeed, beginning to pay for outcomes, rather than procedures, is a way to have the market work to introduce best practices and control costs. They need not be mutually exclusive.
There have been a number of studies which show that what determines a doctor's treatment of an individual varies based on such factors as when the doctor was trained, with whom he/she was trained, whether the doctor is part of a multi-practice specialty group, and whether the doctor is employed or is in private practice (the latter factor influencing the ability of the organization to have influence over the practioner).
Development of science based medicine, studies of effectiveness of alternatives practices, procedures and products, are all public goods, or at least goods that if discovered, need to be costlessly diffused to other practioners, and therefore deserve public expenditures to subsidize additional production or capacity over what would be privately produced.
Posted by: Bill at Oct 23, 2009 10:56:27 AM
I guess the amazing thing is how many laws it takes to create a competitive market (and how profitable it is to prevent one).
Posted by: Lord at Oct 23, 2009 12:53:37 PM
This is a huge peeve of mine.
I'm a geek, I admit it. When I make a major purchase I research the living daylights out of it.
When I was searching for a hospital for my wife and I to deliver our baby at this year, the lack of quality data astounded me.
I understand the arguments against, such as selection bias, but it seems to me that releasing this data in a normalized manner should be a number 1 priority in public health
Posted by: Dustin at Oct 23, 2009 1:42:36 PM
Yes! And the same for education. Sophisticated, but easy-to-understand, report cards are essential. See this paper for a school choice report card example: http://aida.econ.yale.edu/~jh529/papers/Hastings&Weinstein_InfoChoiceOutcomes.pdf
Posted by: Michael Bishop at Oct 23, 2009 1:52:54 PM
This is pretty well documented in the medical literature; especially for complicated procedures (AF node ablation, heart transplants, etc.) it is very clear that the more procedures the doctor/hospital has performed, the better the outcomes.
Posted by: Bob Montgomery at Oct 23, 2009 1:56:46 PM
"it seems to me that releasing this data in a normalized manner should be a number 1 priority in public health"
The problem is that for a hospital, or an individual physician, there is a downside to releasing this data because with bad numbers, patients will stop coming. But there is no upside: with good numbers, they can still only charge the Medicare rate, but will have longer wait lists.
One of the pinnacles of human achievement is creating markets, which goes against innate tendencies, so it doesn't surprise me how difficult it is to create a market in health care.
Finding the metrics in designing these markets is the most difficult part. Especially given that most market metrics, especially price, are usually customer based. By this I mean that these metrics are optimal for a customer, based on their personal preference - local -, rather than globally optimal according to for example a population yardstick, like 'national happiness'.
The metric for food production, i.e. 'more food is more expensive', over the past centuries has been very succesful, and has led to huge improvements in food production. The result of this pricing mechanism is that there is now (in the developed world) arguably 'too much food production'. The result is that (channeling Prof. Cowen) customers now signal with overpaying for certain foods (organic), and more importantly, become obese, which is what they personally prefer, but which is certainly not optimal from any 'national overwheight index'.
Though 'more food is more expensive' worked wonders in eliminating famines, it is clearly less appropriate now. Possibly, or not, new metrics need to be found for food production.
Analogously, in improving health care delivery systems the problem is that the optimal metrics or pricing mechanisms have not yet been found. At the patient level, especially in chronic disease, patients often prefer a 'nice doctor', who does not have a wait list, and they care much less about outcomes, trusting that the profession will take care of that. If there was a real market, patients would probably pay more for a nice doctor than a not-so-nice doctor with a 5 hour wait in the clinic, notwithstanding differences in outcome. Good doctors are not necessarily nice.
But this metric of personal patient satisfaction is very different from what proponents of the current proposals for health care reform want to measure health care delivery by. These are global metrics like 'average longevity per $', or 'average cancer survival per $'.
And also different from many of the current metrics, often amounting to "the more you follow the rules in this Medicare coding book, the more you get paid".
Allow me one last comment:
"When I make a major purchase I research the living daylights out of it. When I was searching for a hospital [...] the lack of quality data astounded me."
This is not unqiue to healthcare. When I was looking for the best lawyer in immigration matters a few years ago, there was no information at all about the percentage of succesful cases, or any other metrics. I basically had to go by recommendations.
Posted by: adam at Oct 23, 2009 2:36:02 PM
Alex, in all services, spending is highly variable and so are outcomes. For another example, read
http://www.thefrisky.com/post/246-money-honeys-the-economics-of-prostitution/
Posted by: E. Barandiaran at Oct 23, 2009 3:40:05 PM
To Alex's point: more data now!
Is data a public good, or is data a means to private profit.
Drug companies see data as the means to private profit. They collect lots of data, specifically on the history of people taking their drugs and of those taking baselines, like competing drugs and placebos in double blind studies (work is being done on finding ways to do the same for surgical procedures), as well as in large population studies. They then selectively present the data to the FDA for drug approval, and then selectively release three different sets of data to doctors and to the public and to their lawyers. The complete data set is something they resist releasing vigorously.
Scholars working in medicine, especially, but other disciplines, run into an ethical dilemma - do they promote knowledge, or do they serve those who restrict and distort knowledge for profit. This isn't a new problem, nor one limited to profit - the translation and printing of the Bible was immediately seen as a world changing "data now" movement that needed to be quashed.
I see two worlds in terms of data as public goods. In one world, the debate is active and activist, and the other is either hostile to data as a public good, or passive, arguing the "market" will supply the right data.
In the half century of 1930-1980 or maybe 1920-1970 (Hoover was a technocrat among technocrats in my reading), I think there was a golden era of government seeking to collect and public data as a public good. (Milton Friedman was a young data collection technocrat, Keynes old.)
Economic data as a pubic good took a great leap forward in the 30s - think of employment data which reflects the economy in transiton of the 30s - the non-farm labor categories with job codes for the guys who shoe horses. The definition of "poverty level" is still based on the arbitrary decision of a mid-level technocrat in the 60s.
In the area of science, especially applied science, but also in history, a huge government increased effort to collect and publish data was made. Farm data collection and publishing had been done by government from colonial times, but the efforts were stepped up in natural production, especially as a labor shortage developed. And the efforts in materials science and manufacturing were greatly stepped up in the run up to WWII, and then continued in the cold war efforts of the 50s, 60, and 70s.
As I write this I'm listening to scifri talk about the Framingham Heart Study which was started in 1948, an NIH department longitudinal study spanning now three generations.
Circa Reagan, the model advanced was to get government out of funding data collection and publishing, making it a private good, in the theory that the profit motive would promote more data collection. I see this effort to be contrary to the public welfare, failing in some important areas to happen, and in other areas succeeding to the detriment to society.
On a related note, if you think screws, nuts, and bolts are a pain because there are so many kinds, you should know that they are vastly simplified as a result of government promoting industry standards to promote world wide parts interchange. Before the ANSI and ISO standards simplified things, you had to buy them by brand, with different brands by nation - a US weapon required US branded screws while a British weapon required British branded screws. Medical data is collected by branded products in branded data base formats. Medicare data is one brand, many insurers have their brand of data, different medical record systems have their brand of data.
Making medical data more available is first going to require a major government led effort to standardize its format, but don't expect that to be a single database format, or a simple one - think screws, nuts, and bolts - and then government will need to actively collect data and publish it as the BLS and Census and other agencies do.
Posted by: mulp at Oct 23, 2009 3:45:18 PM
Bill: "beginning to pay for outcomes, rather than procedures, is a way to have the market work to introduce best practices and control costs."
I don't know, Bill. I think the result suggested by DrYes 80 seconds earlier is more likely:
DrYes: "If you're a physician that takes high risk patients when no one else does, and you are penalized by such outcome measures, odds are you will not take the chance to treat someone who might otherwise not survive. For that 10-20% of patients who do 'recover' if treated, that's a death sentence."
Research in 2003 by Y. Shen of Boston University revealed that:
"Under performance-based contracting, the likelihood of a participant in the program being in the most severely ill group decreased (P 0.01), suggesting that adverse selection was occurring in response to the financial incentive."
Posted by: John Dewey at Oct 23, 2009 3:53:08 PM
Sometimes hospitals will seek to add a practice area, like open heart surgery, and end up attracting the least experienced doctors. In addition, the new hospital service, because it is new, is not performed as often, and not only does the doctor not get additional experience, but support staff do not develop it as well. Having outcomes published limits hospital product extensions, but benefits consumers with better quality care.
Posted by: Bill at Oct 23, 2009 3:55:04 PM
If spending and outcomes are both highly variable, the question that springs immediately to my mind is whether they're correlated. Should be easy enough to figure out - does anyone know if this analysis exists? I suspect they're not that highly correlated.
Posted by: Simon Kinahan at Oct 23, 2009 3:55:44 PM
According to my daughter, who is a doctor, ifyou want to do your own survey of medical services, look at where doctors go to have procedures performed on them and by whom. Ask a doctor with prostate cancer where and with whom he had his treatment or surgery.
Posted by: Bill at Oct 23, 2009 3:58:49 PM
The power of transparency has been shown time and again to improve the effectiveness of organizations by forcing them to make changes. In a health care system where insurers have near monopolies and hospitals are businesses, this transparency is all the more important. An annual national report card on hospitals would likely go a long way in pushing for efficiencies.
Posted by: Wellescent Health at Oct 23, 2009 4:21:18 PM
@John, Pay for outcomes must be done on a risk adjusted basis to avoid the problem you mentioned. You can look for the book by George Halverson at Kaiser Permanente for additional information and citations of research work done on risk adjustment for outcomes. Good point.
Posted by: Bill at Oct 23, 2009 4:29:13 PM
@Bill,
how exactly do you adjust for that risk? It seems to me that it would be very very easy to fudge for the smaller players.
Posted by: Anonymous Coward at Oct 23, 2009 4:36:07 PM
@John,
After reviewing the article you identified, it stands for the conclusion that outcomes are improved with performance based compensation, that pay for procedures results in poorer care, and that adverse selection is a problem that should be remedied by risk adjustment.
Here are some conclusions from the study:
Despite these limitations, a few very preliminary conclusions can be drawn. Incentives require very careful design. Three studies showed that documentation, rather than actual use of the preventive service, improved statistically significantly with a financial incentive (41, 48, 49). Shen (38) showed that adverse selection may have occurred with performance-based contracting in settings where providers can avoid sicker patients. These findings are important because they suggest that there is a response to incentives, although not necessarily the desired one. The challenge, then, is to design incentives with the intended goal in mind.
......
Most physicians and hospitals are paid the same regardless of the quality of the health care they provide, producing no financial incentives for quality and, in some cases, disincentives. Thus, there is increasing enthusiasm for the idea of linking payment to performance. Despite widespread implementation, we found few informative studies of explicit financial incentives for quality. This literature review suggests some positive effects of financial incentives at the physician level, the provider group level, and the health care payment system level. The findings also suggest that ongoing monitoring of incentive programs is critical to determine whether incentives are having unintended effects on quality of care. A suggested research agenda for moving the field ahead is provided in Table 2 . Rigorous research, including randomized, controlled trials and observational studies with concurrent control groups, is needed to guide implementation of explicit financial incentives for health care quality and to assess their cost-effectiveness.
Your citation proves the point.
Posted by: Bill at Oct 23, 2009 4:41:10 PM
@anonymous coward,
Re risk adjustment for outcomes. I know that Mayo Clinic and an associated organization I believe called the Insititute for Quality Improvement sponsored by Mayo and health plans have such measurements when they do comparative studies of the effectiveness of treatment in order to normalize what goes into their assessment. Will see if I can find some cites.
Posted by: Bill at Oct 23, 2009 5:20:33 PM
