Comparative Effectiveness Research - What is it Good For?

One oft floated solution to rising health care costs is the use of comparative effectiveness research (CER) to guide use of more efficient/efficacious therapies from the outset, reducing the need for costly readmission, diagnostic tests and trials of different therapies. CER involves a set of tools that help compare two or more different treatment strategies with each other, often in the context of a randomized clinical trial. An added wrinkle to all this is the the (in)famous Cost Effectiveness Study (CEX), where the outcome returns to different treatments are scaled/compared by their cost.

While proponents of CER are gung-ho about its clinical and policy utility, there are potential downsides to such research. In general, most of our clinical trials recover average effects for a population of interest. That is, we compare drug X against drug Y in randomized groups of 15-75 year olds with certain manifestations of disease Z. This is great for getting an average effect estimate for a particular population. That is, if we randomly draw a 15-75 year old with certain manifestations of disease Z, on average we can expect drug X and Y to work a certain way.

However, there is an increasing realization that drugs work differently for different people. Individuals may vary in the manner in which they metabolize certain drugs or the nature of their underlying illness, while equivalent to the average clinician, may differ in its responsiveness to treatment (see here for a great discussion on this.) If this is the case, widespread use of CER and CEX may not make people better off. In some cases, it might make some people worse off. For example, if some people are better off with drug X, but the average person benefits more from drug Y, the use of the latter will make some people worse off.

In a very interesting paper (see here for a non-gated, older version), Anirban Basu, Anupam Jena,and Tomas Philipson provide a real clinical example of this latter point from psychiatry. They build a model where CER and CEX information is used by insurers/payers to guide clinical care. That is, when a study comes out showing that drug Y > X, these parties are only willing to pay from drug Y. They then show that, in the case of schizophrenia, overall health may have been reduced because people who were formally doing well on drug X were forced to take drug Y, which was actually worse for their health and well-being. The authors go on to call for a more nuanced understanding of how CER and CEX research can be used to guide treatment, especially in an era where individualized treatments are becoming more popular (Basu has a great essay on this point here; see here for a technical paper on how CER can be individualized). Certainly, a regime where CER/CEX can be maximally useful will involve directed clinical trials that take heterogeneous treatment effects into account in the a priori design.

(PS: A great summary essay on CER/CEX, which covers many of the above points, can be found in a recent issue of the Journal of Economic Perspectives. Also, hat tip to AKN for bringing several of these papers to my attention.)