Remystifying data

I know, oxymoron.  The esoteric and mysterious is preferable to the mundane for aesthetic reasons, but, for practical action, the mundane in the domain.


I wanted to revisit the placebo, as some folks commented on it and there is more useful information than that critique (if that was at all useful).  In the clinical trial, we look at outcomes from the treatment (or active) group and the sham (placebo) group.  It is pretty common, especially when looking at things like pain, that the placebo group has positive result (which I already described).  If the active group has a positive result that enough better than the placebo group, then one can say the results are ‘significantly better.’  Significance is a technical term here, commonly meaning that the number of times we find these results by chance are less that 5 time in 100 similar studies.  Significance is a tricky notion, in that the amount of difference needed for something to be significant depends, in part, on how effective the treatment is and how many people are in the study.  Really effective treatments need less people.  Less effective treatments need more.  So saying that there is a ‘statistically significant’ difference in a treatment over a placebo doesn’t give you lots of information.  A better way to think about it is to see how many patients have to be treated before the effect of the treatment is seen in one patient.  This is called ‘numbers needed to treat’ or NNT.  

Here is an example:  If a 40% of folks in the placebo group respond to a treatment and 50% of the active respond, then we know that 4 out of 10 responders in the active group   (where 5 out of 10) are likely to be responding to the ‘placebo,’ so, after we subtract those from the active, we have 1 in 10.  That means that 10 people have to be treated for 1 to show a response to the treatment.  So, a treatment may be ‘significantly’ better than the placebo, but the number of patients who actually benefit may not be all that high.  Here is some data on pain meds.  Pretty clear that the best ones only provide the level of benefit to 2 out of 3 patients.:

Table 1: The Oxford league table of analgesic efficacy (commonly used and newer analgesic doses)

The 2007 Oxford league table of analgesic efficacy (at least 3 trials or 200 patients)
Numbers needed to treat are calculated for the proportion of patients with at least 50% pain relief over 4-6 hours compared with placebo in randomised, double-blind, single-dose studies in patients with moderate to severe pain. Drugs were oral, unless specified, and doses are milligrams. Shaded rows are intramuscular administration
Analgesic and dose (mg)	Number of patients in comparison	Percent with at least 50% pain relief	NNT	Lower confidence interval	Higher confidence interval
Etoricoxib 180/240	248	77	1.5	1.3	1.7
Etoricoxib 120	500	70	1.6	1.5	1.8
Diclofenac 100	545	69	1.8	1.6	2.1
Celecoxib 400	298	52	2.1	1.8	2.5
Paracetamol 1000 + Codeine 60	197	57	2.2	1.7	2.9
Rofecoxib 50	675	54	2.3	2.0	2.6
Aspirin 1200	279	61	2.4	1.9	3.2
Ibuprofen 400	5456	55	2.5	2.4	2.7
Oxycodone IR 10 + Paracetamol 650	315	66	2.6	2.0	3.5
Diclofenac 25	502	53	2.6	2.2	3.3
Ketorolac 10	790	50	2.6	2.3	3.1
Naproxen 400/440	197	51	2.7	2.1	4.0
Piroxicam 20	280	63	2.7	2.1	3.8
Lumiracoxib 400	370	48	2.7	2.2	3.5
Naproxen 500/550	784	52	2.7	2.3	3.3
Diclofenac 50	1296	57	2.7	2.4	3.1
Ibuprofen 200	3248	48	2.7	2.5	2.9
Pethidine 100 (intramuscular)	364	54	2.9	2.3	3.9
Tramadol 150	561	48	2.9	2.4	3.6
Morphine 10 (intramuscular)	946	50	2.9	2.6	3.6
Naproxen 200/220	202	45	3.4	2.4	5.8
Ketorolac 30 (intramuscular)	359	53	3.4	2.5	4.9
Paracetamol 500	561	61	3.5	2.2	13.3
Celecoxib 200	805	40	3.5	2.9	4.4
Ibuprofen 100	495	36	3.7	2.9	4.9
Paracetamol 1000	2759	46	3.8	3.4	4.4
Paracetamol 600/650 + Codeine 60	1123	42	4.2	3.4	5.3
Paracetamol 650 + Dextropropoxyphene (65 mg hydrochloride or 100 mg napsylate)	963	38	4.4	3.5	5.6
Aspirin 600/650	5061	38	4.4	4.0	4.9
Paracetamol 600/650	1886	38	4.6	3.9	5.5
Ibuprofen 50	316	32	4.7	3.3	8.0
Tramadol 100	882	30	4.8	3.8	6.1
Tramadol 75	563	32	5.3	3.9	8.2
Aspirin 650 + Codeine 60	598	25	5.3	4.1	7.4
Paracetamol 300 + Codeine 30	379	26	5.7	4.0	9.8
Tramadol 50	770	19	8.3	6.0	13.0
Codeine 60	1305	15	16.7	11.0	48.0
Placebo	>10,000	18	N/A	N/A	N/A

What is really interesting is when you start adding in the negative effects of treatment and then comparing the ‘benefit’ to the harm.  The folks at www.thennt.com are kind enough to do that for us in some areas.  May be surprising that mammograms are NOT beneficial and PSA (prostate cancer screening) does more harm than good.  But that never stopped us from pursuing them.  Oh, and if your really want to have some fun, next time your health care provider wants you to start a drug, ask then what the NNT is, so you can make a proper determination.