Say you have chronic pain in your Achilles tendon and your doctor wants to operate. He read in a medical journal that surgery has a 90 percent success rate. The success rate sounds good, but is it a number that can be trusted?
These authors reviewed 26 studies that reported on Achilles surgery between 1969 and 1999. The authors looked at the studies’ scientific methods–the way they got their information–in order to figure out whether these published success rates were reliable.
Ideally, scientific methods keep information pure and accurate. This makes published success rates more trustworthy. If, on the other hand, a researcher only chooses to report on patients who have good results from Achilles surgery, the success rate may paint an overly positive picture.
The authors scored each study’s scientific methods on a scale from zero to 100. A perfect score of 100 meant the study was completely sound and unbiased. Actual scores ranged from two to 74. Only nine studies got scores over 50. That means most of them got “failing” grades.
The average success rate of Achilles surgery in these studies was 77 percent. Compared to studies with lower success rates, the ones with rates over 70 percent tended to use poorer methods. Higher success rates may be unrealistic if they come from sketchy methods.
Fortunately, success rates after Achilles surgery appear to have gotten more trustworthy over the years. The methods were better for recent studies than for those done back in the 70s.
So, when checking out medical information, it’s a good idea to look at how it was gathered in the first place. The authors suggest ways for researchers to improve their analysis of Achilles surgery results. In the meantime, patients should be aware that if it all sounds too good to be true, it might be.