The Trouble With Studies

I generally tend to trust the results and conclusions drawn from most studies in the exercise science field (especially if it supports my various theories *wink*).  But there are times when I’ll read an exercise science study and it will be interesting and provoke some thought but will ultimately not say anything that pertains to me or my profession. 

The scientific method and the results we draw from it is the best tool we have come up with to date to investigate and learn about the world we inhabit.  From the invention of the wheel to the apple falling on Newton’s head to Los Alamos science and its method have made this world a better place for each successive generation.  We use applied math to get physics, applied physics to get chemistry, applied chemistry to get biology (anatomy and physiology), and then applied biology to get kinesiology and exercise science.  Those last two are anything but straightforward.

Extrapolation

This is probably the biggest problem area when it comes to drawing conclusions.  Most studies are done on a specific population or done in a way that extremely isolates the variable being studied.  Studies need to be done this way otherwise the results are not terribly valid since there will be too many confounding.

I came across a recent study done in Japan that showed that when cardio was performed after strength training there was a higher instance on the biomarkers of muscle synthesis (muscle building)… in middle-aged, untrained men.  Oh.  Most, if not all, exercise science studies are very specific.  They are specific to population and method.  An increase in muscle protein synthesis is great for my middle-aged, untrained, male clients but extrapolating that out to every other client (trained, young, female, etc.) is something that I cannot do.  I can say that there may be a correlation for all other populations based on this one study but that advice should be taken with a grain of salt.

Then there is the whole getting the same results with different methods thing.  In the study the participants performed either 8x8 leg extensions at 70% of their 1-RM, 40 minutes of cycling at 55% of peak VO2 output, or both.  It is suspect at best to say that you’ll get an increase in muscle synthesis when you perform an upper body dominant workout followed by running even if you are an untrained, middle-aged male.  Leg extensions and cycling are both relatively quad dominant exercises which may have had something to do with the results. 

Correlation vs. Causation

There is not one shred of evidence that says smoking causes cancer.  Oh, there is a very high correlation, but causality has never been proven.  I’ll let you unexploded your head and put down that shotgun before I explain.  Lots of people who smoke get cancer right?  Yep.  Well so do a lot of people who don’t smoke.  And a lot of people who do smoke don’t get cancer.  If smoking caused cancer, then 100% of smokers would get cancer.  But they don’t.  A lot of smokers die healthy as a horse.

My whole point is that causality is an extremely difficult thing to prove when it comes to the inner workings of the human body (I used this to win an argument against a kid that goes to MIT.  It may have been my proudest moment ever.).  I would never say that there are no harmful side effects to smoking.  But there is no way to prove that all the smokers who have gotten cancer wouldn’t have gotten it anyway if they hadn’t smoked.  But seriously, don’t smoke.

This brings me to a literal cash cow in the research field.

The Current State of Obesity Research

As the collective American waistline has exploded so has obesity research.  It seems that just about every other week there is a new study that shows that some compound, either exogenous (coming from outside the body) or endogenous (coming from within), leads to weight loss or lowers blood pressure or yadda yadda yadda.  In the coming years I feel the biggest implications will be for those who are at risk for developing type II diabetes.

Obesity and all of the diseases that are highly correlated with it- cancer, diabetes, heart disease, metabolic syndrome,etc.- is a condition that is primarily brought on by lifestyle.  Yes there are people who have legitimate hormonal claims for their excess weight, but they are few and far between.  The rest of us it seems would rather live a life of excess and take umpteen pills per day instead of eat healthfully and exercise.

Then there are those who fall victim to all of the marketing companies that disguise themselves as supplement companies.  Whenever you see a claim that looks like this, “Scorch 57% more fat with Raspberry Ketones (or any weight loss supplement)!  Clinically proven to melt stubborn belly fat!”  I just shake my head.  I first ask myself, “57% of what?”  More often than not it’s 57% of three pounds or so.  This means the compound helped you lose about 1.5 more pounds however long the study was, usually 12-16 weeks.  That may sound like a boon town until you realize that proper eating habits and consistent exercise can help you do that in a week instead of four months and at no extra cost.  You take your pick.

A big topic of discussion these days on obesity research is now dealing with the hyper-palatability of food.  It seems our brains are not wired to be able to handle the high amounts of sugar, salt, and fat that are present in most foods, especially processed foods.  In a nutshell, all the fatty, sugar-filled, processed food desensitizes our brain’s pleasure centers and leads us to seek more of the food that gives us the happy happy good time feelings that we crave.  In other words, we become addicted to it. 

While it is somewhat fascinating learning what is going on in our heads when we start on the second pound of Doritos for the night I can’t help but think, “So eat something else.”  It is a conundrum that people will devour food that is of horrible quality and not much better tasting when they have much better tasting options readily available to them.  I would much rather eat six dollars’ worth of steak that I bought from the grocery store than six dollars’ worth of Taco Bell.  Yet that’s what most people do.

And right now pharmaceutical companies are hard at work to develop medication that reverses this process in the brain and all the other damage done by diabetes, heart disease, and metabolic syndrome.  You know what else works for all of those things?  Yep, eating healthfully and exercising regularly (especially strength training).  But big pharmaceutical companies don’t want you to know that.  They have bottom lines to protect.

Methods to the Madness

When looking at exercise related studies it is important to note the methods of the study.  The methods section is where you find out exactly how the study was carried out.  It will tell you specifics of the population(s) and specifics of what exactly they did to get the results the researchers got, and what exactly the researchers measured to get the results that they came up with.  This section is also where studies usually fall apart.

A recent study was performed to test whether kettle bell exercises or barbell exercises were superior to increase athletic performance, i.e. jumping and sprinting.  The kettle bell group performed their exercises with a fixed weight of 35 pounds whereas the barbell group performed their exercises at a given percentage of their 1 rep maximum.  Right there the study is deeply flawed.  For the most part there should only be one variable changed between the groups, kettle bells vs. barbells.  But now you’ve got differing loading parameters.  Any results after that is uncovered should be taken with a grain of salt at best and thrown out at worst.

And that’s just one example.  As unfortunate as it is many times researches will be pressured from the people that are funding the study to get results that are consistent with the funder’s interests.  It’s hard to circumvent this as most research is privately funded by groups that are not giving away money for the pure love of scientific research.  They want a return on their investment.  If you want a certain answer, chances are there is a way to find it.

Reductionism

Reductionism is the process of controlling for all variables except for the one being experimented on.  If you’ve ever gone through a human biology class or even thought about all the things your body does without you even thinking about it you realize reductionism is not how the body works.

Many kinesiological studies are done with EKG which measures muscle activity in one or more muscles.  The problem with this is that the body is much more coordinated than many think.  The other problem is that different people have different compensations.  Which means that in a study measuring gluteus maximus activity in a certain exercise could have widely varying degrees of muscle activity due to differences in how people’s bodies coordinate muscle activity.  It’s not the movement that is different; it’s how people’s bodies complete the task.

The same goes for physiological (hormones, body fluids, metabolism, etc.) studies.  There are so many different hormones and compounds at work when a person eats and digests a simple compound, like glucose, the simplest sugar for instance, that to control and study any one of them is exceedingly difficult at best.  These studies can give us a good idea of what is going on but will never tell the whole story.

This brings me to the subject of the GI and GL (glycemic index and glycemic load) of foods.  Foods with a higher GI tend to have a greater impact on insulin and blood sugar levels and produce a big glycemic load.  Foods with a low GI will not have a big impact on insulin, blood sugar levels, and glycemic load.  Those numbers were derived from test subjects eating a certain amount (I think it was 100 grams) of only that food and then having blood sugar and insulin tested.  Rarely is that done in the real world.  People eat meals of different foods.  A food that has a high GI when eaten alone will have a totally different GI when eaten with protein, fat, or even another food with a lower GI.

GI and GL are good reference points for food choices, but they should not be the only consideration.

Finally, The Good

I don’t want this to sound like I’m railing against science and finding things out, quite the contrary.  There are many people doing good science for the right reasons.  And that is to improve quality of life.  Studies that show increased bone density in women who strength train,correlate intermittent fasting with improvements in efficacy of cancer medications, and strength and movement training leads to reductions in ACLtears in female athletes are all examples of these. 

There is a difference to these studies and the ones that I’ve been talking about.  These studies are not anthropometric (easily measurable results such as strength increases, reduction of body fat/weight, muscle gain, etc.) whereas most of the studies that are a little off base are.  The biggest factor is time.  I can’t look at a woman and say, “Yep, that strength training protocol really increased your bone density.”  X-ray vision would be cool but for now I’ll rely on scientists.

In Conclusion or, What the Hell Are We Looking At?

In the end studies only provide you a very myopic view on training.  I would much rather trust anecdotal evidence that has been proven true time and time again in training halls and training dungeons alike than a new study done in the sterile conditions of a lab.  Studies can show us increases in protein kinase or increased phosphorylation rates or an upregulation of mRNA but it doesn’t mean anything if it doesn’t produce tangible results. 

We aren’t after the biomarkers of hypertrophy, strength, and fat loss.  We want the results.  Anecdotal evidence and experience should drive the choices we make regarding training and nutrition.  Did something work for you and not someone else, or vice versa?  If you’ve been exercising and experimenting with it for any period of time you know that that’s how it goes.  Studies should merely serve as a means of confirmation. 

If I find something that helps me improve strength and hypertrophy and fat loss that bucks scientific literature I really don’t care, and neither should you.  Fitness is a journey, not an end point.  Keep learning and reading but never stop experimenting.