When reading about health information (on the Web or in print), it is important to understand how scientific research is conducted. Once you know the basics, you'll have a better idea of whether the information is reliable and whether the study results are convincing enough to suggest a change in your lifestyle.
If you're looking at different treatment options with your doctor, you'll be able to consider whether the research you are reading about relates to your situation.
Scientific and medical research is conducted for many reasons, including:
- to develop and test new drugs, treatments, vaccines, or diagnostic techniques
- to discover how diseases develop and why they occur in some people and not others
- to better understand the aging process
Medical research is based on widely accepted methods. The type of study or study design depends on the nature of the question or theory to be tested.
It is important to keep in mind that scientific studies cannot prove anything with absolute certainty. For example, it is not possible to prove that condition "A" causes disease "B," or that treatment "C" cures disease "D" 100% of the time. Given the way that most articles on research appear in the popular press, this may come as a surprise. But, in fact, medical research uses the concept of probability, a mathematical calculation of how likely it is that something (an event or outcome) will or will not occur.
The parts of a research paper
Sometimes the information about a research study reported in the media or on the Web provides you with a clear sense of its value (or lack of value). Other times, however, you may want to learn more. It's useful to know the standard format of research papers in the world of scientific publication if you search on the Web for a summary (abstract) of a research paper, or even access the paper itself. More and more medical journals are publishing their entire contents online.
Medical research papers generally consist of five elements, each of which is explained in more detail below:
The abstract provides an overview or summary of the study in one or two paragraphs. It briefly explains the following:
An easy way to think of this is to ask the questions:
- The objective or purpose of the study, ie, why was the study done?
- The methods, ie, how was the study done?
- The results, ie, what did the researchers find?
- The conclusions, ie, what does it mean?
Many abstracts are available for free through the National Library of Medicine's (NLM) MedLine/PubMed (http://www.nlm.nih.gov/hinfo.html), a search service of medical journal articles provided by the government, or through searches of other databases.
- What? and
- So what?
As stated above, the introduction indicates why the study was done-in other words, its objective or purpose. The introduction also includes background information about why the researchers are asking the question and why it is important.
The methods section describes exactly how the study was done, usually in quite a bit of detail. It includes information on the following:
The methods should also describe how and why participants were selected and why certain individuals were excluded or not allowed to participate (eg, they weren't the age of others in the group being studied, or they had multiple illnesses). Knowing all these details is important so that results can be duplicated to confirm the scientific soundness of the research. If a similar study is conducted and the results vary from the first study, additional research is usually needed to find out the reasons why.
- study design (eg, a retrospective versus a prospective study)
- setting (eg, hospitalized patients versus outpatients)
- participants (eg, their ages, sex, state of general health or specific condition)
- treatments or procedures that were done
The results section summarizes the data and other information that was collected by performing the procedures described in the methods section. It also describes the statistical significance of the findings.
The concept of statistical significance is often indicated by a number called the "p-value." The p-value is used to quantify the probability that the same results could have been obtained by chance.
For a result to be considered statistically significant, the "p-value" must be less than or equal to 0.05. This means that, given a p-value of 0.05, if a particular study were to be conducted 100 separate times, the same results would occur 95 times, and different results would occur only 5 times.
In the conclusions section, the researchers explain and interpret the results from their perspective. These subjective explanations should indicate whether the expected results were obtained or not, and why the researchers think this is so. Any limitations of the study should also be explained. Results are often discussed from the standpoint of how they "fit in" with those of previous studies. The researchers often note the need for a broader study or a follow-up study to further confirm results or, if the results differed from those of previous studies, to determine the reason(s) why.
back to top
As you read a study, be sure to keep in mind the difference between "Results" and "Conclusions." In the Results section, the researchers report summaries of data and other technical, often highly quantified information, based on statistical calculations. In the Conclusions section, the researchers offer subjective explanations for the Results. As you might guess, sometimes, there is a great deal of room for interpretation of research results. Other experts may see the Results differently and draw different Conclusions than the original researchers.
Some types of research studies
There are two main types of research studies: observational and experimental (also known as interventional).
- Observational studies. In an observational study the researcher "observes" the effect of a risk factor, diagnostic test, or treatment on a particular outcome, such as the relationship between high blood pressure and stroke. In this type of study, the researcher may use data that have been collected and recorded in the past, which are called "retrospective" data, or data that are collected once the study begins, which are called "prospective" data.
- Clinical trials are experimental research studies conducted with human participants, in order to test a new medical treatment, drug or device. The purpose of clinical trials is to find new and improved methods of treating different diseases and conditions in people. The U.S. Food and Drug Administration (FDA) oversees clinical trials. There are three phases to systematic clinical trails: Phase I trials determine the criteria for safe and effective use of the therapy. Phase II trials, using relatively small groups of patients, establish whether the therapy, at safe and optimal doses, works. Phase III trials, which usually require much larger numbers of patients, compare the new therapies to established therapies and/or placebo. See the ACT section for information on participating in clinical trials.
- Experimental (or interventional) studies.
In an experimental study, the researcher controls the intervention, that is, the researcher determines who will receive the treatment or test (such as hormone replacement therapy or placebo). Experimental studies are usually done to study a new drug or other treatment, or a new diagnostic test.
It is important to be aware of and understand certain characteristics about how the study was conducted. The best studies are usually randomized, controlled, and double-blind, but what does this mean?
- Randomized means that participants are randomly assigned to intervention groups. This helps to ensure that the different study groups are comparable to one another, given the possibilities of known and unknown risk factors for the disease or condition.
- Controlled means that there is a "control" group or "comparison" group. This group does not receive the treatment or intervention. In a study that is placebo-controlled , one group receives the treatment and the "control" group receives a placebo (an inert substance that has no effect).
- Double-blind means that neither the study participants nor the researchers know which specific participants are receiving the treatment or intervention and which participants are not. When the study ends, the treatments are "unmasked' to see who actually received the placebo and who received the experimental treatment.
| back to top
Things you should know about research studies
Now that you know about the different types of studies, the following questions can help you further consider other aspects of the research.
- Was the study done in laboratory animals or in people? Research conducted with people usually tells you more about how the research results might affect you.
- Is the study you're reading the first study on a particular treatment? If it is, take a "wait-and-see" approach. Researchers generally gather and evaluate information from a number of studies as part of an evidence-based approach. Doctors often wait for results of several studies and conclusions of experts before making decisions about patient treatment.
- Is at least one of the authors an expert in the field? Have the authors published other research in the field? You can search on their names in PubMed's MedLine (http://www.ncbi.nlm.nih.gov/entrez/query.fcgi) and see what else they have published.
- Is the study published in a peer-reviewed journal? In a peer-reviewed journal, the research report has been reviewed by several experts in the same field of study before they are published. In a peer-review process, typically, the names of the authors are not given to the reviewers, and vice versa. Studies published in peer-reviewed journals (eg, Annals of Internal Medicine, Journal of the American Medical Association, Lancet, New England Journal of Medicine, Journal of the American Geriatrics Society) are generally considered to be of higher quality than those found in non-peer-reviewed journals. However, the peer review system is far from perfect, and judging research based on where it is published can have some limitations. A June 2002 article from The New York Times called "When Peer Review Yields Unsound Science" points to some of these issues.
Here are some excerpts from the article by Lawrence K. Altman, MD:1
"Medical journals are the prime source of information about scientific advances that can change how doctors treat patients in offices and in hospitals. And to ensure the quality of what journals publish, their editors, beginning 200 years ago, have increasingly called on scientific peers to review new findings from research in test tubes and on animals and humans.
The system, known as peer review, is now considered a linchpin of science - yet for all its acclaim, the system has long been controversial. Despite its system of checks and balances, a number of errors, plagiarism and even outright fraud have slipped through it. At the same time, the system has created a kind of Good Housekeeping Seal of Approval that gets stamped on research published in journals. Although most research is solid, and in some cases groundbreaking, problems have persisted. A particular concern is that because editors and reviewers examine only what authors summarize, not raw data, the system can provide false reassurances that what is published is scientifically sound."
- Was there any selection bias in the study? That is, did the study participants have something in common that could possibly influence the results? For example, were they all very healthy? If so, the study results might not apply to less healthy people. Researchers should comment on the limitations of the study in the Conclusions section.
- How many people participated in (and dropped out of) the study? The larger the size of the study group (sample size), the more likely it is that the relationships between the disease or condition and the treatment are not due to chance alone. On the other hand, smaller randomized, placebo-controlled, double-blind studies can also yield important information because of the excellent study design. You should note how many people dropped out of the study (ie, how many "withdrew" from the study or were "lost to follow up"). The more people who dropped out, the weaker the findings. The usual acceptable drop-out rate is 20-30%.
- Are the results reported in a clear, understandable way? An example of clearly stated results is: "Age, living alone, activities of daily living restriction, cognitive impairment, poor/fair self-perceived health, and contact with services increased the risk of [nursing home] entry. Home ownership decreased it. Gender was not a significant predictor of [nursing home] entry once other factors were controlled for, and no significant effect was found for income." 2
- In addition to "statistical significance", do the authors discuss "clinical significance" as well? It's important that the study results relate to the real world rather than just to the controlled conditions of the study. A study involving a large number of participants may generate results that have statistical significance. However, these results may not be clinically significant. For example, a large study may show that a certain drug lowered blood pressure by an average of two points. However, lowering blood pressure by 2 points may not decrease a patient's symptoms of the condition, meaning that the drug's effect is not clinically significant.
- How current are the references that support the findings? Look at the bibliography or reference list at the end of the research article. This is a list of other publications the authors referred to in designing, conducting, and evaluating their research. The study is not likely to be based on current research if the references are all 10-20 years old.
- Who funded or sponsored the study? If the study involves a commercial product, see if the manufacturer funded the study. You might want to do an Internet search for papers on the same topic to see if those studies had similar results.
Would you like to see an example of how a study is read and evaluated? On HealthNewsReview.org, you can see exactly how that particular organization rates health-related stories. The site describes how news stories are identified and provides a comprehensive overview of its review criteria.
Visit this page of HealthNewsReview.org at: http://www.healthnewsreview.org/ratings_info.php
back to top