If you search the internet, you will find many opposing views about certain foods or diets. You can also find studies that find seemingly conflicting results. One person claims that a diet will solve all of our problems, and the next person claims that the same diet will kill you. It is no wonder that people are confused about nutrition. In many cases, the people providing misinformation do not actually know that the information is wrong and probably actually believe that it is true. There are also people out there who try to sell products or promote support for an industry by spreading misinformation, however, most people out there just want to provide useful information. We all have the potential to unintentionally spread misinformation that might seem logical on a surface level. So how can we tell the difference between correct information and misinformation? How could you find out if I am lying or spreading misinformation about nutrition? There are many different levels of evidence in nutrition. So, the first step is to assess the quality of the evidence being provided. If we want to meaningfully discuss nutrition, we first need to understand the tools that are used to discover new information and try to think like a scientist. Are all studies equally useful? What is sufficient evidence anyway?
Types of evidence
What kind of evidence is sufficient to support a claim? If a person provides evidence, can we be confident in that evidence? The picture below shows the hierarchical pyramid of evidence. The evidence at the bottom of the pyramid is typically of lower quality and more biased. The evidence at the top of the pyramid is typically of higher quality and less biased. Of course, these studies at the top may also have some bias, but the pyramid shows general trends. Although there some debate about the correct order of the evidence hierarchy,1,2 the picture below shows the general hierarchy. In the next section, I will introduce the different types of evidence from the bottom to the top of the pyramid.
Sources that are not good evidence
Most our sources of information are not actually good evidence. We typically get our information from friends, family, the news, society, books, documentaries, podcasts, videos, or this blog. These sources may be correct and can provide useful information, but they can also spread misinformation. These media may be biased and are not very good at proving cause and effect. In this section, I will discuss the flaws in these sources.
Anecdotes
An anecdote is an individuals personal and subjective experience. For example, if a person claims to start feeling better after eating a certain diet, that is an anecdote. This is not good scientific evidence. For one thing, the person may exaggerate their experience and they may have confirmation bias. We tend to remember and share positive experiences that support our pre-existing beliefs. However, we also tend not to share experiences that are negative or contradict our beliefs. There are many people who describe their experiences truthfully, but we cannot know that for sure. In many cases, people are not very aware of their personal biases.
Also, anecdotes cannot prove cause and effect. When a person starts a new diet, they change many variables at the same time, so we cannot determine which variable caused their claimed outcome. Furthermore, because an anecdote is an “experiment” on only one person, the result may not be representative of the general population. The outcome could be completely different in someone else’s unique context. We tend to empathise more with people’s personal stories than with statistics. Especially if we trust a person, we may trust their anecdotes even more. An anecdote may feel like important evidence, but the truth is that it is only useful to that person. Of course, if someone consistently feels unwell because they ate a particular food, that is important personal information. However, that result is not a good reason for other people to not eat that food. While we should to some extent listen to our body’s signals, following science-based information at the same time is a good approach.
Social media
There are a lot of people on social media who spread personal anecdotes. There are also people on social media who spread science-based information. Social media allows us to share and learn a lot of important, science-based information. However, there are also many people who provide misinformation through social media. Unfortunately, social media’s algorithms do not do a very good job of checking for misinformation and providing warnings. Also, the nature of social media’s algorithms allow misinformation to spread rapidly.
People who present extreme information, people who post content that provoke strong reactions, or people who provide information that confirms viewer’s bias tend to get more views than people who publish factual or nuanced information. So the algorithms tend to show more of this extreme content to us. Also, if a piece of content has more views, we may subconsciously think that the content is more trustworthy. However, the number of views is not representative of the quality of the information, just the content’s popularity. Algorithms can create information echo chambers. The more we see content that confirms our bias, the more the algorithm will show it to us. However, the fact that others shares our opinions does not prove that our opinions are correct.
Some people create content on popular and controversial topics or promote products in order to make money. If an influencer feels the need to post content consistently, they may not spend sufficient time researching or verifying their information well enough. If a person does not read an entire journal article and only presents general conclusions from the abstract of an article, they may miss important nuances and details. Also, one study does not represent an entire field of science. In order to draw conclusions about an area, all of the articles and the evidence relevant to the field must be considered. Furthermore, if subscribers expect a consistent message from an influencer, or if their message becomes part of their identity, they may not want to stop spreading that misinformation.
Spreading misinformation is much easier than fact-checking and debunking. The original source of misinformation unfortunately tends to get more attention and views than sources of information that prove it wrong. So it is a good approach to be a little suspicious when listening to people’s content, and check information with other more reliable sources.
Other media
There are a number of primary information sources which we will discuss later. Secondary information sources usually talk about primary information sources. So secondary media do not directly create the information themselves. The people who create secondary media might be familiar with the topic and may share useful or nuanced summaries. However, in many cases, the translation and interpretation of primary sources into secondary sources leads to the loss of important information and context. Sometimes secondary media is pure opinion. There may be biases, and they may be persuasive or manipulative. Furthermore, this kind of media is typically not peer-reviewed. For example, many news articles report on scientific studies, but are less useful than the relevant journal articles on the topic. An editorial is an opinion piece written by an editor. Editorials, even if published in academic journals, are less useful than peer-reviewed science.
There are many books about diet and nutrition. The content presented in a book may seem logical and coherent. Also, if a medical doctor or someone with a PhD writes the book, it may seem more trustworthy. However, the more you read diet books, the more you will see a variety of contradictory messages. So we should not automatically believe everything just because it comes in the format of a book. Should we ignore all books? Not quite. There are many high quality evidence-based books out there too. The challenge for us readers is to figure out which books provide good information. First of all, it is a good idea to evaluate the type and quality of the references cited in a book. It is also a good idea to read a variety of opposing books. This way, we can learn about different perspectives on a subject. We then have to figure out why there are inconsistencies between the books we have read. We also shouldn’t automatically trust book reviews. Many readers leave reviews based on the impression or feeling they received from reading a book. However, many readers do not know how to judge the quality of the information provided. An editor may read and edit a book, but this is not the same as having a book peer reviewed by an expert on the subject.
Watching a documentary to learn about a concept for the first time may be a good approach, but it is an even better idea to look at the documentary’s cited articles to get more specific information. Many documentaries are well-intentioned and provide correct facts, but we should compare and verify those facts with other kinds of evidence. There also tends to be many anecdotes in documentaries, even in ones that promote plant-based diets. A documentary would not be very interesting if they only explained the nuances within journal articles the whole time, and many viewers would not watch it until the end. The average person cannot relate very well to statistics from a scientific study, but they can relate to the story of a person in front of them. As I mentioned before, we can learn about possible outcomes from anecdotes, but we cannot gain any generalisable information from them. For example, a documentary called The Game Changers showcased professional athletes who eat a plant-based diets. This documentary did not prove that these athletes perform better at sport due to the plant-based diet. However, we can conclude from this documentary that it is possible to perform well athletically while eating a plant-based diet. This distinction is subtle but important.
Other media such as blogs, podcasts, and YouTube videos are also not primary sources. This blog is also not high-quality primary evidence. I do not directly prove anything through this blog. Instead, I simply synthesise what other scientists have already proven. So, should you just ignore this blog? Finding and interpreting journal articles on our own is time-consuming. So if you want to read this blog instead, it would be a good idea to check my sources and see the information for yourself. A cited primary source is always more important than a secondary source.
In short, these media are an important vessel for spreading information to the public. There are evidence-based media platforms, but at the same time there is also a lot of misinformation out there too. We should not ignore all of these media. Instead, we should look for media that cite high-quality evidence. Also, if a media sources happens to spread some misinformation, we should not hold that against them forever. Besides that misleading piece of information, that source may also provide a lot of good information. So we should check each fact individually before believing or dismissing it.
Low-quality evidence
Now let’s talk about scientific evidence. Even if a piece of evidence is science-based, and even if it is published in an academic journal, it may not provide us with important or useful information about nutrition. When I say that the quality of a piece of evidence is low, I am not trying to convey that the scientists involved are not doing a good job. Researchers may design a study well, control all variables , and follow the scientific method, but the results of a study still may not be useful to us because of the shortcomings of a type of experiment. Every study is important to some extent and shows a different side of the whole picture. The quality of a study is not representative of the abilities of the researchers. Instead, the quality of the evidence represents an experiment’s probability of bias and the likelihood of proving cause and effect.
Expert opinion
In the field of nutrition, there are many experts. For example, someone with a PhD or a professor who has studied nutrition could be called an expert. Experts know a lot about the topics that they have studied. However, experts are still human. Experts have opinions and biases, and may even share anecdotal information. Even if an expert is familiar with a topic, they may not have complete information on topics outside of their expertise. In general, if we haven’t studied a topic well, we should have low confidence in our knowledge regarding that topic. There are also many experts out there who provide evidence-based opinions. Expert opinion should be valued more than a person who knows nothing about a subject. However, the opinion of one expert is less important than that of a group of experts. Also, peer-reviewed articles are more useful than expert opinions. When a scientist conducts an experiment or discovers something unique that has not been published before, they write an article and publish it. I will introduce published evidence in the next section.
Narrative reviews
A narrative review summarises information on a topic. These reviews cite multiple studies, look for common trends, discuss contradictory results, recommend future studies, and try to draw general conclusions. However, a systematic method of finding, selecting, and interpreting articles is not implemented. Instead, the authors try to find as many studies on a subject as possible and include and interpret those studies however they like. When done well, narrative reviews can provide a lot of useful understanding. However, narrative reviews can be subjective. The author’s experiences and biases can influence the process in which the review is conducted. Typically, this kind of review does not combine and interpret all of the statistics from relevant studies. As I will explain later, it is better to refer to a systematic review or meta-analysis to reduce these shortcomings.
Mechanistic studies
Mechanistic studies focus on biological and chemical processes within the body. These processes are usually referred to as a mechanism. Mechanistic evidence tries to explain how nutrients interact with the body and why certain results are observed when people eat certain foods. There are a number of ways to obtain this evidence. For example, we can experiment on isolated cells in a laboratory and observe the relevant processes or genes within the cells. We can feed an animal, such as a mouse, and observe the processes that occur within them. Alternatively, we can feed a person and measure the biomarkers that arise within their bodies to show the interaction between those biomarkers and a health outcome. Biomarkers are chemicals that signal the state of the body. For example, cholesterol levels or blood pressure.
Epidemiology
Epidemiology studies populations and looks at the relationship between behaviours and health outcomes. Epidemiology investigates the frequency, effects, and trends of all diseases. There are several types of epidemiological studies, and some epidemiological methods provide better quality evidence than others.
Statistics
Epidemiological studies utilise several statistical concepts to interpret the results. So, if we want to understand epidemiological research, we need to understand a little bit about these concepts.
The effect size quantifies the magnitude of an outcome caused by a treatment, food, or variable. For example, the risk ratio (relative risk) is the ratio of the likelihood of developing a disease between groups due to a food or a treatment. These results are usually used in prospective cohort studies or randomised controlled trials. To illustrate this, let’s consider a hypothetical study comparing a group of people who drink soda every day with a group of people that does not drink soda. If the study found that during the observed period, people who drank soda every day had a 6% chance of developing heart disease, and those who did not drink soda had a 5% chance of developing heart disease, then the risk ratio of drinking soda every day would be 6/5 = 1.2. Therefore, the results of the study suggest that people who drink soda daily are 20% more likely to develop heart disease. The odds ratio is a similar concept, but it is usually used in retrospective cross-sectional studies and case controlled studies. The odds ratio compares the probability that a person who has a health outcome ate or behaved a certain way, with the probability that a person who does not have a health outcome ate or behaved a certain way. The odds ratio is usually an overestimate compared to the risk ratio and hence it is less useful.5
Clinical significance is a judgment that takes into account the effect size, the population investigated, the cost of a treatment or intervention, and the practicality of a treatment or intervention. Clinical significance evaluates if the results of the study should be used as a general treatment. Statistical significance is a measure that quantifies the probability that a result from a study is due to random chance. If the results are statistically significant, we can be more confident that the results are real. The confidence interval is the possible range of the effect size. Usually, a 95% confidence interval is used. For example, if the the risk ratio is 1.2 and the 95% confidence interval is 1.1-1.3, we can be 95% confident that the actual result lie within that range. The wider the confidence interval, the more uncertain the effect size. If the confidence interval includes a null result, for example if the confidence interval of a risk ratio was 0.9-1.4, we cannot be sure that there was an actual effect, and the results are said to be non-statistically significant.5,6 Results are not automatically clinically significant if they are statistically significant.7 For example, a risk ratio result with a statistically significant effect size of 2% is probably not that import to general public.
Case studies
A case study focuses on one person or several people. The study usually focuses on what they eat and any relevant health outcomes. These studies are more controlled than anecdotes, but the results from these studies may not be generalisable to the public. Different people or populations may react differently in the exact same situation compared to the small sample of investigated participants. The results from a case study may suggest a hypothesis, but further testing is required on larger populations to confirm the hypothesis.
Ecological studies
Ecological studies look at large populations. For example, they observe and compare countries or regions. Ecological studies relate the population’s average diet to their average health outcomes. Ecological surveys are more generalisable than case studies because they look at larger populations, but they have many drawbacks. For example, a correlation observed at the large population level may not be representative of everyone in that population. The study may also not consider the confounding variables that may have led to the observed correlation. Moreover, it is also problematic to compare different populations. Because countries and populations have different lifestyles, socioeconomic statuses, and diets, eating the same amount of a food can lead to different results depending on the population. Therefore, it is more useful to compare different people within one country or population rather than comparing different countries or populations. This way, confounding variables are somewhat more controlled. Also, ecological studies do not take time into account. Cause and effect can change over time. If a result appears several years after a participants begins eating an unhealthy dose of a particular food, this cross-sectional study cannot show this relationship. As with case studies, hypotheses can be generated from the results of ecological studies, but more controlled studies are needed to test those hypotheses.
Cross-sectional studies
A cross-sectional study measures people’s health at a single point in time and draws conclusions from that information alone. It is like taking a thin slice from a tomato and trying to make conclusions about what a tomato is based only on that slice. There are some problems with this approach. First of all, a person’s biomarkers can naturally change throughout the day. So, if you measure a certain biomarker at one point in time, the results might be too high or too low. If we measured biomarkers at different times of the day and calculate the average, we could reduce this problem to some extent. The participant’s diets are measured via a survey. A person’s answers to these survey questions may not be complete or representative of their actual diet. Therefore, it is difficult to relate the food that they eat to relevant biomarkers or the probability of developing a disease. However, if there are many participants, the impact of this shortcoming is somewhat reduced.
In addition, cross-sectional studies cannot reliably prove causation. For example, a cross-sectional study cannot conclude that eating a certain food causes a heart attack, even if there was a correlation between eating that food and having a heart attack. Maybe something else caused the heart attach, or maybe people who have heart attacks just tend to prefer that kind of food. Even if certain biomarkers are high in a person who has a heart attack, we cannot conclude for certain that the biomarker in question caused the heart attack. In different contexts, different biomarkers may have different meanings. For example, exercising increases blood pressure. However, high blood pressure can increase the risk of developing heart disease. So, should we then conclude that exercise is bad for our health? Of course, exercise is good for our health. However, maintaining a consistently high blood pressure throughout the day is bad for our health. So context is very important.
Similar to ecological studies, cross-sectional studies do not take time into account. Symptoms of a disease or related biomarkers may develop years after beginning to each a particular food. Chronic diseases can become gradually more severe, so if we measure someone’s biomarkers or the food they eat at just one instant in time, we may miss out on important effects later on. Cross-sectional studies, like the previously mentioned epidemiological studies, provide some initial understanding about a research question, but we need to better test any hypotheses that come from these results.
Case control studies
Case-control studies compare people with a health outcome of interest, such as heart disease, with those who do not. Usually, these groups are designed to include people of similar demographics. Case control studies try to find correlations by comparing what the people in each group ate in the past. However, participants may not remember or report exactly what they ate.
Prospective cohort studies
Case control studies start with an outcome, look into the past, and try to figure out why an outcome occurs. Hence, case control studies are retrospective in nature. These experiments are relatively cheap and good for generating hypotheses. In contrast, prospective cohort studies follow people for a long period of time who do not have a health outcome of interest and try to figure out why some people in that cohort develop a disease. Therefore, prospective cohort studies are longitudinal in nature. Prospective cohort studies take longer to conduct and are more expensive, but they provide higher quality information. In this case, participant’s diet and lifestyle are measured throughout the observational period. This study structure is better at showing cause and effect than the epidemiological studies described earlier, because the participants diet and lifestyle are measured before health outcome arises rather than after, and this relationship is tracked over a long period of time. Furthermore, because cohorts usually include a large number of participants, these studies provide more generalisable results.
Interventions
The type of epidemiological studies mentioned before are referred to as observational research. The researchers observe people’s behaviour and the associated consequences, but do not provide any treatment or intervention. However, there are also experiments in which researchers control what the participants eat.
Controlled clinical trials
Randomised controlled trials prescribe participants diet’s and observe the results. This study design is more like a traditional scientific experiment. Participants are randomly assigned to different groups. One group is required to eat an experimental diet, food, or supplement, which is referred to as the intervention group. The other group is required to eat a placebo diet or a normal diet, which is referred to as the control group. Random assignment seeks to reduce the bias of the outcome. Usually, in order to make a fair comparison between these groups, the two groups usually contain people of similar demographics. Not informing the participants and researchers who was assigned to which group seeks to prevent bias in the results or their interpretation. However, sometimes due to the nature of the study, researchers are not able to hide the assigned groups very well. Information such as biomarkers and health outcomes are measured and recorded throughout the experiment. A few years after the end of a trial, the subject’s health outcomes are sometimes followed up on to determine the long-term effects of an intervention. Randomised controlled trials are the best experimental design for demonstrating cause and effect.
Non-randomised controlled trials are the same, however the groups are not assigned randomly. These studies are more biased than randomised trials, but they are sometimes necessary. For example, if you want to experiment on vegans and end up randomly assigning them to a non-vegan diet, they may not want to eat animal products.
Critical appraisals
Critical appraisals are a systematic summary or interpretation of all of current evidence regarding a topic. This kind of evidence is of higher quality because they are representative of the entire field.
Evidence-based guidelines
Evidence-based guidelines are up to date evidence-based recommendations from groups of experts, nutrition organisations, or health organizations. The experts usually assess the quality of current evidence to create guidelines based the highest quality evidence available. Usually, these guidelines are similar to systematic reviews, which are described in the next section. The public, politicians, and people working in the health sector can use these guidelines to inform decisions.
Systematic reviews
Unlike narrative reviews, systematic reviews are less biased. In this case, the researchers systematically find and interpret studies. Because the criteria for including studies are well defined, researchers are less likely to subjectively select or exclude studies. Systematic reviews, like experiments, attempt to answer a research question using the current evidence available. These reviews usually assess the quality and bias of the evidence that they include, summarise current evidence, and provide recommendations.
Meta-analyses
Meta-analyses combine the results of a variety of studies, interpret them statistically, and provide overall effect sizes and confidence intervals, such as the overall risk ratio. Meta-analyses include studies that attempt to answer similar research questions or that have similar experimental designs. Meta-analyses can provide higher quality information because they include more data. Like a systematic review, a meta-analysis includes or excludes studies in a systematic way. Meta-analyses of randomised controlled trials are of higher quality than meta-analyses of prospective cohort studies or cross-sectional epidemiological studies. In the analysis, the results of studies with more participants or that are of higher quality have a greater weighting on the final outcome. If the results of the included studies do not agree with each other, the researchers will attempt to explain the difference. Occasionally, meta-analyses will accompany a systematic review. The studies included in a meta-analysis are important. The selection criteria for meta-analyses may be too strict or not strict enough, which can lead to missing out on important studies or the inclusion of studies with too much bias.
How can you access high-quality information?
So where can you access these studies? It’s a good idea to search Google Scholar as a first step. This search engine is similar to a normal Google, but you can search for an article’s subject, title, author, year of publication, journal, or digital object identifier (DOI). You can read the abstracts of the articles for free, but in most cases you have to pay to read the full articles. Sometimes, a whole article can be read for free if it is open access. Fortunately, this publishing approach is becoming more common. Sometimes authors can legally send articles for free, but this depends on the academic journal’s policies. Some researchers post pre-prints of their articles for free on sites such Research Gate or Academia.
Many articles can be difficult to understand at first because they typically use complex terminology. Some articles are accompanied by a plain language summary. However, the more you read scientific literature, the easier it will become. Most of the journal articles are published in English. So you can use translation software to get them into your native language.
Conclusions
So where do you get you nutrition information from? It is important to check the quality of our information whether it is regarding nutrition or any other topic. If we refer to higher quality sources of information, then we are more likely to make choices that are correct and that have a lower risk of bias. Every study has its advantages and disadvantages. Low-quality evidence can still provide some important information, and high-quality evidence can still have some bias. By considering all of the evidence as a whole, we can develop a better overall understanding. So, the next time someone else claims something to you, be sure to check their sources. In the next post I will explain common mistakes and misconceptions related to interpreting nutrition studies.
References
1. Clarke B, Gillies D, Illari P, Russo F, Williamson J. Mechanisms and the evidence hierarchy. Topoi. 2014;33:339-360.
2. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evidence Based Medicine. 2016;21(4):125-127.
3. Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. Journal of clinical nursing. 2003;12(1):77-84.
4. Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid hierarchy? Circulation. 2008;118(16):1675-1684.
5. George A, Stead TS, Ganti L. What’s the Risk: Differentiating Risk Ratios, Odds Ratios, and Hazard Ratios? Cureus. 2020.
6. Nakagawa S, Cuthill IC. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biological Reviews. 2007;82(4):591-605.
7. Schober P, Bossers SM, Schwarte LA. Statistical Significance Versus Clinical Importance of Observed Effect Sizes: What Do P Values and Confidence Intervals Really Represent? Anesthesia and analgesia. 2018;126(3):1068-1072.