Diet and health. What can you believe: or does bacon kill you?
This post is not about quackery, nor university politics. It is about inference, How do we know what we should eat? The question interests everyone, but what do we actually know? Not as much as you might think from the number of column-inches devoted to the topic. The discussion below is a synopsis of parts of an article called "In praise of randomisation", written as a contribution to a forthcoming book, Evidence, Inference and Enquiry. About a year ago just about every newspaper carried a story much like this one in the Daily Telegraph, What, I wondered, was the evidence behind these dire warnings. They did not come from a lifestyle guru, a diet faddist or a supplement salesman. This is nothing to do with quackery. The numbers come from the 2007 report of the World Cancer Research Fund and American Institute for Cancer Research, with the title ' Food, Nutrition, Physical Activity, and the Prevention of Cancer: a Global Perspective'. This is a 537 page report with over 4,400 references. Its panel was chaired by Professor Sir Michael Marmot, UCL's professor of Epidemiology and Public Health. He is a distinguished epidemiologist, renowned for his work on the relation between poverty and health. Nevertheless there has never been a randomised trial to test the carcinogenicity of bacon, so it seems reasonable to ask how strong is the evidence that you shouldn't eat it? It turns out to be surprisingly flimsy. In praise of randomisation Everyone knows about the problem of causality in principle. Post hoc ergo propter hoc; confusion of sequence and consequence; confusion of correlation and cause. This is not a trivial problem. It is probably the main reason why ineffective treatments often appear to work. It is traded on by the vast and unscrupulous alternative medicine industry. It is, very probably, the reason why we are bombarded every day with conflicting advice on what to eat. This is a bad thing, for two reasons. First, we end up confused about what we should eat. But worse still, the conflicting nature of the advice gives science as a whole a bad reputation. Every time a white-coated scientist appears in the media to tell us that a glass of wine per day is good/bad for us (delete according to the phase of the moon) the general public just laugh. In the case of sausages and bacon, suppose that there is a correlation between eating them and developing colorectal cancer. How do we know that it was eating the bacon that caused the cancer - that the relationship is causal? The answer is that there is no way to be sure if we have simply observed the association. It could always be that the sort of people who eat bacon are also the sort of people who get colorectal cancer. But the question of causality is absolutely crucial, because if it is not causal, then stopping eating bacon won't reduce your risk of cancer. The recommendation to avoid all processed meat in the WCRF report (2007) is sensible only if the relationship is causal. Barker Bausell said: [Page39] "But why should nonscientists care one iota about something as esoteric as causal inference? I believe that the answer to this question is because the making of causal inferences is part of our job description as Homo Sapiens." That should be the mantra of every health journalist, and every newspaper reader. Briefly, a lady claims to be able to tell whether the milk was put in the cup before or after the tea was poured. Fisher points out that to test this you need to present the lady with an equal number of cups that are 'milk first' or 'tea first' (but otherwise indistinguishable) in random order, and count how many she gets right. There is a beautiful analysis of it in Stephen Senn's book, Dicing with Death: Chance, Risk and Health. As it happens, Google books has the whole of the relevant section Fisher's tea test (geddit?), but buy the book anyway. Such is the fame of this example that it was used as the title of a book, The Lady Tasting Tea was published by David Salsburg (my review of it is here) Most studies of diet and health fall into one of three types, case-control studies, cohort (or prospective) studies, or randomised controlled trials (RCTs). Case-control studies are the least satisfactory: they look at people who already have the disease and look back to see how they differ from similar people who don't have the disease. They are retrospective. Cohort studies are better because they are prospective: a large group of people is followed for a long period and their health and diet is recorded and later their disease and death is recorded. But in both sorts of studies,each person decides for him/herself what to eat or what drugs to take. Such studies can never demonstrate causality, though if the effect is really big (like cigarette-smoking and lung cancer) they can give a very good indication. The difference in an RCT is that each person does not choose what to eat, but their diet is allocated randomly to them by someone else. This means that, on average, all other factors that might influence the response are balanced equally between the two groups. Only RCTs can demonstrate causality. Randomisation is a rather beautiful idea. It allows one to remove, in a statistical sense, bias that might result from all the sources that you hadn't realised were there. If you are aware of a source of bias, then measure it. The danger arises from the things you don't know about, or can't measure ( Senn, 2004; Senn, 2003). Although it guarantees freedom from bias only in a long run statistical sense, that is the best that can be done. Everything else is worse. Ben Goldacre has referred memorably to the newspapers' ongoing "Sisyphean task of dividing all the inanimate objects in the world into the ones that either cause or cure cancer" ( Goldacre, 2008). This has even given rise to a blog. " The Daily Mail Oncological Ontology Project ". The problem arises in assessing causality. It wouldn't be so bad if the problem were restricted to the media. It is much more worrying that the problem of establishing causality often seems to be underestimated by the authors of papers themselves. It is a matter of speculation why this happens. Part of the reason is, no doubt, a genuine wish to discover something that will benefit mankind. But it is hard not to think that hubris and self-promotion may also play a role. Anything whatsoever that purports to relate diet to health is guaranteed to get uncritical newspaper headlines. At the heart of the problem lies the great difficulty in doing randomised studies of the effect of diet and health. There can be no better illustration of the vital importance of randomisation than in this field. And, notwithstanding the generally uncritical reporting of stories about diet and health, one of the best accounts of the need for randomisation was written by a journalist, Gary Taubes, and it appeared in the New York Times ( Taubes, 2007). The case of hormone replacement therapy In the 1990s hormone replacement therapy (HRT) was recommended not only to relieve the unpleasant symptoms of the menopause, but also because cohort studies suggested that HRT would reduce heart disease and osteoporosis in older women. For these reasons, by 2001, 15 million US women (perhaps 5 million older women) were taking HRT ( Taubes, 2007). These recommendations were based largely on the Harvard Nurses' Study. This was a prospective cohort study in which 122,000 nurses were followed over time, starting in 1976 (these are the ones who responded out of the 170,000 requests sent out). In 1994, it was said ( Manson, 1994) that nearly all of the more than 30 observational studies suggested a reduced risk of coronary heart disease (CHD) among women receiving oestrogen therapy. A meta-analysis gave an estimated 44% reduction of CHD. Although warnings were given about the lack of randomised studies, the results were nevertheless acted upon as though they were true. But they were wrong. When proper randomised studies were done, not only did it turn out that CHD was not reduced: it was actually increased. The Women's Health Initiative Study ( Rossouw et al., 2002) was a randomized double blind trial on 16,608 postmenopausal women aged 50-79 years and its results contradicted the conclusions from all the earlier cohort studies. HRT increased risks of heart disease, stroke, blood clots, breast cancer (though possibly helped with osteoporosis and perhaps colorectal cancer). After an average 5.2 years of follow-up, the trial was stopped because of the apparent increase in breast cancer in the HRT group. The relative risk (HRT relative to placebo) of CHD was 1.29 (95% confidence interval 1.02 to 1.63) (286 cases altogether) and for breast cancer 1.26 (1.00 -1.59) (290 cases). Rather than there being a 44% reduction of risk, it seems that there was actually a 30% increase in risk. Notice that these are actually quite small risks, and on the margin of statistical significance. For the purposes of communicating the nature of the risk to an individual person it is usually better to specify the absolute risk rather than relative risk. The absolute number of CHD cases per 10,000 person-years is about 29 on placebo and 36 on HRT, so the increased risk of any individual is quite small. Multiplied over the whole population though, the number is no longer small. Several plausible reasons for these contradictory results are discussed by Taubes,(2007): it seems that women who choose to take HRT are healthier than those who don't. In fact the story has become a bit more complicated since then: the effect of HRT depends on when it is started and on how long it is taken ( Vandenbroucke, 2009). This is perhaps one of the most dramatic illustrations of the value of randomised controlled trials (RCTs). Reliance on observations of correlations suggested a 44% reduction in CHD, the randomised trial gave a 30% increase in CHD. Insistence on randomisation is not just pedantry. It is essential if you want to get the right answer. Having dealt with the cautionary tale of HRT, we can now get back to the 'Sisyphean task of dividing all the inanimate objects in the world into the ones that either cause or cure cancer'. The case of processed meat The WCRF report (2007) makes some pretty firm recommendations. Don't get overweight Be moderately physically active, equivalent to brisk walking for at least 30 minutes every day Consume energy-dense foods sparingly. Avoid sugary drinks. Consume 'fast foods' sparingly, if at all Eat at least five portions/servings (at least 400 g or 14 oz) of a variety of non-starchy vegetables and of fruits every day. Eat relatively unprocessed cereals (grains) and/or pulses (legumes) with every meal. Limit refined starchy foods People who eat red meat to consume less than 500 g (18 oz) a week, very little if any to be processed. If alcoholic drinks are consumed, limit consumption to no more than two drinks a day for men and one drink a day for women. Avoid salt-preserved, salted, or salty foods; preserve foods without using salt. Limit consumption of processed foods with added salt to ensure an intake of less than 6 g (2.4 g sodium) a day. Dietary supplements are not recommended for cancer prevention. These all sound pretty sensible but they are very prescriptive. And of course the recommendations make sense only insofar as the various dietary factors cause cancer. If the association is not causal, changing your diet won't help. Note that dietary supplements are NOT recommended. I'll concentrate on the evidence that lies behind "People who . . . very little if any to be processed." The problem of establishing causality is dicussed in the report in detail. In section 3.4 the report says " . . . causal relationships between food and nutrition, and physical activity can be confidently inferred when epidemiological evidence, and experimental and other biological findings, are consistent, unbiased, strong, graded, coherent, repeated, and plausible." The case of processed meat is dealt with in chapter 4.3 (p. 148) of the report. "Processed meats" include sausages and frankfurters, and 'hot dogs', to which nitrates/nitrites or other preservatives are added, are also processed meats. Minced meats sometimes, but not always, fall inside this definition if they are preserved chemically. The same point applies to 'hamburgers'. The evidence for harmfulness of processed meat was described as "convincing", and this is the highest level of confidence in the report, though this conclusion has been challenged ( Truswell, 2009) . How well does the evidence obey the criteria for the relationship being causal? Twelve prospective cohort studies showed increased risk for the highest intake group when compared to the lowest, though this was statistically significant in only three of them. One study reported non-significant decreased risk and one study reported that there was no effect on risk. These results are summarised in this forest plot (see also Lewis & Clark, 2001) Each line represents a separate study. The size of the square represents the precision (weight) for each. The horizontal bars show the 95% confidence intervals. If it were possible to repeat the observations many times on the same population, the 95% CL would be different on each repeat experiment, but 19 out of 20 (95%) of the intervals would contain the true value (and 1 in 20 would not contain the true value). If the bar does not overlap the vertical line at relative risk = 1 (i.e. no effect) this is equivalent to saying that there is a statistically significant difference from 1 with P < 0.05. That means, very roughly, that there is a 1 in 20 chance of making a fool of yourself if you claim that the association is real, rather than being a result of chance ( more detail here), There is certainly a tendency for the relative risks to be above one, though not by much, Pooling the results sounds like a good idea. The method for doing this is called meta-analysis . Meta-analysis was possible on five studies, shown below. The outcome is shown by the red diamond at the bottom, labelled "summary effect", and the width of the diamond indicates the 95% confidence interval. In this case the final result for association between processed meat intake and colorectal cancer was a relative risk of 1.21 (95% CI 1.04-1.42) per 50 g/day. This is presumably where the headline value of a 20% increase in risk came from. Support came from a meta-analysis of 14 cohort studies, which reported a relative risk for processed meat of 1.09 (95% CI 1.05 - 1.13) per 30 g/day ( Larsson & Wolk, 2006). Since then another study has come up with similar numbers ( Sinha etal. , 2009). This consistency suggests a real association, but it cannot be taken as evidence for causality. Observational studies on HRT were just as consistent, but they were wrong. The accompanying editorial ( Popkin, 2009) points out that there are rather more important reasons to limit meat consumption, like the environmental footprint of most meat production, water supply, deforestation and so on. So the outcome from vast numbers of observations is an association that only just reaches the P = 0.05 level of statistical significance. But even if the association is real, not a result of chance sampling error, that doesn't help in the least in establishing causality. There are two more criteria that might help, a good relationship between dose and response, and a plausible mechanism. Dose - response relationship Doll & Peto, 1978 There have been six studies that relate consumption of processed meat to incidence of colorectal cancer. All six dose-response relationships are shown in the WCRG report. Here they are. This Figure was later revised to This is the point where my credulity begins to get strained. Dose - response curves are part of the stock in trade of pharmacologists. The technical description of these six curves is, roughly, 'bloody horizontal'. The report says "A dose-response relationship was also apparent from cohort studies that consumption in I simply cannot that any relationship whatsoever is They are certainly the least dose-response relationships I have Nevertheless a meta-analysis came up with a for response that just the level of statistical significance. The conclusion of the report for processed meat and colorectal cancer was as is a of evidence, with a dose-response relationship apparent from cohort studies. There is strong evidence for plausible in meat is a cause of colorectal But the dose-response curves look and it is reasonable to ask whether public should be based on a 1 in 20 chance of being quite in 20 at best Senn, 2008). I certainly wouldn't want to risk my on like that, never it as a for public So we are with as the bit of evidence for causality. who has much experimental work knows that it is possible to up a plausible of any result Most are and so is a pretty play is of the fact that meats contain and but there is no real evidence that the they contain is The main source of in the diet is not from meat but from vegetables vegetables like and which - of The in processed meat is - but - 400 with a limit of - 400 Dietary intake was not with risk for colorectal cancer in two cohort 2004; for Research on 2006). to the another cohort study on over people compared and from and from all were not different between and meat ( et al., more the incidence of all was among than among meat the was colorectal cancer which a incidence in than in meat ( et al., et compared cohort studies and RCTs for of diet on risk of coronary heart disease. for was for intake of and and of intake of and foods with a There was also a bit less strong evidence for of and for intake of whole dietary and and But RCTs showed evidence only for and for of the As a final in the of case studies, to & from a of case studies in association was between consumption (at least one of per and the risk of of the with relative risks of for and for for for and for on is one to make of should be on the Health to a reduction in As the authors may simply a general and of a It is observations like this that to make a of making causal inferences from studies. They are simply the observed association even The most about the of red meat and processed meat is not only that they are small but also that they only just the 5 level of statistical significance. It has been in these real are to be in size ( Senn, 2008). still, there as some good reasons to think that many (perhaps even of the that are in this sort of study are not real ( The of the of is to be in small studies, so it is that the large meta-analysis by Larsson & Wolk, "In the present the of the relationship of processed meat consumption with colorectal cancer risk was than in the earlier This is all with the well tendency of randomized trials to show a good effect of but trials to show The reasons, and the for this worrying problem are discussed by ( & & 2009) do randomized studies tell The only of evidence for causality from randomised controlled The in people to over long of time are and that is no one reason why there are RCTs than there are observational studies. But when they have been the results often those from cohort studies. The RCTs of hormone replacement therapy above contradicted the cohort studies and the advice given to women about more illustrations of how plausible about diet can be by RCTs supplements and RCTs have shown that various of supplement do no good and may even do (see At least we now know that per do you no The that might be good for you was never more than a plausible and like so many plausible it has out to be a The is now no more than a though it very for unscrupulous The randomised Women's Health Initiative Dietary ( et al., showed of dietary on though the conclusion has been challenged on the of the possible of reported diet ( et al., 2006). to much about ( et al., 2009) no in over two years between very different They randomly overweight to one of The of from and in the were and and and and and difference could be between the different all that for was the number of It should be added, though, that there were some reasons to think that the may not have to their very well ( 2009). The one gets from RCTs is that the of diet are not like as important as has been inferred from observational studies. So does processed meat give you After all we can to the sausages or bacon give you colorectal The is that really I do know that, on the of the evidence, it seems to to be an to that evidence is that processed meat is a cause of In the there were 5 cases of colorectal cancer per 10,000 population in so a 20% even if it were real, and would result in 6 rather than 5 cases per 10,000 That makes the risk sound trivial for any the other there were cases of colorectal cancer in the in A 20% increase would if the association were causal, about cases as a result of eating processed but no cases if the association were not causal. For the purposes of public health about the question of causality is One has for the that they have to because they are to on the of If it were not already the discussed above make it very that the only sound to causality is a randomised The only to that are when are really The relative risk of lung cancer for a is 20 times that of a and there is a very relationship between dose per and response cancer as shown That is a increase in risk, very different from the 20% for processed meat (and many other dietary could the causality in that The about whether to eat bacon and sausages has to be a It depends on your to the principle. The observations do in my strong evidence for causality, but they are certainly with causality. It could be true so if you want to be on the then avoid course would not be much if your were based on things that just could be true. would be to any relative risk based on observational if it was less than about The Cancer Institute ( 2002) that relative risks less than should be with but to what with means in so the advice very In fact any of the relative risks reported in the WCRF report (2007) this all relative risks are less than (or than for it is best to worrying and get on with your At some point it to to diet on the of There is a to for being It the risk of making people that has a sound It the risk of of And it science into when people at the contradictory of observational The question of how diet and other health is to There is reason to think that it For example one study that breast cancer incidence increased in women who to and up to in the Since then has been put into out The great was but that is the only similar have come to after of and or and cancer count as The WCRF report (2007) has 537 and over and we don't I think we should don't rather more to Ben The of the Part 1 and Part and at Risk The and of about diet and supplements up the of seems to like the line at according to his blog. of the The report has been in Project This studies up to The result of of the was to reduce the apparent risk from eating processed meat from 1.21 (95% CI = 1.04-1.42) in the study to (95% CI = in the The is small to much, though it is in for the that the dose-response curves are The evidence for causality is by of the of processed meat and colorectal cancer
Read more