top of page
Search

When Bad Study Design Comes Back to Bite You

A recent study published by Séralini et al. (Séralini study) about the long term toxicity of Roundup® herbicide and Roundup-tolerant genetically modified (GM) maize NK603 in feed of rats1 has caused quite a stir. Opponents of genetically modified crops have seized upon the results, which showed that rats fed a diet containing GM maize NK603 or given water containing Roundup® at residue levels permitted in the United States, died earlier than rats fed a standard diet. Others, including the European Safety Authority (EFSA) have issued statements about the scientific merit of the study due to serious design flaws.2,3 Criticisms from EFSA include unclear objectives, insufficient numbers of animals, use of an inappropriate rat strain and statistical methods, absence of suitable control groups for all treatment groups and information about feed composition or intake.2,3 The design flaws are so significant that the authors have been accused of deliberately planning the study to obtain the desired outcome.4  The Séralini study is a great case study of the importance of proper study design. If the study design was virtually bulletproof, and the same results were found, the impact of this study could have been entirely different. The authors could have been basking in the limelight of an important discovery that could have influenced regulation of GM maize NK603 and testing requirements for GM foods. Instead, the study will go down in history as “rubbish”.5 If the investigators could turn back the clock and perform the study again, they could do the following to help gain acceptability by the scientific community:

  1. Have clear objectives. The authors of the study did not state any objectives. “Research objectives define crucial factors such as the study design, correct sample size and the statistical methods used to analyze the data – all of which have a direct impact on the reliability of the findings.”3 Well said, EFSA.

  1. Perform the study according to an established guideline for chronic (e.g. OECD 451, FDA Redbook Section IV.C.5a or comparable), carcinogenicity (e.g. OECD 451, FDA Redbook Section IV. C. 6 or comparable) or combined chronic/carcinogenicity (e.g. OECD 453 or comparable) studies in rats. The authors designed the study based on guidelines for subchronic toxicity studies and in a subsequent publication acknowledged that the study design was not suitable to assess long term carcinogenicity.6 The conclusions about carcinogenicity of the test materials that were original publication were effectively nullified by this admission.