EFSA’s toxicological assessment of aspartame: was it even-handedly trying to identify possible unreliable positives and unreliable negatives?

Background A detailed appraisal is provided of the most recent (December 2013) assessment of the safety and/or toxicity of the artificial sweetener aspartame by the European Food Safety Authority’s Panel on Food Additives and Nutrient Sources Added to Food. That appraisal is prefaced with a contextualising chronological account drawn from a documentary archive of the key highlights of the antecedent scientific and policy debates concerning this sweetener from the early 1970s onwards. The appraisal focuses specifically on Section 3.2 of the panel’s review, which is headed ‘Toxicological data of aspartame’. Methods The methodology of the appraisal focusses on the extent to which the panel was symmetrically alert to possible false positives and false negatives, which in toxicological terms denote misleading indications of possible toxicity or misleading indications of safety. The methodology involved identifying and tabulating the prima facie indications of each of 154 empirical studies, and then comparing them with the way in which the panel chose to interpret the studies’ findings, by focussing primarily on whether the panel deemed those studies to be reliable or unreliable. If the panel had been even-handed, the criteria for assessing reliability should have been the same for both putative positive and negative studies. Results Eighty-one studies were identified that prima facie did not indicate any possible harm, and of those the panel deemed 62 to be reliable and 19 as unreliable. Seventy-three studies were identified that prima facie did indicate possible harm; of those the panel deemed all 73 to be unreliable; none were deemed reliable. A qualitative comparative review of the criteria of appraisal invoked by the panel to judge the reliability of putative negative and positive studies is also provided. Conclusion The quantitative result indicate that the panel’s appraisal of the available studies was asymmetrically more alert to putative false positives than to possible false negatives. The qualitative analysis shows that very demanding criteria were used to judge putative positive studies, while far more lax and forgiving criteria were applied to putative negative studies. Discussion That quantitative and qualitative patterns are very problematic for a body supposed to prioritise the protection of public health. Given the shortcomings of EFSA’s risk assessment of aspartame, and the shortcomings of all previous official toxicological risk assessments of aspartame, it would be premature to conclude that it is acceptably safe. They also imply that the manner in which EFSA panels operate needs to be scrutinised and reformed. Electronic supplementary material The online version of this article (10.1186/s13690-019-0355-z) contains supplementary material, which is available to authorized users.

No consistent statistically significant effect in body weight was observed.
No adverse clinical conditions were reported and 100% survival was reported. No treatment related histopathological changes were reported from the high dose groups.
With only 5 rats per gender per dose group, the absence of a consistent statistically significant finding is not a reliable indicator of the absence of an adverse effect. Implicitly, the wording acknowledged that there were 'effects' but discounted them (Appendix G p 204) as they were not 'unequivocal'.

Appendix G page 204
Decreased food consumption in high dose females, weeks 2-3. Clear, viscous fluid coating intestinal mucosa in treated rats.
"100% survival; no unequivocal effect on body weight or food consumption, except high dose females, wks 2-3." (emphasis added) No adverse physical/behavioural changes; no treatment related pathological changes except for intestinal mucosa.
"N.B. only examined 5 controls and all high dose animals; remaining animals were discarded." (p 204) Mice, n=36/gp. NOAEL = 4000mg/kg bw/day, organ weights unaffected except for dose effects on heart/thyroid in females. "The authors concluded that aspartame, administered to the mouse for 104 weeks in the diet at dose levels of 1000, 2000 and 4000 mg/kg bw/day exhibited no adverse effects regarding survival rate, and that there was no evidence of an effect with respect to the incidence of neoplasms or with regard to nonneoplastic changes in any organ or tissue. The Panel agreed with this evaluation and identified a NOAEL for this study of 4000 mg/kg bw/day, the highest dose level tested." Deviations from guidelines x 5 "...methods implemented were not sufficiently robust to support the results" Rats. n= 30/gp. "The authors of the study concluded that L-phenylalanine on its own or in combination with aspartic acid decreased maternal and pup body weight, which reproduced the observed effects of aspartame on these endpoints. The Panel agreed with the author's conclusion but noted the poor survival of control pups." Significant adverse effects discounted, but reasons not given.

Cont Appendix I page 227
Yes, at the high dose. NOAEL = 2000 mg/kg bw/day, abortions in high dose group, but that group did not receive 2000 mg/kg bw/day but less! E54 1974 uP and ELlow 3.2.5.1.2 page 75 Yes, "…a significant decrease in fetal body weight and skeletal anomalies..." Rabbits. "…the Panel concluded from these observations that the developmental effects on body weight and skeletal development reported in the aspartame feeding studies may be caused by the significant depression of feed consumption in the high dose group." Although: "The actual aspartame doses were reported to be 1880 and 1870 mg/kg bw/day for the intended 2000 and 4000 mg/kg bw/day groups, respectively. The Panel noted that a significant decrease in fetal body weight and skeletal anomalies were reported for the 4000 mg/kg bw/day group but not for the 2000 mg/kg bw/day group even though both groups received the same dose of aspartame based on feed intake." But if they had the same intake then the panel's reasoning is flawed! 3.2.5.1.2 page 75 Yes, "There were no deaths in the pair-fed control group, two deaths in the 1400 mg/kg bw group and three deaths in the 2400 mg/kg bw group. One female in the high dose group had an early delivery. Conception rates were 96, 81 and 77 % in the control, 1400 and 2400 mg/kg bw groups, respectively." Rabbits. n=26/gp. "There were no deaths in the pairfed control group, two deaths in the 1400 mg/kg bw group and three deaths in the 2400 mg/kg bw group. One female in the high dose group had an early delivery. Conception rates were 96, 81 and 77 % in the control, 1400 and 2400 mg/kg bw groups, respectively."

Cont Appendix I page 229
No comment, but states: NOAEL=2400 mg/kg bw/day. Rabbits. 37 received dose of 750 mg/kg bw/day and 95 had 2000 mg/kg bw/day. "The study was confounded by poor health of the animals and the misdosing. As a result, maternal mortality was high. The administration of aspartame was associated with decreased feed consumption by up to 36 %. The control animals were pair-fed to match the aspartame-treated animals with the lowest feed intake. However, especially aspartame-treated animals with a very restricted feed consumption died and the aspartame-treated animals with a more normal food consumption survived. As a result, the food consumption in the pair-fed control animals was noted to be lower than the food consumption of the aspartame-treated animals. The authors concluded that no embryotoxic or teratogenic effects were observed. However, the Panel considered the study not to be adequate to reach such a conclusion."

Cont Appendix I page 230
Y "Major malformations in 7 fetuses of high dose group"findings not mentioned on page 76 NOAEL = 750 mg/kg bw/day E90 1975 uP and ELlow 3.2.5.1.2 page 76 Yes. "A number of animals died spontaneously during the study… mainly due to misdosing. … No abortions were detected in the control, mid dose and L-aspartic acid groups, two abortions in the low dose aspartame group and 24 abortions were observed in the high dose aspartame group (a significant increase compared to controls) and four in the L-phenylalanine group" Rabbits."...contained a sufficient number of animals for the evaluation of developmental toxicity" high dose effect. NOAEL = 1000mg/kg bw/day. A number of animals died spontaneously during the study… mainly due to misdosing. … No abortions were detected in the control, mid dose and Laspartic acid groups, two abortions in the low dose aspartame group and 24 abortions were observed in the high dose aspartame group (a significant increase compared to controls) and four in the Lphenylalanine group. .. Since the decrease in body weight started several days before (13 and 18 days) the abortions (28 days "...in utero exposure to aspartame in CF-1 mice did not affect the physical and functional development of the visual system of the pups." Mice, 20/group. Significant changes, but authors concluded not biologically meaningful, and the panel agreed. "Time of eye opening was statistically significantly later than control at the lowest and highest doses (14.3 ± 0.15 for both doses vs. 14.8 ± 0.15 in controls), as well as the development of the visual placing which was statistically significantly lower than control at the lowest dose of 500 mg aspartame/kg bw/day (18.8 0.28 vs. 20.5 ± 0.40 in controls). The authors considered these findings as isolated points that may vary in either direction. They were not dose-related and as such, the results were not considered to be biologically meaningful. " Discounted as non-monotonic dose-response relationship. Significant increases to "Maternal plasma phenylalanine and tyrosine levels" and "Fetal plasma tyrosine was significantly higher in aspartame-fed animals compared to controls..." and other changes too.
Yes Rabbits, n=30. Significant increases to "Maternal plasma phenylalanine and tyrosine levels" and "Fetal plasma tyrosine was significantly higher in aspartamefed animals levels" and other changes but the ratios between concentrations changed little, so level changes were discounted by the authors "The authors concluded that the treatment of pregnant rabbits with a high dose of aspartame did not affect the transport of phenylalanine and tyrosine across the placental membrane since the ratios of fetal/maternal plasma amino acid concentrations were unaffected by the treatment." 3.2.6 Other studies on aspartame E94 year not reported uP 3.2.6.1 page 82 Yes. Original typescript of report stated p. 5: "Major lesions…were largely confined to mid-line structures, namely the hypothalmic arcuate nucleus, the subfornical organ and the area postrema." Mice. No results described in report. "The Panel however noted that the design of the study was not adequately described and no control group was included, and, therefore, this study was considered not relevant for the overall risk assessment." Reynolds et al 1976 uP 3.2.6.1 page 82 Yes Reported by the ANS panel as a study on mice. No results were described in the ANS's report. The Panel "...noted the absence of control animals in the study and therefore did not take this study into consideration." But the paper was mis-reported by ANS panel. It was not simply a study of aspartame with mice, it tested both aspartame and msg in both neonatal mice and infant macaque monkeys. In effect the MSG and the aspartame groups were used as if controls in relation to each other, as the purpose of the study was primarily comparative. Moreover the authors said, when comparing the effects of aspartame with those caused by consuming msg: "The lesion encountered at 2 g/kg of aspartame is quite similar to what is seen in a neonatal mouse treated at 0. Rabbits, brain structure -dismissed because of experimental, reporting and analytical problems "...authors conclude that juvenile rats and rabbits are particularly susceptible to neurotoxic effects induced by aspartame." However, the Panel noted that the interpretation of these studies was not possible because of the lack of experimental details, the absence of appropriate control animals and of statistical analysis of the data." 2 separate studies, reported together by the Panel. "Epidemiological data on aspartame were previously reviewed by SCF (2002). The Panel considered and agreed with the conclusions of SCF." This item was cited above, and has already been counted as a 'uP'. The ANS panel's endorsement of the SCF assessment is a candidate for 'instututional inertia'.
Halldorsson et al 2010 uP 3.2.7.1 page 86 Yes. "Statistically significant trends were found in the risk of pre-term delivery with increasing consumption of artificially sweetened drinks (both carbonated and noncarbonated), but not for sugarsweetened drinks." Retrospective epidemiological study : 1996-2002. Limitations mentioned, replication suggested by authors. Panel agree with authors that replication wasrequired, implying that it agreed with authors that their study had major strengths (p 87, para 5, first line), and that there were no important flaws in the methods used. However, panel speculated that risk estimates may have been inflated by residual confounding (including by year of delivery). No account was taken of other dietary sources of methanol, and use of aspartame specifically was not distinguished from that of other artificial sweetener). Therefore, given these limitations, the Panel agreed with the authors who concluded that replication of their findings in another setting was warranted." However, if the findings were treated as reliable the conclusion of the section and the report would have been conspicuously different, but the panel concluded: "Overall, currently available epidemiological data do not suggest that consumption of artificially sweetened soft drinks is a cause of pre-term delivery." (p 88) The panel did not set a temporary ADI contingent upon attempts to replicate the findings. Yes, also sugar sweetened. Meta analysis of above 2 studies, but elevated risk was evident with sugar sweetened drinks too! Panel said: "The analysis indicated similarly elevated risks of pre-term delivery with higher consumption both of sugar-sweetened and of artificially sweetened drinks. This lack of specificity in the associations again points to possible residual confounding. Currently available epidemiological data do not suggest that consumption of artificially sweetened soft drinks is a cause of pre-term delivery." (p 88) Maslova et al 2013 uP 3.2.7.1.2 page 88 Yes, increased risk of asthma.
Asthma, "...weakly suggestive of hazard..." Limitations mentioned, further exploration suggested. "Because in epidemiological terms, the elevations of risk were only small and inconsistent, the findings from this study can only be considered weakly suggestive of hazard i.e. an association between the consumption of artificially sweetened beverages during pregnancy and the diagnosis of asthma or allergic rhinitis in children. Before a final conclusion can be reached with regard to aspartame, the findings need to be explored further with more detailed assessment of exposure to specific artificial sweeteners."