The harmful health effects of recreational ecstasy: a systematic review of observational evidence

Search strategy for electronic databases

A comprehensive search syntax using indexed keywords (e.g. MeSH, EMTREE) and free-text terms was developed. The search strategy is shown in full in Appendix 3.

Databases searched

The following electronic databases were searched: MEDLINE, EMBASE and PsycINFO (all via Dialog DataStar); Web of Knowledge.

Inclusion/exclusion criteria

The relevance of all evidence was appraised with respect to the following criteria:

Papers in languages other than English

Only studies published in English were included in the review.

Meeting abstracts

Reports published as meeting abstracts wereincluded in the review only if sufficient methodological details were reported to allow critical appraisal of study quality.

General approach

Initially, all included evidence was reviewed to establish a taxonomy of reported outcomes. For each outcome, the available evidence was categorised in a predefined hierarchy of research design:

• Level I Pre-existing systematic research syntheses (systematic reviews, meta-analyses, syntheses of qualitative data)

• Level II Controlled observational studies (cohort studies, case–control studies, etc.)

• Level III Uncontrolled observational evidence (case reports and case series).

Where adequately designed and conducted, Level I evidence was preferred.

Where no adequate Level I evidence was identified for a given outcome, any Level II evidence was systematically reviewed. The quality of research was appraised and described, and findings were reported. Where possible and appropriate, quantitative synthesis of study outcomes was also undertaken (for methods, see Quantitative synthesis of Level II data: general approach, below).

Where neither Level I nor Level II evidence was available, Level III evidence was systematically surveyed.

Critical appraisal

Data extraction

Data were extracted using a bespoke database. Because of the very large volume of material retrieved, it was not possible to satisfy our protocol requirement that all data extraction would be double-checked by a second reviewer; however, the data extracted from the 20 studies on which our syntheses relied most heavily were checked by a second reviewer. There were no major errors, and minor errors were corrected. Data extraction tables have not been reproduced in this report because they would run to many hundreds of pages. Details are available from the authors.

Quantitative synthesis of Level II data: general approach

In deciding the approach to the meta-analysis of outcomes of the included studies, a number of aspects of this dataset need to be considered:

• substantial heterogeneity in the design, risk of bias, population and definition of ecstasy and control exposures

• the wide range and large number of outcome measures reported (in total, 915 different outcome measures were identified in the evidence-base)

• substantial level of multiplicity:

  1. multiple comparisons, i.e. inclusion of more than one ecstasy exposure (e.g. heavy ecstasy users versus light ecstasy users versus ecstasy-naïve controls; current ecstasy users versus former ecstasy users versus ecstasy-naïve controls) or more than one control arm (e.g. ecstasy users versus polydrug-using controls versus drug-naïve controls) in a single study

  2. multiple outcomes, i.e. inclusion of more than one outcome measure assessing a given outcome domain within a single study, either through the reporting of several relevant subscales from a single instrument (e.g. individual immediate memory trials from the RAVLT) or through the reporting of several relevant measures (e.g. the RAVLT and the RBMT)

  3. repeated measures, i.e. comparison between exposure and control over more than one time point (e.g. follow-up over a period of abstinence, with repeated measurements at regular intervals)

• observational basis of comparisons.

Collectively these issues pose a substantial methodological challenge to the application of standard meta-analysis methods. Our methodological approach to each of these issues is discussed below.

Quantitative synthesis of Level II data: technical approach

Depressive symptomatology

The meta-analysis by Sumnall and Cole 200558 of self-reported depressive symptomatology in community samples of ecstasy users found a significantly increased level of depressive symptoms in ecstasy users compared to a mix of polydrug and drug-naïve controls – 22 studies, effect size 0.31 (95% CI 0.18–0.44; p < 0.001). The authors state that they used polydrug controls where available rather than drug-naïve controls, but do not specify more detail. Weighted metaregression analysis showed that estimated lifetime ecstasy use, but not duration of use, dose per episode or abstention period, predicted effect size and that this effect remained after partially controlling for alcohol, amphetamine and cannabis. The effect size for studies was significant using the Beck Depression Inventory (BDI) (0.48; 95% CI 0.29–0.66; p < 0.001) and the Symptom Checklist-90-revised (SCL-90R) (0.26; 95% CI 0.02–0.50; p < 0.05), but using the original SCL-90 it was not. Metaregression also showed decreasing effect size as study size increased: only studies with fewer than 40 subjects produced a significant effect size (16/22). As the funnel plot was significantly asymmetrical, publication bias is likely in this review and is identified as an issue by the authors. There is no narrative synthesis or quality assessment of the studies and the methods of the included studies are unclear.

Memory and neurocognition

Three previous syntheses have conducted meta-analyses based on systematic identification of studies. 59,60,73 None provide a critique of the quality of the included studies.

Kalechstein et al. 73 reported that in their ‘lenient’ group of studies (n = 23), exposure to MDMA, was associated with poorer performance in each of the neurocognitive domains: attention [SMD (Cohen’s d) = 0.40], verbal learning and memory (0.73), non-verbal learning and memory (0.58), motor/psychomotor speed (0.55) and executive systems functioning (0.52) (p < 0.001 for each domain). It is not clear what the matched controls were in terms of other drug use. For the more stringent group of studies (n = 11), results were similar, with verbal learning and memory still showing the greatest effect (SMD = 0.85). No narrative synthesis of the studies was included, so no detail of the quality of the included studies is available.

The effect sizes of Verbaten59 are based on comparisons between the highest ecstasy-using group and a non-ecstasy-using control from each of the 10 included studies. For short-term memory, the mean effect size of – 1.15 remained significant after controlling for lifetime exposure to ecstasy (– 0.95; p < 0.01) and cannabis use (– 0.67; p < 0.01). For long-term memory, the mean effect size of – 1.25 remained significant after controlling for lifetime ecstasy consumption, but not after controlling for lifetime cannabis use (– 1.15; p > 0.05). For sustained attention-processing speed, the mean effect size of 0.41 (p < 0.01) remained significant after controlling for lifetime ecstasy consumption. For attention performance, the mean effect size of – 0.82 remained significant after controlling for lifetime ecstasy consumption and lifetime cannabis consumption.

Laws and Kokkalis60 provide an updated meta-analysis for Verbaten59 of 28 studies. On short-term memory, ecstasy users performed worse than controls in 22 of 25 studies (SMD – 0.63; 95% CI – 0.91 to – 0.41). For long-term memory, ecstasy users performed worse than controls in 17 of 19 studies (SMD – 0.87; 95%CI – 1.38 to – 0.45). Ecstasy users performed worse than controls on verbal memory (SMD – 1.00; 95% CI – 1.45 to – 0.59) and visual memory (SMD – 0.27; 95% CI – 0.55 to – 0.03). While the effect size was larger for long-term than short-term memory, this difference was not significant. Deficits were significantly greater for verbal than for visual memory. No significant differences in effect sizes were observed when comparing drug-naïve with non-naïve controls. There was no effect of lifetime exposure to ecstasy or cannabis use on effect sizes.

Recruitment

Recruiting users of illegal drugs for research studies is challenging and authors have used various methods. Some subjects have been recruited from those attending programmes in drug addiction centres or admitted to long-term rehabilitation programmes, which include urine monitoring for MDMA and other drug use. Other studies recruited active users at raves/dance parties, while others used advertising either in specialist media or via their research institution. The snowball technique has been used extensively: participants initially recruited are encouraged to recruit others by word of mouth. These methods are very likely to provide a non-representative sample of ecstasy users. The samples chosen could reflect subjects with a high proportion of problems associated with ecstasy use (in those already in drug addiction programmes) or those who share certain characteristics unrelated to ecstasy use, such as those who choose to respond to an advertisement. The extent to which results from any of these studies can be generalised to the whole ecstasy-using population is therefore uncertain.

Recruitment of the control group may also lead to bias in the result. Often the control group comprised individuals from the research establishment who reported no illicit drug use. These may be students at a university or health-care workers. Such individuals may be reluctant to report illegal drug use and are also likely to differ systematically from the ecstasy users in other ways, such as socioeconomic status and educational attainment. In some studies, urine samples were screened during the study period, so that self-reported recent drug use could be objectively validated.

Study size

In the majority of studies, a power calculation was not performed. Without a power calculation it is not known what chance the study had of detecting a difference between groups, if a true difference exists. Given the very small sample size of many of these studies, it has to be assumed that the chance of declaring false-negative findings (type 2 error) is high. This point is especially relevant where authors have reported that ecstasy-using groups did not differ from controls in terms of baseline characteristics.

Confounding

Given the lack of randomised and other prospective studies, a major issue for this review was the extent to which confounding variables could be identified and controlled for in the included studies. Some sought to control for potential confounding by matching of groups, stratifying patients according to variables thought to be important, such as amount of ecstasy use (e.g. Dafters et al. 75), or by conducting analyses using potentially important variables as covariates (e.g. Heffernan et al. 76). Many studies, however, did not control for the effect of differing prior exposure, or other confounders, in either the design or the analysis plan. Studies also varied in the extent to which they quantified prior exposure to ecstasy and other drugs. In some, an estimate of total lifetime exposure was made by the authors or sufficient data were presented to enable an estimate to be made.

A limitation around the use of studies describing matched groups is that there is no uniformity amongst the variables considered important to match. One study (Back-Madruga et al. 46), which describes groups as matched, actually uses historical archival controls in which ecstasy use was not questioned. In most cases, matching has been restricted to basic demographic variables, but some also include educational attainment, IQ, socioeconomic variables and concomitant drug use. However, in 27 studies, the analyses had not been adjusted to account for potential confounders. For example, Butler and Montgomery77 found that impulsivity and risk taking was greater in ecstasy users than in non-users and further that risk-taking scores were higher amongst high ecstasy users than low ecstasy users. However, there were significant differences in the use of cocaine, amphetamines and LSD between the groups which were not allowed for in the analysis and so the extent to which this result can be attributed to ecstasy use is uncertain. Similarly, another analysis of depressive symptomatology reports an ‘Ecstasy using’ cohort whose history, when compared to that of controls, featured significantly more consumption of alcohol, nicotine, cannabis, psilocybin, amphetamine, LSD, amyl nitrate, ketamine, cocaine and opiates78 but did not attempt to adjust the results to account for these differences. Attributing harmful health effects to MDMA use rather than to other drugs is therefore extremely difficult. 79

Using only cross-sectional data also limits the extent to which effects can be attributed to a possible cause, as the causal association, should there be one, can go in either direction. For example, a group of studies have noted that novelty-seeking behaviour is stronger among ecstasy users; in these cross-sectional studies the explanation could equally well be either that ecstasy leads to such behaviour or that individuals who already exhibit that behaviour are more likely to use ecstasy.

A small number of studies obtained cross-sectional data and then followed patients up for a period of days to several years to obtain further data. We have classified such studies as ‘ambidirectional’ because, although they have a prospective component (observing different groups over time), the original exposure precedes enrolment into the study and the results may be confounded by factors that were present on enrolment.

Disappointingly, we were compelled to exclude one of the very few prospective studies in this area (Lieb et al. 80) from our review because it only reports results from a cohort exposed to ‘ecstasy, amphetamine or related compounds’ (contact with the authors failed to elicit data limited to those exposed to ecstasy only). Similarly, the longitudinal follow-up study by Daumann et al. 81 conflates the use of ecstasy and amphetamine for follow-up measurements (though not for baseline data, which are included in our review). We appreciate that such classifications are more reflective of common usage patterns; additionally, this means that they are more practical to adopt from a study recruitment perspective. However, it is very difficult to make use of such data in a policy-making context because it is impossible to disentangle the contributions of the various substances to the reported results.

Dose-related effects

Determining any dose-related effects of MDMA is more problematic than with prescribed medication in clinical trials or other studies for a number of reasons. Illegal drugs are not produced with pharmaceutical quality assurance procedures and there is ample evidence of great variability in the dose of MDMA contained in available tablets. Consequently, there is no assurance of the dose taken by participants even if they can recall accurately the number of tablets they have taken. Aside from variability in content of the desired active drug there is also variability in content of contaminants, some of which may exert a pharmacological action. Participants in these studies are perhaps also more likely than patients in clinical trials to have inaccurate recall or to lie about their drug consumption. Any claims for a dose effect must therefore be interpreted very cautiously.

Despite this caution, a number of studies attempt to investigate the suggestion that long-term harm from ecstasy use is associated with heavy use rather than low episodic use. There is variation in the thresholds that different researchers have set for low and high use, but all estimates are based on self-reported use and are subject to recall bias, particularly where use over a number of years is recorded.

Abstinent period

To maximise a study’s ability to distinguish long-term effects from acute and subacute sequelae of drug consumption, it is important to ensure that participants are tested after a period of abstinence long enough to rule out any residual effects of their last dose(s). We did not routinely extract information about the extent of abstinence required by each study before testing, or the means by which compliance with such criteria was verified. However, we note that studies varied widely in this respect. For example, Gerra et al. 82 required participants to have ceased consumption of illegal drugs 3 weeks before testing, and used urine screening three times a week to ensure compliance. In another study, the same author ensured abstinence over a 12-month period by the same method. At the other end of the spectrum, Quednow et al. 83 relied upon subjects’ self-declaration that they were drug free for 3 days before participation in the study.

Blinding

Many studies do not state whether the researchers carrying out the assessments were blinded to the exposure status of the participant.

Outcome measures and reporting bias

A feature of the dataset for this review is the large number and diverse range of outcome measures that researchers have assessed. In many cases the outcomes assessed are subjective and rely on the participants’ self-report of a characteristic. In some cases well-established outcome measures are used, whereas in others the validation of the assessment tool is less clear. Studies assessing personality dimensions and mood tended to make use of subjective measures, while those assessing memory and cognitive function made greater use of objective measures. In many cases the studies did not identify a primary outcome measure but subjected the range of data to statistical analyses and hypothesis tests. In most cases no adjustment to significance level has been made for the multitude of hypothesis tests conducted, and the findings of such studies should be regarded as exploratory and hypothesis generating.

In addition, studies have not always reported all outcomes investigated, but have included only those which yielded positive results. Together with the uncertain, but often large, number of outcomes investigated, this selective reporting adds to the interpretation difficulties and increases the likelihood that many results are chance findings.

The Netherlands XTC Toxicity Study

The Netherlands XTC toxicity study is the only study meeting the inclusion criteria for this review that provided data which can truly be described as prospective. A number of objective tests were employed to assess different aspects of memory and visuospatial functioning, and although references are provided it is not clear to what extent the measurement tools used have been validated. Statistically significant differences between the groups were only observed for measures of verbal memory. A large number of statistical comparisons have been made and it would be a moot point to discuss whether the p-values used to declare significance should have been adjusted to reflect this. The authors also chose to use one-tailed tests as they hypothesised that ecstasy use could have been associated only with impaired performance and not with enhanced performance. It would have been more conservative to have used two-tailed tests, keeping p < 0.05 constant as the level at which to declare statistical significance. The conclusion that exposure to even a low dose of MDMA may impair verbal memory has recently been challenged. It was noted that the difference in scores between the groups arose because the increased performance on retest was greater in the ecstasy-naïve group than in the incident ecstasy-using group, i.e. verbal memory test scores numerically increased in both groups but to a lesser degree in the ecstasy group. The scores remained within the normal range. There is some debate as to whether the relative decline in scores is attributable to ecstasy affecting verbal memory in a way that serves to blunt the benefit of a retest some 18 months after the initial test. The conclusion that these effects are apparent even after a low cumulative dose has also been challenged as the range of ecstasy use was reported as 0.5–70 tablets. In response to this challenge the authors present some sensitivity analysis excluding four subjects (approximately 7% of the sample) who used in excess of 10 tablets, yielding a new group mean consumption of 1.95 tablets (range 0.5–6), which was found to have little effect on the results. Dose of ecstasy per occasion was also considered briefly with data presented showing that 95% of users took no more than two tablets per occasion and that during the period of study the mean dose of MDMA per tablet was 78 mg. The authors conducted logistic regression analysis which showed an increased risk of a decline in a verbal learning test with increased consumption.

The strength of the Netherlands XTC toxicity study is the prospective nature whereby a cohort of ecstasy-naïve subjects was followed up for around 2 years. The sampling methods resulted in a study population that is probably not representative of the general population of young people, but the varied situations from which recruitment occurred and the fact that both the eventual ecstasy-using and the control groups came from this same sample make this study stand out from many of the others. It presents a range of objective cognitive measures, and subjective mood and personality measures.

Although many potential confounders are possible, the authors attempt to identify these and adjust their analysis accordingly. The principal concerns are centred on the direction of results in both the active and control groups in one of only three measures out of a possible 12 that were statistically significant, and the relatively large p-values associated with these in the context of multiple one-tailed hypothesis tests.

Methods

This study started in 2002 with the aims of examining:

• the causality of ecstasy use in observed brain pathology in humans

• the long-term course of brain pathology in ecstasy users

• the clinical relevance of observed brain pathology in ecstasy users.

The study design included three arms:

• a cross-sectional study of heavy users of ecstasy and controls using varying amounts of other drugs

• a prospective cohort study of subjects who were ecstasy-naïve at recruitment but had a high risk for future first ecstasy use

• a retrospective cohort study of lifetime ecstasy users with matched controls.

As this study is the only one we have identified that has included prospective data, we report its methodology and results separately from the rest of the Level II evidence that is purely cross-sectional in nature.

We have identified nine publications from the whole study. Two84,85 describe the methodology, including a detailed assessment of the recruitment techniques,85 particularly the possibility that the investigators’ approach encouraged the drug-naïve subjects to start using ecstasy. Two more publications86,87 report findings from the cross-sectional study; one presents qualitative data from older ecstasy users86 and the other presents neuroimaging data (functional magnetic resonance imaging),87 which are not included in this review. A third report from the cross-sectional arm, identified through an update search and also not fully included in this review, presents cognitive effects in 71 subjects with a spectrum of drug-using histories using a range of instruments. The remaining four publications present results from the prospective cohort arm; two of these report functional magnetic resonance imaging data and are not included in this review,88,89 whereas the others report cognitive90 and depression, impulsivity and sensation-seeking91 data. To date, no publications have been identified that report findings from the retrospective cohort study of lifetime ecstasy users and matched controls identified from a pre-existing longitudinal study in the Netherlands.

Results of the prospective study

One hundred and eighty-eight ecstasy-naïve subjects who were considering ecstasy use in the near future and preferably had at least one friend currently using ecstasy, were recruited over a 2-year period from April 2002 to April 2004. All 188 underwent initial assessment; 158 completed all the follow-up questionnaires of whom 64 said they had started ecstasy use since inclusion in the study and 59 of these 64 participated in the follow-up assessment session, 16–19 months after the initial assessment, together with 61 of the 94 subjects who said they had not used ecstasy, matched for age, sex and IQ (Dutch Adult Reading Test). Subjects were young (average age 21 years) with slightly more women (57%).

At initial assessment, there were no significant differences between those who started using ecstasy and those who did not in terms of age, sex, IQ, educational status and the consumption of other drugs (alcohol, tobacco, amphetamine and cocaine) with the exception of cannabis (greater in those who started using ecstasy, mean joints per week 48.8 versus 17.2, p < 0.05 Mann–Whitney test). There were also no significant differences in any of the neuropsychological or psychopathological tests between the two groups at baseline. 86,87 The mean cumulative dose of ecstasy in those who started using it was three or six tablets, depending on which paper you read.

Baseline total scores for depression (BDI), impulsivity (BIS) and sensation-seeking (SBL) did not predict incident ecstasy use, even after controlling for years of education and alcohol, cannabis and cocaine use. 87 At follow-up, there were significant differences between those using ecstasy and the ecstasy-naïve subjects in three of the subscales of the SBL: experience-seeking (β-coefficient 1.76; 95% CI 0.09–3.42), disinhibition (β-coefficient 3.31; 95% CI 1.74–4.88) and general sensation-seeking (β-coefficient 0.54; 95% CI 0.20–0.87) even after correcting for baseline scores. After correcting for potential confounders, ecstasy use had a significant effect on only the SBL general score and the disinhibition subscale. Cannabis use in the last year had a positive predictive value on future ecstasy use [odds ratio (OR) 1.30; 95% CI 1.08–1.56]. The thrill- and adventure-seeking subscale unexpectedly had a negative predictive value on future first ecstasy use (OR 0.95; 95% CI 0.91–1.00).

At follow-up approximately a year later, there was a significant difference in the change in scores (follow-up minus initial) between those subjects stating that they had started using ecstasy (mean cumulative dose three tablets) and those who remained ecstasy-naïve for immediate and delayed verbal memory (0.86 versus 3.90, p = 0.03; – 0.52 versus 0.65, p = 0.03 respectively). A higher proportion of the ecstasy-using group showed a decline in verbal recognition (22.4% versus 6.7%, p = 0.02). The effect of ecstasy use on delayed verbal memory remained after controlling for cocaine and amphetamine use. All other neuropsychological tests showed no significant differences. The ecstasy-naïve subjects showed a normal retest effect, but this was not demonstrated in the ecstasy-using group even after controlling for other drug use. 92 Overall test performance for all subjects remained within the normal range of an age- and sex-comparable general population (indeed, all the RAVLT memory scores for which differences were found represent very high-functioning performance, when compared with norms).

In conclusion, the only prospective study we have identified for this review found that a low cumulative dose of ecstasy is associated with a (small) decline in verbal memory and may increase certain aspects of sensation seeking, but is not associated with depression or impulsivity.

Rey Auditory Verbal Learning Test verbal recall (immediate) – MDMA users versus polydrug controls

The Rey Auditory Verbal Learning Test (RAVLT) is one of the most widely used neuropsychological assessment instruments in our evidence-base. Amongst a broad range of subscales reflecting immediate memory, the most commonly reported was the sum of items remembered across all five initial trials in the test. These data are shown and synthesised using a random-effects meta-analysis in Figure 2. We include one study91 for which reported data are based on the Dutch translation of the test.

FIGURE 2.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (immediate) (sum of trials 1–5) – ecstasy users versus polydrug controls: random-effects meta-analysis.

The evidence for worse performance in ecstasy-exposed populations is strong, with a mean difference of around four items. This difference equates to slightly more than half a standard deviation in the normative population (the norm for those aged 20–29 is 56.1 items; SD 7.3). 92

Sensitivity analysis with aggregated comparisons for each study provides a mean difference estimated at – 3.758 (95% CI – 7.126 to – 0.391), suggesting that our primary analysis may marginally overestimate the difference between populations. More notable than this slight reduction in effect estimate is the revised hypothesis test: whereas, in the primary analysis, evidence is strong for a difference between populations (p = 0.007), the sensitivity analysis provides a p-value that, while still comfortably within the bounds of conventional statistical significance, is somewhat less compelling (p = 0.029).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.336), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 15 covariates, shown in Table 5. There was no evidence of a dose–response effect per se (see Figure 88 in Appendix 7). There was the suggestion of an association between duration of use and extent of memory deficit, with those who had used ecstasy for the longest performing worst, relative to their respective controls. However, this trend was not strong enough to achieve conventional statistical significance.

For metaregressions assessing the influence of confounding (inter-arm asymmetry), significant results were seen for age, gender and cocaine exposure. For age (Figure 3), a positive coefficient was estimated, suggesting that the younger the ecstasy users were in comparison to controls, the worse their relative performance in the memory test.

FIGURE 3.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (immediate) (sum of trials 1–5) – ecstasy users versu.polydrug controls: mean difference in score against inter-arm asymmetry in age.

The effect of gender imbalance is shown in Figure 4. It can be seen that the plot is characterised by a number of studies in which gender distribution is well balanced (six of the datapoints appear on or close to the graph’s y-axis). Aside from these studies, it appears to be the case that a negative inter-arm gender difference (indicating that the proportion of males was lower in the ecstasy-exposed arm than in the polydrug controls) is associated with little or no difference between arms in memory performance. Conversely, those studies in which greatest difference was seen between populations were also those in which ecstasy-exposed arms had a greater proportion of men than their respective control groups. These findings may not be a surprise because women are often found to score more highly in the RAVLT than men. 48

FIGURE 4.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (immediate) (sum of trials 1–5) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in gender.

Figure 5 depicts the influence of imbalances in cocaine exposure on measured memory performance. If the model estimated in this analysis were to be accepted, confounding by exposure to cocaine would account for most of the difference between cohorts. The adjusted estimate of mean difference (i.e. the difference that would be expected if groups were perfectly matched for cocaine exposure) is – 1.669 (95% CI – 5.294 to 1.955). Under this model, the evidence for an underlying difference in populations appears weak (p = 0.367).

FIGURE 5.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (immediate) (sum of trials 1–5) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in cocaine exposure (standardised mean difference).

Rey Auditory Verbal Learning Test verbal recall (delayed) – MDMA users versus polydrug controls

In addition to the estimate of immediate memory, we were able to synthesise one RAVLT subscale reflecting delayed verbal memory: items remembered in trial 8. Seven studies provided data on this outcome measure. Details are presented, along with a random-effects meta-analysis, in Figure 6.

FIGURE 6.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (delayed) (trial 8) – ecstasy users versus polydrug controls: random-effects meta-analysis.

The results of this analysis reflect those obtained for RAVLT immediate memory (see Figure 2) fairly closely. Ecstasy-exposed individuals are estimated to recall a little over one item fewer than polydrug controls. Again, this difference equates to approximately half a standard deviation in the normative population (the norm for those aged 20–29 years is 11.3 items; SD 2.3). 92 The probability of such results occurring if there were no underlying difference between cohorts is very small (p < 0.001).

Sensitivity analysis with single, pooled comparisons for each study provides a mean difference estimated at – 1.134 (95% CI – 1.805 to – 0.463), which is extremely close to the primary analysis.

Egger’s test for small-study bias falls some way short of significance (p = 0.145); nevertheless, we note that the funnel plot for this dataset (Figure 7) shows that the most extreme effect estimates tended to come from the least precise studies.

FIGURE 7.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (delayed) (trial 8) – ecstasy users versus polydrug controls: funnel plot.

Sufficient data were available to attempt metaregression analyses for 13 covariates, shown in Table 6. There was a weak suggestion of a dose–response effect, with more extreme effects seen in participants with greater ETLD of ecstasy. However, this finding is based on a small and – visually, at least – not especially convincing dataset (see Figure 89 in Appendix 7).

The one metaregression that did produce a p-value < 0.05 was that using asymmetry in baseline intelligence as covariate (Figure 8). However, the negative coefficient means that the direction of this effect is counterintuitive, suggesting that greater memory deficits can be expected whenever ecstasy-exposed cohorts are more intelligent than their comparators. It is difficult to explain this finding, so it is tempting to infer a Type I error, especially in the context of multiple testing.

FIGURE 8.

Rey Auditory Verbal Learning Test (RAVLT) verbal recall (delayed) (trial 8) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in baseline intelligence measures (standardised mean difference).

Rivermead Behavioural Memory Test prose recall (immediate) – MDMA users versus polydrug controls

The Rivermead Behavioural Memory Test (RBMT) is another instrument that is well represented in the assembled evidence-base. In particular, the prose (story) recall test was administered by enough investigators to make meta-analysis possible for both immediate and delayed memory (for the latter, see next section).

In total, this analysis includes 12 comparisons, drawn from six different studies (eight comparisons from six studies providing data for current ecstasy users and four comparisons from four studies providing data for former ecstasy users). One study was excluded from analysis because it presented only scaled scores. 101 We included data from the 2006 study by Reneman et al. ,97 which presents results as the sum of two consecutive administrations of the test, by halving the reported figures (although this may not provide an accurate estimate of dispersion).

When meta-analysed (Figure 9), these data suggest that ecstasy-exposed cohorts recall an average of two-thirds of an item fewer than polydrug controls. It should be noted that there is a fairly wide range of performance, with control group scores ranging from 4.3 to 9.5. Sensitivity analysis using the aggregated data approach generated a comparable effect estimate (MD – 0.720); however – because this approach was, in this instance, subject to greater uncertainty than the primary analysis – the evidence for an exposure effect has a less statistically robust appearance [95% CI – 1.572 to 0.133; p(null MD) = 0.098]. There is no evidence of small-study bias in this dataset (Egger’s p = 0.332), and the funnel plot (not shown) had an unremarkable appearance.

FIGURE 9.

Rivermead Behavioural Memory Test (RBMT) prose recall (immediate) – ecstasy users versus polydrug controls: random-effects meta-analysis.

Sufficient data were available to attempt metaregression analyses for 12 covariates; details are shown in Table 7, with those analyses with results that achieved or approached conventional levels of significance discussed in detail below. There was no evidence of a dose–response effect (see Figure 90 in Appendix 7).

The most statistically robust – and intuitively appealing – metaregression assesses the relationship between baseline intelligence imbalances and RBMT performance, as shown in Figure 10. The positive coefficient implies that, the greater the extent to which ecstasy users outperformed ecstasy-naïve participants on intelligence measures, the less they could be expected to suffer in comparison to controls when it came to the outcome of interest. This strongly suggests that the apparent exposure effect is confounded by this variable. The intercept of the metaregression – which, in an analysis of this type, provides an adjusted estimate of effect size accounting for the influence of the covariate – is, at – 0.471 (95% CI – 1.126 to 0.183), somewhat reduced compared to the primary analysis. More notably still, the hypothesis test assessing the evidence against a null effect appears much weaker (p = 0.158). According to this model, then, the apparent difference between cohorts is at least partially ascribable to unequal intelligence status.

FIGURE 10.

Rivermead Behavioural Memory Test (RBMT) prose recall (immediate) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in baseline intelligence measures (standardised mean difference).

The second metaregression producing results that would, conventionally, be considered statistically significant covaries asymmetry in exposure to alcohol against the outcome of interest. The relationship between these variables is depicted in Figure 11. While, at first glance, this looks like a relatively strong association, it should be noted that a positive correlation is estimated, suggesting that those studies in which ecstasy-exposed participants exhibited better memory performance were those in which ecstasy users drank more alcohol than controls.

FIGURE 11.

Rivermead Behavioural Memory Test (RBMT) prose recall (immediate) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in exposure to alcohol (standardised mean difference).

Rivermead Behavioural Memory Test prose recall (delayed) – MDMA users versus polydrug controls

This analysis includes the same 12 comparisons from six studies described for immediate recall, above. Once more, the study by Zakzanis et al. 101 was excluded, and the datapoints from Reneman et al. 97 were halved.

When meta-analysed (Figure 12), these data provide a very similar picture to that seen in the synthesis of immediate recall data (Figure 9), suggesting that ecstasy-exposed cohorts recall an average of around three-quarters of an item fewer than polydrug controls. Again, a fairly wide range of performance was seen, with control group scores ranging from 3.85 to 8.95. Sensitivity analysis using the aggregated data approach generated a similar effect estimate and, in this instance, the reanalysis retained an exposure effect that would conventionally be considered statistically significant [MD – 0.864; 95% CI – 1.688 to – 0.039; p(null MD) = 0.040].

FIGURE 12.

Rivermead Behavioural Memory Test (RBMT) prose recall (delayed) – ecstasy users versus polydrug controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.571), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 12 covariates; details are shown in Table 8, with those analyses with results that achieved or approached conventional levels of significance discussed in detail below. There was no evidence of a dose–response effect (see Figure 91 in Appendix 7).

A pronounced positive correlation was found between baseline intelligence imbalances and RBMT delayed memory performance, as shown in Figure 13. This association – which closely reflects the results for RBMT immediate memory (see Figure 10) – suggests that results may be at least partially explained by asymmetry in intelligence. However, even when results are adjusted for this confounding, fairly strong evidence of an underlying exposure effect remains (p = 0.026).

FIGURE 13.

Rivermead Behavioural Memory Test (RBMT) prose recall (delayed) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in baseline intelligence measures (standardised mean difference).

The effects of confounding in exposure to amphetamines and alcohol are shown in Figures 14 and 15 respectively. Once again, these findings are reminiscent of the results seen in equivalent metaregressions for RBMT immediate memory. In both cases, a fairly strong positive correlation is seen, again suggesting that the studies in which ecstasy users also had additional exposure to other substances, when compared to controls, were also those studies in which they performed best on the RBMT.

FIGURE 14.

Rivermead Behavioural Memory Test (RBMT) prose recall (delayed) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in exposure to amphetamines other than MDMA (standardised mean difference).

FIGURE 15.

Rivermead Behavioural Memory Test (RBMT) prose recall (delayed) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in exposure to alcohol (standardised mean difference).

Because the analyses for immediate and delayed memory as measured by the RBMT are based on the same studies, and have very similar results, it is not a surprise to see analogous pictures emerging in metaregression analyses. With this in mind, it should be emphasised that the repetition of surprising results does not necessarily lend further credence to them. If the first paradoxical result-set is dismissed as a Type I error – that is to say, the apparently suggestive results have occurred by chance variation – then one might expect to see that artefactual pattern repeated in other analyses that are based on closely related data.

Digit span (forwards) – MDMA users versus polydrug controls

We identified seven comparisons, drawn from five studies, in which a forwards digit span was used to assess differences in verbal memory between ecstasy users and polydrug controls (six comparisons from five studies providing data for current ecstasy users and a single comparison providing data for former ecstasy users). The digit span data reported by de Win and colleagues90,91 was excluded from this analysis, because the investigators had used modified methods (with three instead of two series of digits per length), leading to scores that were not directly comparable with the other studies.

A random-effects meta-analysis of the identified data (Figure 16) suggests that ecstasy users have an average span of approximately 0.4 digits less than polydrug controls. This effect is just strong enough to achieve conventional statistical significance. Sensitivity analysis using the aggregated data approach generated a very similar effect estimate [MD – 0.412; 95% CI – 0.746 to – 0.078; p(null MD) = 0.016].

FIGURE 16.

Digit span (forwards) – ecstasy users versus polydrug controls: random-effects meta-analysis.

None of the average scores recorded by ecstasy users or controls are outside the normal range for this test (Lezak et al. 48 refer to any span of six or higher as ‘well within normal limits’).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.945), and the funnel plot (not shown) had an unremarkable appearance.

The small size of the dataset meant that we were able to attempt metaregression analyses for only five covariates, none of which provided any significant results. Details are shown in Table 9. There was no evidence of a dose–response effect (see Figure 92 in Appendix 7).

Digit span (backwards) – MDMA users versus polydrug controls

We identified eight comparisons, drawn from six studies, in which a backwards digit span was used to assess differences in verbal memory between ecstasy users and polydrug controls (all data related to current ecstasy users). Once more, we excluded data from the studies by de Win’s group,90,91 because of their inconsistent methods.

Meta-analysed results (Figure 17) are fairly similar to those seen in the forwards digit span, with a significant difference between ecstasy users and controls of about 0.6 digits. Again, all average scores appear to be within the normal range. Sensitivity analysis using the aggregated data approach generated a very similar effect estimate [MD – 0.638; 95% CI – 1.096 to – 0.181; p(null MD) = 0.006].

FIGURE 17.

Digit span (backwards) – ecstasy users versus polydrug controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.416), and the funnel plot (not shown) had an unremarkable appearance.

There were sufficient data to attempt metaregression analyses for seven covariates; details are shown in Table 10. None of the analyses provided significant results, and there was no evidence of a dose–response effect (see Figure 93 in Appendix 7).

IQ (National Adult Reading Test) – MDMA users versus polydrug controls

Of all the measures in the assembled evidence-base, the most frequently reported was IQ as measured by the National Adult Reading Test (it should be noted that we include here studies using foreign-language translations of the test). In the majority of cases, investigators did not present these data as outcomes of interest in their own studies, but rather used them to estimate the underlying intelligence of their participants (the most notable reason for doing so being to ensure a reasonable balance between cohorts). Nevertheless, we have included the National Adult Reading Test as an outcome measure of interest in our analyses because we believed it was reasonable to look for differences between populations with regard to this measure. Of course, if the assumptions underpinning most investigators’ use of the test are correct, then we would hope to see no difference between ecstasy-exposed and ecstasy-naïve populations.

Figure 18 shows our random-effects meta-analysis of these data. It suggests that, while the IQs of ecstasy-exposed individuals were rated an average of 0.321 points lower than those of polydrug controls, this is unlikely to represent a significant difference (p = 0.498). Interestingly, the evidence of lower IQ scores among ecstasy users was somewhat stronger in the ex-users’ stratum: former users had IQs an average of 2.75 points lower than controls, with reasonable evidence against a null effect (p = 0.035). It should be noted that this finding is based on a relatively small number of studies, so high susceptibility to Type I error may be inferred. In comparisons between current ecstasy users and polydrug controls, the difference between groups was very nearly zero.

FIGURE 18.

IQ (National Adult Reading Test) – ecstasy users versus polydrug controls: random-effects meta-analysis.

Sensitivity analysis with single, pooled comparisons for each study suggests that our primary analysis may very slightly underestimate the discrepancy between cohorts (by less than 0.1 of an IQ point: MD 0.418; 95% CI – 1.614 to 0.778). As might be expected, the evidence of a difference between cohorts is equally weak in this analysis (p = 0.493).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.862), and the funnel plot (not shown) had an unremarkable appearance.

In total, sufficient data were available to attempt metaregression analyses for 16 covariates; details are shown in Table 11. It should be noted that, in other analyses in this review, we have used baseline intelligence measures as an explanatory variable in metaregression analyses. In this instance, where the intelligence of study participants is the response variable of interest, these covariates have been excluded.

There was no evidence of a dose–response effect: cohorts with high exposure to ecstasy were no more disadvantaged against controls than those who had consumed comparatively little (see Figure 94 in Appendix 7).

Figure 19 compares mean difference in IQ with asymmetry in the amount of alcohol exposure between study arms. It should be noted that this analysis generates a positive coefficient, suggesting that those studies in which higher IQs were found in the ecstasy-exposed participants were those in which ecstasy users drank more alcohol than controls. However, because this dataset shows a reasonable balance across the spectrum of imbalance of alcohol exposure, this variable has little influence on the estimated average effect of ecstasy exposure: the adjusted mean difference is less than 0.1 IQ points greater (– 0.506; 95% CI – 1.540 to 0.529), and just as consistent with a null difference (p = 0.338).

FIGURE 19.

IQ (National Adult Reading Test) – Ecstasy users versus polydrug controls: mean difference in IQ against inter-arm asymmetry in exposure to alcohol (standardised mean difference).

IQ (National Adult Reading Test) – MDMA users versus drug-naïve controls

As in the comparison with polydrug controls, we included studies using foreign-language translations of the test.

Figure 20 shows our random-effects meta-analysis of these data. It shows an extremely similar picture to that seen in the comparison with polydrug controls. Ecstasy-exposed individuals’ IQs rated an average of 0.474 points lower than those of polydrug controls, but this is unlikely to represent a significant difference (p = 0.417). Again, ex-users appear more disadvantaged than current users although, in this case, the former users’ stratum is even more underpowered (comprising only two datapoints). In the comparison between current ecstasy users and controls, a non-significant average difference of less than 0.4 IQ points was seen.

FIGURE 20.

IQ (National Adult Reading Test) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

Sensitivity analysis with single, pooled comparisons for each study generated results very similar to those of the primary analysis (MD – 0.491; 95% CI – 1.755 to 0.772; null effect p = 0.446).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.992), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for nine covariates; details are shown in Table 12. Once more, we excluded intelligence measures as explanatory variables. None of the analyses were able to provide a statistically convincing explanation of the heterogeneity seen amongst base-case effect estimates. There was no evidence of a dose–response effect (see Figure 95, in Appendix 7).

Ecstasy users compared to polydrug controls

In 12 of 16 domains analysed, a significant effect of ecstasy exposure was seen (p < 0.05 against the null hypothesis of no exposure effect). Estimated effect sizes ranged from 0.143 to 0.509, with most estimates falling between 0.15 and 0.4. According to Cohen’s rule of thumb,119 such differences can be considered to fall in the range of ‘small’ effects, with some approaching ‘medium’ effect sizes.

The only domain in which an effect greater than 0.5 SD was found was that of self-rated memory. This is based on a small sample of studies (n = 5) reporting a collection of subjective outcome measures, both factors that would tend to increase uncertainty in the finding. Self-rated measures of impulsivity and anxiety also suggested a comparatively pronounced effect.

Among objective measures, the largest effects were seen in the domains of working memory (SMD – 0.391; 95% CI – 0.589 to – 0.192), delayed verbal memory (SMD – 0.377; 95% CI – 0.498 to – 0.257) and immediate verbal memory (SMD – 0.332; 95% CI – 0.451 to – 0.214). For the outcomes we have categorised as relating to attention, we identified a significant inter-population difference in the ‘focus–execute’ component, but not for the ‘sustain’ component. Amongst our executive function meta-outcomes, an exposure effect was seen for the ‘planning’ component, but not for ‘response inhibition’ or ‘shifting’.

Ecstasy users compared to drug-naïve controls

Eight of 12 domains analysed suggested a significant effect of ecstasy exposure, with estimated effect sizes ranging from 0.272 to 1.037. As in the polydrug-controlled comparisons, self-rated measures generated some of the most sizeable effect estimates, while the largest effects in objective measures were seen in the domains of immediate verbal memory (SMD – 0.840; 95% CI – 0.990 to – 0.690) and delayed verbal memory (SMD – 1.037; 95% CI – 1.734 to – 0.341).

Verbal memory (immediate) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 100 datapoints, representing a total of 40 pairwise comparisons, drawn from 27 different studies (35 comparisons from 27 studies providing data for current ecstasy users and five comparisons from five studies providing data for former ecstasy users). For data published in multiple studies originating from Liverpool John Moores University, data from a single publication120 only were included in this analysis, because it was not possible to deduce the extent of duplicate reporting across the full range of papers. In total, 46 different outcome measures are included, the most common being RBMT: prose recall (10 datapoints), RAVLT: sum of trials 1–5 (10 datapoints) and digit span – backwards (five datapoints). The complete dataset is detailed in Table 51, in Appendix 6.

The meta-analysis (Figure 21) suggests that ecstasy-exposed cohorts tended to perform worse than polydrug controls by around one-third of a standard deviation, with strong evidence against the null hypothesis of no difference between groups (p < 0.001). The stratified analysis identified no difference in exposure effect between current and former ecstasy users.

FIGURE 21.

Verbal memory – immediate (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

To contextualise the magnitude of this difference, we note that, in the Zakzanis et al. study,101 a standardised mean difference of precisely – 0.332 SD was seen between arms as a result of current ecstasy users scoring 0.2 less than controls on the RBMT immediate prose recall test (1.5 versus 1.7; scaled scores).

A sensitivity analysis in which all individual arms were aggregated to provide single, study-level estimates of effect for each outcome measure before meta-analysis revealed a very similar result (SMD – 0.339; 95% CI – 0.444 to – 0.234). This suggests that our primary analysis is robust to the assumptions underpinning the pooling of data.

There is little evidence of small-study bias, as indicated by Egger’s test (p = 0.330); similarly, the funnel plot (Figure 22) shows no clear trend, although there may be a slight tendency for the least precise studies to produce the most extreme effect estimates.

FIGURE 22.

Verbal memory – immediate (composite measure) – ecstasy users versus polydrug controls: funnel plot.

Sufficient data were available to attempt metaregression analyses for 17 covariates; details are shown in Table 15. There was no evidence of a dose–response effect (see Figure 96 in Appendix 7).

Figure 23 plots estimated effect size against average education level, showing that there is a tendency for differences in performance to diminish as education level increases. Notably, the four comparisons in this dataset estimating the greatest deficit for ecstasy users are also those in which participants have the lowest average education levels. If this model were to be believed, one would not expect to see a difference between cohorts if it could be assumed that a study’s participants had received around 15½ years of education. However, the dataset is a restricted one: only 12 of the 40 pairwise comparisons available in the full meta-analysis provide covariate data (although the estimated effect size in this subgroup is comparable to that seen in the full analysis: SMD – 0.371; 95% CI – 0.659 to – 0.083).

FIGURE 23.

Verbal memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against education of participants (average value across all cohorts).

Figure 24 plots estimated effect size against inter-arm asymmetry in intelligence. The fact that most comparisons are located in the ‘south-west’ quadrant of the plot shows that, in the majority of studies, ecstasy-exposed participants not only performed worse in the memory tasks but also were less intelligent than controls. Conversely, the effect size is smaller (indeed, in several cases suggesting an advantage for the ecstasy users), when intelligence measures favour those cohorts. The regression analysis suggests that there may be a general trend for worse performance in those studies in which ecstasy users had lower intelligence scores than controls. However, even if this model is to be believed, asymmetry in intelligence does not explain differences between cohorts entirely, and the evidence for worse performance in ecstasy-exposed cohorts remains strong (adjusted effect estimate: SMD – 0.240; 95% CI – 0.384 to – 0.096; p = 0.003).

FIGURE 24.

Verbal memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in baseline intelligence measures (standardised mean difference).

Verbal memory (immediate) – MDMA users versus drug-naïve controls

The dataset assembled for this measure comprises 41 datapoints, representing a total of 18 pairwise comparisons, drawn from 12 different studies (14 comparisons from 12 studies providing data for current ecstasy users and four comparisons from four studies providing data for former ecstasy users). Twenty different outcome measures are included, the most common being RBMT: prose recall (seven datapoints), digit span – backwards (six datapoints) and RAVLT: sum of trials 1–5 (five datapoints). The complete dataset is detailed in Table 52 in Appendix 6.

When this dataset was meta-analysed (Figure 25), both current and former ecstasy users tended to perform worse than drug-naïve controls by around 0.8 of a standard deviation, with strong evidence against the null hypothesis of no difference between groups (p < 0.001). According to Cohen’s rule of thumb, this would qualify as a ‘large’ inter-population difference. To give an indication of the magnitude of this standardised difference in real terms, the datapoint from Morgan’s 1999 study102 appears relatively typical of the pattern of results seen here. In this study, current ecstasy users recalled an average of 1.95 fewer items than drug-naïve controls in the immediate prose recall task of the RBMT (SMD – 0.852).

FIGURE 25.

Verbal memory – immediate (composite measure) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

The stratified analysis identified no difference in exposure effect between current and former ecstasy users.

Sensitivity analysis with aggregated comparisons for each study suggested that our primary analysis may underestimate the difference between populations by around 0.1 SD [revised SMD – 0.959; 95% CI – 1.285 to – 0.633; p(null SMD) < 0.001].

There is some evidence of small-study bias (Egger’s p = 0.023). The funnel plot for this dataset (Figure 26) shows that the four estimates with the highest precision provide a smaller-than-average estimate of exposure effect and, conversely, that those datapoints suggesting greatest difference between cohorts tend to be amongst those that are subject to the greatest uncertainty. Accordingly, one might conclude that, had every relevant test ever undertaken been available to this meta-analysis, the estimated exposure effect may have been somewhat lower.

FIGURE 26.

Verbal memory – immediate (composite measure) – ecstasy users versus drug-naïve controls: funnel plot.

Sufficient data were available to attempt metaregression analyses for 10 covariates; details are shown in Table 16. None of the metaregressions generated results that achieved or approached conventional levels of significance, and there was no evidence of a dose–response effect (see Figure 97 in Appendix 7).

Verbal memory (delayed) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 49 datapoints, representing a total of 32 pairwise comparisons, drawn from 22 different studies (27 comparisons from 22 studies providing data for current ecstasy users and five comparisons from five studies providing data for former ecstasy users). Twenty-two different outcome measures are included, the most common being RBMT: prose recall (10 datapoints), RAVLT: trial 8 (seven datapoints) and Buschke: overall score (four datapoints). The complete dataset is detailed in Table 53 in Appendix 6.

The meta-analysis, shown in Figure 27, suggests that ecstasy-exposed individuals’ long-term verbal memory is worse than that of polydrug controls by a little under 0.4 SD. According to Cohen’s guidelines, this would probably be thought of as somewhere between a ‘small’ and a ‘medium’ difference. The effect might appear to be greater in former ecstasy users, whom controls outperformed by almost 0.5 SD (a ‘medium’ difference, according to Cohen). However, there is insufficient evidence to reject a null hypothesis of homogeneous strata (p = 0.533). Sensitivity analysis with single, pooled comparisons for each study provides a SMD estimated at –0.402 [95% CI – 0.515 to – 0.288; p(null SMD) < 0.001], which is extremely close to the primary analysis.

FIGURE 27.

Verbal memory – delayed (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

To translate these findings back into a more easily interpretable scale, it may be useful to return to the raw data on which the analysis was based, to see which individual datapoints are closest to the calculated average. For the comparison between current users and controls, a relatively typical datapoint is the WMS-III delayed memory index score from the study by Groth et al. ,126 in which the ecstasy-using cohort registered lower scores than polydrug controls by an average of 3.8 points (108.4 versus 112.2; SMD – 0.356). Where former users were compared to controls, the most representative datapoint was that from Curran and Verheyden,104 where the difference between cohorts was 1.69 items on the RBMT delayed prose recall test (5.825 versus 7.515; SMD – 0.506).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.254), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 15 covariates; details are shown in Table 15. There was no evidence of a dose–response effect (see Figure 98 in Appendix 7).

Figure 28 plots estimated effect size against average education level, showing that there is a tendency for differences in performance to diminish as education level rises. This is a very similar picture to that seen for immediate verbal memory (see Figure 23). It should be noted, however, that both analyses are based on fairly restricted datasets.

FIGURE 28.

Verbal memory – delayed (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against education of participants (average value across all cohorts).

Verbal memory (delayed) – MDMA users versus drug-naïve controls

The dataset assembled for this measure comprises 28 datapoints, representing a total of 20 pairwise comparisons, drawn from 12 different studies (15 comparisons from 12 studies providing data for current ecstasy users and five comparisons from four studies providing data for former ecstasy users). Fifteen different outcome measures are included, the most common being RBMT: prose recall (seven datapoints), prose retained (three datapoints) and prose recall (three datapoints). The complete dataset is detailed in Table 54 in Appendix 6.

In the meta-analysis (Figure 29), ecstasy-exposed individuals’ delayed verbal memory is estimated to be inferior to that of drug-naïve controls by very nearly 1 SD. However, the forest plot shows very clearly that one effect estimate – that from Yip and Lee’s study128 – is entirely atypical of results from other studies. If this single datapoint is excluded from the meta-analysis, the estimated SMD falls to – 0.717 (95% CI – 0.915 to – 0.518); however, the evidence for an overall exposure effect remains strong (p < 0.001).

FIGURE 29.

Verbal memory – delayed (composite measure) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

Yip and Lee’s anomalous datapoint represents a composite of two subtests from the RAVLT, in both of which the performance of ecstasy-exposed participants was less than half the standard achieved by drug-naïve controls. There are a number of possible explanations for this extreme result. First, it should be noted that the outlying datapoints are those based on the Chinese version of the RAVLT; this is the only study in the evidence-base to rely on this instrument, the validity and characteristics of which are unclear to us. Second, it is possible that there are environmental and/or genetic factors that make ecstasy exposure effects unusual – or, at least, difficult to generalise to a UK context – in a Hong Kong Chinese population. Third, the authors’ description of the population from which their cohorts were drawn implies that Hong Kong clubbers use ecstasy and other drugs in a markedly different way to the patterns seen elsewhere. They claim to have recruited relatively uncontaminated ecstasy-using and control cohorts, excluding participants with exposure to other substances, including tobacco and alcohol (more than one drink per week). The fact that nearly two-thirds of potential participants were excluded for violating these criteria would tend to enhance such claims; most other included studies had broad eligibility rules, and appear to have included most or all prospective participants. Accordingly, it could be argued that – although it remains subject to all the limitations of the observational paradigm – Yip and Lee’s study overcomes some of the confounding seen in other research, with exposure to ecstasy providing the only clearly observable difference between cohorts. Nevertheless, it would be a substantial step to extend this argument to the suggestion that Yip and Lee’s estimate provides a ‘true’ exposure effect, while the additional confounding inherent in other studies serves drastically to underestimate the real difference.

Unsurprisingly, this outlying estimate has a substantial effect on calculated heterogeneity statistics. With Yip and Lee’s data included, tests reveal an extremely heterogeneous dataset (p < 0.001; I² = 96.0%), whereas reanalysis without the anomalous estimate reveals a picture that suggests a much more homogeneous dataset (p = 0.047; I² = 39.6%). Similarly, initial tests are strongly suggestive of interstratum heterogeneity (p = 0.003) but, on closer inspection, it becomes clear that this result is driven entirely by the single atypical estimate from Yip and Lee’s study: the reanalysis without this datapoint is wholly consistent with a homogeneous effect across strata (p = 0.595).

Sensitivity analysis with single, pooled comparisons for each study provides a mean difference estimated at –1.253 (95% CI – 1.936 to – 0.571). This may appear to be a relatively substantial discrepancy from the primary analysis; however, further analysis reveals that this is because the aggregated approach is affected to an even greater extent by Yip and Lee’s outlying estimate (without this datapoint, the sensitivity analysis generates a pooled estimate of – 0.745; 95% CI – 0.991 to – 0.499, which is close to the primary analysis).

Returning to the raw data on which the analysis was based, the individual datapoints that are closest to the calculated averages are – for the full dataset including Yip and Lee – the RBMT prose recall subscore reported by Dafters and colleagues75 [in which heavy ecstasy users scored an average of 1.85 less than controls (SMD – 0.979)], and – for the restricted dataset without the outlying estimate – the delayed (trial 8) RAVLT recall score from Lamers et al. 98 [in which the deficit for ecstasy users is estimated at 1.5 items (SMD – 0.701)].

When applied to the full dataset, Egger’s test suggested that there was no evidence of small-study bias in this dataset (p = 0.578). Once more, however, this result is substantially affected by the single outlying estimate: if Yip and Lee’s data are excluded, then Egger’s test returns a p-value of 0.021, suggesting that the null hypothesis of no small-study effect is difficult to support. The trend for more precise studies to estimate a smaller difference can be clearly visualised in the funnel plot for this dataset (Figure 30), as can the distorting influence of Yip and Lee’s study.

FIGURE 30.

Verbal memory – delayed (composite measure) – ecstasy users versus drug-naïve controls: funnel plot.

Sufficient data were available to attempt metaregression analyses for 13 covariates (Table 18); none provided significant results, and there was no evidence of a dose–response effect (see Figure 99 in Appendix 7).

Visual memory (immediate) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 66 datapoints, representing a total of 22 pairwise comparisons, drawn from 16 different studies (19 comparisons from 16 studies providing data for current ecstasy users and three comparisons from three studies providing data for former ecstasy users). Forty-one different outcome measures are included, the most common being Corsi Block: span (six datapoints), Corsi Block: span plus one (five datapoints) and WMS-R: visual reproduction (four datapoints). The complete dataset is detailed in Table 55 in Appendix 6.

The meta-analysis (Figure 31) suggests that ecstasy-exposed cohorts performed worse than controls by a small, but nonetheless significant, margin. Sensitivity analysis using the aggregated data approach generated very similar results [SMD – 0.126; 95% CI – 0.233 to – 0.020; p(null SMD) = 0.020]. The inter-population difference appears to be even smaller in the former-ecstasy-using stratum; however, the hypothesis test for interstratum heterogeneity provides no statistical justification for supposing the participants belong to different distributions.

FIGURE 31.

Visual memory – immediate (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

The small magnitude of this standardised difference becomes apparent when one compares the pooled estimate with the raw data on which the meta-analysis is based. For example, in Bolla et al. ,93 ecstasy users scored an average of 0.2 less than controls in WMS-R figural memory (7.3 versus 7.5; SMD – 0.166) and, in the spatial recognition task in Fox et al. ,130 there was an additional response latency of 110 milliseconds in the ecstasy-exposed cohort (2.4 seconds versus 2.29 seconds; SMD – 0.168).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.523), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 17 covariates; details are shown in Table 19. There was no evidence of a dose–response effect (see Figure 100 in Appendix 7).

Figure 32 plots memory performance against average age (classical metaregression, with covariate measured across all participants). It appears that the most marked deficit for ecstasy users may be found when populations with lower average age are assessed (it is notable that the eight lowest effect estimates appear amongst the youngest cohorts). In contrast, inter-arm differences, apparently, tend to be minimal in older cohorts.

FIGURE 32.

Visual memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against average age (all participants).

There may also be a gender effect in evidence: Figure 33 plots the outcome of interest against the gender composition of the populations under analysis (classical metaregression, with covariate measured across all participants). It shows that deficits were greatest in ecstasy-using cohorts that were predominantly made up of men. It is noticeable that the two datapoints contributed by comparisons of 100% male populations are those suggesting the greatest underperformance in ecstasy users.

FIGURE 33.

Visual memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against gender (across all participants).

For differential covariates, a very strong positive correlation was found between immediate visual memory outcomes and baseline asymmetry in intelligence, suggesting that good performance in these tests can be expected wherever one cohort has an advantage over the other in intelligence. This relationship is clear in Figure 34, which plots the variables against each other. It can be seen that datapoints representing worst performance in ecstasy users tend to be those in which they were less intelligent than controls whereas, in studies in which ecstasy users were more intelligent than controls, they could be expected to match or outperform controls in the memory tests. We note that a similar – albeit slightly less compelling – picture was seen in the equivalent metaregression for the analogous measure of verbal memory (see Figure 24).

FIGURE 34.

Visual memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in baseline intelligence measures.

This model suggests that the small exposure effect seen in the primary analysis is ascribable entirely to baseline imbalances in intelligence: when accounting for this confounding, the adjusted SMD is estimated at – 0.028 (95%CI – 0.148 to 0.092), which is consistent with a null effect (p = 0.623). This can be clearly seen in Figure 34, because the metaregression line passes almost directly through the origin of the graph.

In addition to the absolute effect of age (see Figure 32), inter-population asymmetry in age may also have an effect on observed results. Figure 35 shows that this effect has a negative coefficient, suggesting that worse performance by ecstasy-exposed cohorts is seen when they are older than their control groups.

FIGURE 35.

Visual memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in age.

Figure 36 shows the effect of differential amphetamine exposure on observed results. Although there appears to be a trend associating poorer performance with increased exposure asymmetry (i.e. ecstasy users taking more amphetamines than controls), it should be noted that a single datapoint is exerting considerable leverage on this analysis. The bubble on the left-hand side of the plot represents the study by Roiser et al. ,118 in which there was substantially greater exposure to amphetamines in the control group than in the current ecstasy users. If this single datapoint is excluded from the evidence-base, the apparent association with outcome disappears entirely (p = 0.379).

FIGURE 36.

Visual memory – immediate (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in exposure to amphetamines other than MDMA.

Visual memory (immediate) – MDMA users versus drug-naïve controls

The dataset assembled for this measure comprises 25 datapoints, representing a total of seven pairwise comparisons, drawn from six different studies (six comparisons from six studies providing data for current ecstasy users and one comparison from one study providing data for former ecstasy users). Seventeen different outcome measures are included, the most common being PRM: latency (two datapoints), PRM: correct (two datapoints) and CANTAB DMTS: simultaneous–latency (two datapoints). The complete dataset is detailed in Table 56 in Appendix 6.

When meta-analysed (Figure 37), these data suggest that there is little evidence of an exposure effect in this area. The effect estimate is noticeably similar to that seen in the comparison with polydrug controls (see Figure 31) but, in this instance, the analysis is based on a smaller dataset, and is subject to greater uncertainty. Sensitivity analysis using the aggregated data approach did not produced markedly different findings [SMD – 0.132; 95% CI – 0.294 to 0.029; p(null SMD) = 0.107].

FIGURE 37.

Visual memory – immediate (composite measure) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.921), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for six covariates; details are shown in Table 20. None of the analyses were able to provide a statistically convincing explanation of the heterogeneity seen amongst base-case effect estimates. Both covariates relating to gender distribution generated p-values approaching 0.05; however, little credence can be given to these findings, in the context of multiple testing with very limited (n = 6) datasets. There was no evidence of a dose–response effect (see Figure 101 in Appendix 7).

Visual memory (delayed) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 22 datapoints, representing a total of 14 pairwise comparisons, drawn from 10 different studies (12 comparisons from 10 studies providing data for current ecstasy users and two comparisons from two studies providing data for former ecstasy users). Ten different outcome measures are included, the most common being WMS-R: visual reproduction (five datapoints), R-OCFT: total score (four datapoints) and WMS-R: visual paired associates (two datapoints). The complete dataset is detailed in Table 57 in Appendix 6.

When meta-analysed (Figure 38), these data are strongly reminiscent of the immediate visual memory findings discussed above (see Figure 31), with ecstasy-exposed individuals apparently subject to a small but significant deficit in performance. Once more, sensitivity analysis using the aggregated data approach generated extremely similar results [SMD – 0.186; 95% CI – 0.325 to – 0.047; p(null SMD) = 0.009].

FIGURE 38.

Visual memory – delayed (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

A typical datapoint feeding this analysis is found in Reneman et al. ,93 in which the WMS-R visual reproduction score was a single point lower in the ecstasy-exposed arm than in the controls (35.4 versus 36.4; SMD – 0.187).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.173), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for nine covariates; details are shown in Table 21. None of the analyses were able to provide a statistically convincing explanation of the heterogeneity seen amongst base-case effect estimates, and there was no evidence of a dose–response effect (see Figure 102 in Appendix 7).

Visual memory (delayed) – MDMA users versus drug-naïve controls

The dataset assembled for this measure comprises 11 datapoints, representing a total of seven pairwise comparisons, drawn from five different studies (five comparisons from five studies providing data for current ecstasy users and two comparisons from two studies providing data for former ecstasy users). Six different outcome measures are included, the most common being R-OCFT: total score (three datapoints), PRM: correct (two datapoints), and PRM: latency (two datapoints). The complete dataset is detailed in Table 58 in Appendix 6.

Meta-analysis (Figure 39) suggests that, although ecstasy exposure is associated with worse performance in the majority of cases, pooled results do not provide convincing evidence against the null hypothesis of no exposure effect. This is true of the two strata individually and of the overall pooled estimate.

FIGURE 39.

Visual memory – delayed (composite measure) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

As in the analogous measure of delayed verbal memory (see Figure 29), the forest plot shows that the effect estimate from Yip and Lee’s study128 is markedly atypical of results from other studies. If this single datapoint is excluded from the meta-analysis, results become much more suggestive of a homogeneous dataset (Q = 5.68; p on 6 df = 0.460; I² = 0.0%). Without Yip and Lee’s study,128 the estimated SMD falls somewhat to – 0.191 but, because the heterogeneity term in the random-effects model is much reduced, the estimate appears rather more precise (95% CI – 0.423 to 0.041). The evidence for an overall exposure effect remains weak (p = 0.460).

Our initial sensitivity analysis, adopting single, aggregated comparisons for each study, generated an SMD estimated at – 0.520 (95% CI – 1.239 to 0.198), which is noticeably higher than that seen in the primary analysis. However, as previously, this discrepancy appears to be an artefact of the distortions of Yip and Lee’s study: repeated sensitivity analysis excluding the outlier is closely comparable to the primary analysis using the restricted dataset [SMD – 0.234; 95% CI – 0.605 to 0.137; p(null SMD) = 0.216].

When applied to the full dataset, Egger’s test suggested that evidence of small-study bias approached significance (p = 0.053). However, Yip and Lee’s study is exerting considerable leverage in this analysis; reanalysis with the datapoint excluded is much more suggestive of an unbiased dataset (p = 0.338). The funnel plot for this analysis (Figure 40) is unlikely to cause concern about publication bias, though it reinforces the outlying nature of Yip and Lee’s effect estimate.

FIGURE 40.

Visual memory – delayed (composite measure) – ecstasy users versus drug-naïve controls: funnel plot.

Sufficient data were available to attempt metaregression analyses for seven covariates; details are shown in Table 22. None of the analyses were able to provide a statistically convincing explanation of the heterogeneity seen amongst base-case effect estimates, and there was no evidence of a dose–response effect (see Figure 103 in Appendix 7).

Working memory – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 47 datapoints, representing a total of 23 pairwise comparisons, drawn from 15 different studies (20 comparisons from 15 studies providing data for current ecstasy users and three comparisons from three studies providing data for former ecstasy users). Twenty-nine different outcome measures are included, the most common being computation span (three datapoints), spatial recall (three datapoints) and reading span (two datapoints). The complete dataset is detailed in Table 59 in Appendix 6.

When meta-analysed (Figure 41), these data reflect an inter-population difference of approximately 0.4 SD. This effect size approaches a ‘medium’-sized difference, according to Cohen’s rule of thumb. Sensitivity analysis with data pooled at study level produced a closely comparable result [SMD – 0.406; 95% CI – 0.587 to – 0.225; p(null SMD) < 0.001]. There is evidence of interstratum heterogeneity: former users performed less well, in comparison to controls, than current users. For current users, the average inter-arm difference was of the order of one-third of an SD, while ex-users’ scores showed an effect size approaching two-thirds of an SD.

FIGURE 41.

Working memory (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

A representative datapoint from the underlying dataset is found in the 2002 study by Morgan et al. ,103 in which the ecstasy-using cohort made an average of 0.675 more errors than controls in the serial sevens subtraction task (1.725 versus 1.05; SMD – 0.439).

There is no evidence of small-study bias (Egger’s p = 0.238), and the funnel plot for this dataset (not shown) showed no pronounced trend, although there was a cluster of more powerful studies around the null effect point.

Sufficient data were available to attempt metaregression analyses for 15 covariates, shown in Table 23. There was no evidence of a dose–response effect (see Figure 104 in Appendix 7).

TABLE 23 - Working memory (composite measure) – ecstasy users versus polydrug controls: univariate metaregression results

ETLD, estimated total lifetime dose; ETLE, estimated total lifetime exposure; SMD, standardised mean difference.

Covariate	n	Effect modification			Adjusted effect estimate
		β-coefficient	(95% CI)	p	SMD	(95% CI)	p
Average values across all participants
Age (years)	20	0.036	(–0.066 to 0.139)	0.490
Sex (% male)	18	0.244	(–0.975 to 1.464)	0.695
IQ	7	0.074	(–0.069 to 0.217)	0.312
Education (years)	8	–0.088	(–0.524 to 0.348)	0.692
Characteristics of ecstasy exposure
ETLD (tablets)	14	0.000	(–0.001 to 0.001)	0.908
ETLE (occasions)	< 5
Period since last consumption (days)	10	0.000	(–0.001 to 0.001)	0.790
Duration of ecstasy use (days)	17	0.000	(–0.001 to 0.000)	0.526
Frequency of ecstasy use (occasions/month)	< 5
Inter-arm differences
Age (years)	20	–0.053	(–0.185 to 0.078)	0.426	–0.342	(–0.588 to –0.096)	0.006
Sex (% male)	18	–1.922	(–3.550 to –0.294)	0.021	–0.239	(–0.465 to –0.013)	0.038
Baseline intelligence measures (SMD)	16	0.356	(–0.319 to 1.032)	0.301	–0.292	(–0.561 to –0.024)	0.033
Education (years)	8	0.365	(0.036–0.694)	0.030	–0.101	(–0.471 to 0.269)	0.592
Exposure to cannabis (ETLD)	< 5
Exposure to cannabis (SMD)	14	0.014	(–0.518 to 0.546)	0.959	–0.324	(–0.671 to 0.024)	0.068
Exposure to amphetamines (ETLD)	< 5
Exposure to amphetamines (SMD)	13	–0.082	(–1.233 to 1.069)	0.889	–0.220	(–1.010 to 0.571)	0.586
Exposure to cocaine (ETLD)	< 5
Exposure to cocaine (SMD)	8	–0.461	(–1.533 to 0.612)	0.400	0.041	(–0.948 to 1.030)	0.935
Exposure to alcohol (ETLD)	< 5
Exposure to alcohol (SMD)	9	0.383	(0.089–0.677)	0.011	–0.224	(–0.355 to –0.092)	0.001

Figure 42 plots working memory performance against inter-arm asymmetry in gender. The most immediately noticeable feature of the graph is the dense cluster of datapoints around the origin; this suggests that those studies that were well matched for gender tended to show no difference in working memory between arms. Otherwise, the preponderance of data appears in the ‘south-east’ quadrant of the graph, showing that, where ecstasy-using participants were more likely to be men than controls, they tended to record worse test scores.

FIGURE 42.

Working memory (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in gender.

The relationship between inter-arm asymmetry in education and the response variable is visualised in Figure 43. The positive coefficient suggests that, in the various tasks synthesised here, worse performance tends to be seen amongst those ecstasy-exposed cohorts who had also received less education, on average, than their respective controls. There was limited availability of covariate data so this analysis is based on a fairly small subset of the full dataset; however, if the model were to be accepted, it would entirely explain the inter-population difference that might otherwise be ascribed to exposure to ecstasy.

FIGURE 43.

Working memory (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in education.

Figure 44 plots inter-arm asymmetry in exposure to alcohol against working memory performance. It shows that, once more, greater exposure to alcohol appears to be associated with better relative performance in the ecstasy-exposed cohort. The adjusted effect estimate from this analysis is, at – 0.224, a fair amount lower than that calculated in the primary analysis; however, because the regression gradient is relatively shallow, the overall exposure effect remains significant (p = 0.001).

FIGURE 44.

Working memory (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in exposure to alcohol (standardised mean difference).

Working memory – MDMA users versus drug-naïve controls

The dataset assembled for this measure comprises 12 datapoints, representing a total of seven pairwise comparisons, drawn from five different studies (six comparisons from five studies providing data for current ecstasy users and one comparison from one study providing data for former ecstasy users). Ten different outcome measures are included, none of which is adopted in more than one study. The complete dataset is detailed in Table 62 in Appendix 6.

When meta-analysed (Figure 45), this dataset suggests an exposure effect in the order of 0.5 SD, which slightly exceeds that seen in the comparison with polydrug controls and would be classified as a ‘medium’-sized effect in Cohen’s schema.

FIGURE 45.

Working memory (composite measure) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.879), and the funnel plot (not shown) had an unremarkable appearance.

Because of the very small size of this dataset, it was only possible to perform metaregressions on four covariates; none was significant (Table 24), and there was no evidence of a dose–response effect (see Figure 105 in Appendix 7).

Attention (focus–execute) – MDMA users versus polydrug controls

The dataset assembled for this measure is the largest in this review. It comprises 119 datapoints, representing a total of 30 pairwise comparisons, drawn from 19 different studies (26 comparisons from 19 studies providing data for current ecstasy users and four comparisons from four studies providing data for former ecstasy users). In total, 49 different outcome measures are included, the most common being TMT: Part A – time (seven datapoints), TMT: Part B – time (seven datapoints) and Stroop: colour reading – time (six datapoints). The complete dataset is detailed in Table 61 in Appendix 6.

When synthesised in a random-effects meta-analysis (Figure 46), these data suggest that ecstasy-exposed populations tend to perform worse than polydrug controls by a little over 0.2 SD. This would be considered a ‘small’ inter-population difference, according to Cohen’s schema. There is no evidence of interstratum heterogeneity. Sensitivity analysis using study-level aggregated data produced similar results [SMD – 0.256; 95% CI – 0.360 to – 0.153; p(null SMD) < 0.001].

FIGURE 46.

Attention – focus–execute (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

To compare the pooled estimate with a typical datapoint from a well-known instrument from the underlying dataset, a good example would be the WAIS digit–symbol test reported by McCardle et al. ,100 in which current ecstasy users scored 2.01 points less than controls (64.06 versus 66.07; SMD – 0.205).

There is no evidence of small-study bias in this dataset (Egger’s p = 0.768), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 17 covariates; details are shown in Table 25. There was no evidence of a dose–response effect (see Figure 106 in Appendix 7).

Figure 47 depicts the influence of inter-arm asymmetry in age on the outcome of interest. It shows that, in studies in which ecstasy users were younger than controls, inter-population differences tended to be relatively slight but, where they were older, the exposure effect had a tendency to be larger. However, because this dataset is relatively well balanced on this variable, this gradient has no notable effect on the overall pooled effect estimate (the adjusted value is only 0.01 SD lower than the base-case estimate).

FIGURE 47.

Attention – focus-execute (composite measure) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in age.

Attention (focus–execute) – MDMA users versus drug-naïve controls

The dataset assembled for this measure comprises 36 datapoints, representing a total of 16 pairwise comparisons, drawn from 12 different studies (14 comparisons from 12 studies providing data for current ecstasy users and two comparisons from two studies providing data for former ecstasy users). A total of 21 different outcome measures are included, the most common being MFFT-20: total errors (six datapoints), MFFT-20: latency to first response (six datapoints) and TMT: Part B – errors (two datapoints). The complete dataset is detailed in Table 62 in Appendix 6.

Our random-effect meta-analysis of these data (Figure 48) suggests that ecstasy users tended to perform worse than controls by a little over one-quarter of an SD. This result is comparable to that seen in the comparison between ecstasy users and polydrug controls (see Figure 46). Sensitivity analysis using the aggregated data approach generated similar – though slightly more uncertain – results [SMD – 0.295; 95% CI – 0.538 to – 0.052; p(null SMD) = 0.017]. In the dataset on which this meta-analysis is based, the most typical datapoint is the nine-letter comparison speed task reported by Wareing et al. ,136 in which former ecstasy users achieved 0.8 fewer correct items than controls (11.7 versus 12.5; SMD – 0.288).

FIGURE 48.

Attention – focus–execute (composite measure) – ecstasy users versus drug-naïve controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.562); the funnel plot (not shown) suggests a slight trend towards lower exposure effects in higher-precision studies, but all datapoints appear within the expected range.

Sufficient data were available to attempt metaregression analyses for 13 covariates; details are shown in Table 26. None of the analyses were able to provide a statistically convincing explanation of the heterogeneity seen amongst base-case effect estimates, and there was no evidence of a dose–response effect (see Figure 107 in Appendix 7).

Attention (sustain) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 27 datapoints, representing a total of 11 pairwise comparisons, drawn from seven different studies (eight comparisons from seven studies providing data for current ecstasy users and three comparisons from three studies providing data for former ecstasy users). Sixteen different outcome measures are included, the most common being G/N-G: correct responses (four datapoints), visual scanning: non-critical trials – time (three datapoints) and visual scanning: critical trials – time (three datapoints). The complete dataset is detailed in Table 63 in Appendix 6.

Our random-effects meta-analysis of these data (Figure 49) suggests that there is essentially no difference between populations, with no evidence of interstratum heterogeneity. However, for the only occasion in this review, our sensitivity analysis with data aggregated at study level generated markedly different results from our primary analysis, with a significant negative exposure effect estimated [SMD – 0.157; 95% CI – 0.304 to – 0.009; p(null SMD) = 0.037]. This borderline-significant estimate of an exposure effect may represent a more accurate synthesis of the available data, although, even if it is preferred, it remains a very small difference.

FIGURE 49.

Attention – sustain (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

Although Egger’s test did achieve conventional levels of significance (p = 0.024), a positive coefficient is estimated by the test, which suggests that a greater negative exposure effect is associated with high precision estimates. This trend is clearly seen in the funnel plot for this dataset (Figure 50).

FIGURE 50.

Attention – sustain (composite measure) – ecstasy users versus polydrug controls: funnel plot.

Sufficient data were available to attempt metaregression analyses for 17 covariates; details are shown in Table 27. There was no evidence of a dose–response effect (see Figure 108 in Appendix 7). A significant coefficient was estimated for one covariate: inter-arm asymmetry in exposure to amphetamines other than MDMA. This analysis is plotted in Figure 51. A relatively polarised picture can be seen: where ecstasy users had taken fewer amphetamines than controls, their performance was superior, and the opposite is the case where amphetamine consumption was greater in the ecstasy-exposed arms. The adjusted estimate of effect size remains consistent with a null hypothesis of no exposure effect.

FIGURE 51.

Attention – sustain (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in exposure to amphetamines other than MDMA.

Although neither achieves a conventional level of statistical significance, two further metaregressions are worthy of note. First, the relationship between asymmetry in alcohol consumption and test performance appears to show quite a strong trend (Figure 52). As has been seen in other analyses, increased alcohol exposure appears to result in a lesser degree of underperformance in the ecstasy-exposed arms.

FIGURE 52.

Attention – sustain (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against inter-arm asymmetry in exposure to alcohol (standardised mean difference).

Second, Figure 53 shows the relationship between exposure effect and inter-arm imbalance in participant age. Although this metaregression did not reveal a statistically significant relationship, it is worth emphasising the strong similarity between this graph and the analogous analysis for attention focus–execute (see Figure 47). The coefficient estimated in that case suggests that, for every year by which ecstasy users were older than controls, the exposure effect can be expected to grow by 0.049 SD. For sustained attention, a coefficient of – 0.098 SD is estimated.

FIGURE 53.

Attention – sustain (composite measure) – ecstasy users versus polydrug controls: mean difference in score against inter-arm asymmetry in age.

Attention (sustain) – MDMA users versus drug-naïve controls

Only four studies in the evidence-base reported measures of sustained attention in comparisons between ecstasy users and drug-naïve controls,61,83,99,125 so we did not pursue extensive analysis of this dataset. When meta-analysed according to the model used in other analyses, these data generate a non-significant SMD of 0.159 [95% CI – 0.180 to 0.498; p(null SMD) = 0.358].

Executive function (planning) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 40 datapoints, representing a total of 11 pairwise comparisons, drawn from five different studies (10 comparisons from five studies providing data for current ecstasy users and one comparison from one study providing data for former ecstasy users). Fourteen different outcome measures are included, the most common being ToL: Planning time (five datapoints). The complete dataset is detailed in Table 64 in Appendix 6.

Random-effects meta-analysis (Figure 54) estimates a pooled effect size of under 0.2 SD, i.e. less than a ‘small’ difference, in Cohen’s schema. The dataset appears to be relatively homogeneous. Sensitivity analysis with data aggregated at study level generated a very similar result [SMD – 0.179; 95% CI – 0.497 to 0.140; p(null SMD) = 0.271].

FIGURE 54.

Executive function – planning (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.525), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 13 covariates; details are shown in Table 28. There was no evidence of a dose–response effect (see Figure 109 in Appendix 7).

Significant coefficients were estimated for two covariates: study-level average IQ (classical metaregression, Figure 55) and duration of abstinence from ecstasy (Figure 56). For the former, a clear shape is seen amongst the five datapoints, with lower average IQ associated with a larger disadvantage for ecstasy users in the outcomes of interest. However, it is easy to conclude that the neatness of the correlation is dependent on the small number of contributing datapoints. For the latter, a less visually arresting significant association is seen. It may be that, as this picture suggests, the ecstasy users that have been abstinent for the longest are those that perform least well in comparison to controls but, given the small sample and less-than-unequivocal p-value, a Type I error is a real danger, so a degree of scepticism is probably appropriate.

FIGURE 55.

Executive function – planning (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against IQ (across all participants).

FIGURE 56.

Executive function – planning (composite measure) – ecstasy users versus polydrug controls: standardised mean difference against duration of abstinence in ecstasy users.

Executive function (planning) – MDMA users versus drug-naïve controls

We were only able to identify two studies reporting the appropriate comparison for this outcome. 110,125 When meta-analysed according to the model used elsewhere in this review, a small, non-significant SMD of – 0.170 (95% CI – 0.484 to 0.144) was estimated.

Executive function (response inhibition) – MDMA users versus polydrug controls

The dataset assembled for this measure comprises 34 datapoints, representing a total of 21 pairwise comparisons, drawn from 13 different studies (18 comparisons from 13 studies providing data for current ecstasy users and three comparisons from three studies providing data for former ecstasy users). Eighteen different outcome measures are included, the most common being Stroop: interference – time (seven datapoints) and G/N-G: reaction time (four datapoints). The complete dataset is detailed in Table 65 in Appendix 6.

When synthesised in a random-effects meta-analysis (Figure 57), these data suggest that there is no difference between ecstasy users and polydrug controls in this domain. The estimated SMD of approximately 0.1 SD would be considered very small, even if the difference was certain enough to meet conventional levels of significance. Sensitivity analysis with study-level aggregate data reveals a similar picture [SMD – 0.090; 95% CI – 0.338 to 0.159; p(null SMD) = 0.480]. We note that one datapoint in the forest plot appears atypical: the comparison between female ecstasy users and controls in von Geusau et al. 132 Above all, this extreme value is driven by performance in the HvdM Eriksen–Flankers test, in which ecstasy users outperformed controls by 3.7 SD (99.3% correct versus 96.7% correct; it should be noted that although this may seem to be a relatively small discrepancy, both estimates are subject to very small variance, so the difference between them is strongly significant and, when standardised, it becomes substantial). If the entire comparison for the female subgroup in this investigation is removed from analysis, then the estimated overall difference between populations does become borderline-significant, although the effect size remains small [SMD – 0.172; 95% CI – 0.336 to – 0.008; p(effect = 0) = 0.040].

FIGURE 57.

Executive function – response inhibition (composite measure) – ecstasy users versus polydrug controls: random-effects meta-analysis.

There is no evidence of small-study bias in this dataset (Egger’s p = 0.381), and the funnel plot (not shown) had an unremarkable appearance.

Sufficient data were available to attempt metaregression analyses for 14 covariates, shown in Table 29. There was no evidence of a dose–response effect (see Figure 110 in Appendix 7).