Cognitive Impairment (CI) is common and associated with poor outcomes among substance-dependent patients. The Montreal Cognitive Assessment (MoCA), a screening tool for CI has demonstrated superiority over the routinely used Mini Mental State Exam (MMSE) in multiple clinical populations, yet is rarely used in addiction settings or studies. The current study describes the performance of the two instruments in a sample of Asian substance-dependent patients.
The MMSE and MoCA were administered to 104 drug or alcohol-dependent inpatients 8 days after inpatient detoxification to identify those with CI. Total scores and task errors on common cognitive domains, and the level of agreement (Kappa coefficient) between the two screening tools in terms of identified cases of CI using MMSE (score < 24) and MOCA (score < 26) were examined along with the acceptability of the MoCA.
CI was identified in 76% of patients using the MoCA (M = 22.7) and only 5.8% using the MMSE (M = 27.0). Kappa agreement was exceptionally poor (κ = 0.05) and significantly more task errors were observed on the MoCA relative to MMSE domains. The acceptability of the MoCA was generally high, despite a majority (85%) finding it to be at least somewhat demanding.
The MoCA detected far more cases of CI than the MMSE which may reflect its inclusion of items assessing executive functioning. Whilst validation studies are needed to determine its sensitivity and specificity with an Asian population, these initial findings provide further evidence of its promise as a more suitable screening tool for CI in substance-dependent patients.
Substance misuse, Cognitive impairment, Screening tools, MOCA, MMSE
Cognitive Impairment (CI), particularly in executive functioning and decision-making are now recognised as common features of substance dependence, exhibited in 50-80% of persons with alcohol use disorders [1]. A recent meta-analysis concluded that CI is widespread and detectable up to one year following abstinence in alcohol-dependent individuals [2], whilst 69% of polydrug users (including cocaine and opiates) have been found to exhibit global executive functioning impairment [3].
Most psychosocial treatments (e.g. cognitive behavioural therapy, motivational interviewing etc.) require intact cognitive functioning and so the effectiveness of these psychosocial interventions are compromised [4], leading to poorer outcomes [5,6]. Often these deficits aremild-to-moderate in severity and since verbal skills are generally preserved, they can be difficult for clinicians to detect [7]. Therefore effective screening tools are essential since early identification means interventions can be adapted to an individual's cognitive strengths and weaknesses.
The most established and widely used screening tool to detect CI worldwide is the Mini Mental State Examination (MMSE) [8], however this instrument has no items assessing executive functioning. A growing body of research has demonstrated its limitations in the identification of CI in multiple clinical populations e.g. multiple sclerosis, cancer, Parkinson's disease, dementia [9] and alcohol-dependence [10]. The Montreal Cognitive Assessment (MoCA) is an alternative screening tool for CI, published over a decade ago, and was designed to detect mild-to-moderate impairment. It contains items assessing executive functioning and has excellent sensitivity and good specificity [11], yet is rarely applied in substance use settings or used in research. Evidence of its superiority over the MMSE has been demonstrated with a range of clinical populations [12-16].
To date, however, only three studies have examined CI using the MoCA with a substance-dependent population. In the US, a MoCA validation study demonstrated its acceptable sensitivity (83%) and specificity (73%) for detecting CI in a sample of substance (predominantly alcohol-dependent), outpatients of whom 38% had a score in the impaired range [17]. In Europe, a Dutch study concluded that it has discriminatory power for diagnosing patients with alcohol-related CI including Korsakoff's syndrome [18]. Most recently, a French study reported that 68% of alcohol-dependent patients in a rehabilitation setting had a score within the impaired range after 1-2 weeks of abstinence [19]. Whilst there is now some evidence that the MoCA is a useful screening tool with alcohol-dependent patients, it is necessary to establish its utility with drug-dependent users, since services generally treat both substance disorders in the one setting. Furthermore, studies to date have been restricted to Western populations and where MoCA performance of some participants could have been compromised by recent alcohol or benzodiazepine use. In fact, in Asia the examination of CI in substance-dependent individuals more broadly is limited to a single Japanese [20] and a single Taiwanese study [21].
The current study aimed to determine the extent and nature of CI in Asian substance-dependent patients, examine agreement between the MMSE and MoCA, and examine the clinical utility of the MoCA. Based on the literature, it was hypothesised that the MoCA would detect more cases of CI than the MMSE, that agreement between the two screening measures would be poor, and that the MoCA would be acceptable to patients.
Participants were 104 adult inpatients treated between June 2012 and December 2013 at an AOD treatment service in Singapore. Inclusion criteria included a diagnosis of DSM-IV alcohol or drug dependence, ability to speak and write English (for those with a language other than English as their first or dominant language), aged 21-55 years, 7 days abstinence from all drugs (excluding nicotine). Those with a major psychiatric disorder (e.g. schizophrenia, Bipolar Affective Disorder), and patients prescribed medications that could impair cognitive functioning (e.g. an extended benzodiazepine regimen, anti-depressants, antihistamines) were excluded, as were those reporting a current or past history of head/brain injury (defined by a loss of consciousness for more than 5 minutes) or neurological illness.
The Montreal Cognitive Assessment (MoCA): This is a single-page instrument that assesses attention, abstraction, executive functioning, orientation, language, visuospatial and memory [11]. It takes approximately 10 minutes to administer and a score of < 26 out of 30 suggests cognitive impairment. One point is added if the examinee has less than 12 years of education. The English version of the MoCA was used the local Mandarin and Malay versions were not available at the time the study was conducted.
The Mini Mental State Examination (MMSE): This is a single-page instrument that assesses visuospatial skills, concentration, working memory, memory recall, language, and orientation [8]. It takes approximately 5-10 minutes to complete, and a score of < 24 out of 30 suggests cognitive impairment. The English version of the MMSE was used to permit a direct comparison with the MoCA.
Addiction Severity Index-Lite (ASI-Lite): The ASI-Lite assesses 7 domains (i.e., medical, employment/support, alcohol and drug use, legal, family/social and psychiatric issues) in the past 30 days [22]. The alcohol and drug composite scores (i.e., ACOMP and DCOMP) were used as measures of the alcohol and drug use severity.
Patients were administered the above instruments on days 8-14 of their inpatient stay (with the MMSE and MoCA counter-balanced to prevent order effects). Cognitive testing was delayed until day 8 when participants were no longer taking medications to manage withdrawal symptoms (e.g. diazepam) that could compromise their cognitive functioning. All participants were assessed by the same researcher. The clinical utility (acceptability) of the MoCA was assessed via qualitative assessment using two questions: "How demanding was this test overall?" and "How unpleasant was this test?", both of which were rated on a Likert scale ranging from 1 (not at all) to 5 (extremely). The study was approved by the Institutional Review Board and National Healthcare Group's Domain Specific Review Board (Ref: 329/2011).
The sample were predominantly drug-dependent patients, with 65.4% meeting DSM-IV criteria for opioid-dependence, 4.8% sedative-dependence (using benzodiazepines) and 1% hallucinogen-dependence (using ketamine), whilst 28.8% met criteria for alcohol-dependence. Participants were predominantly male (87.5%), with a mean age of 44.3 (SD = 9.2) years. 41.3% of the sample were Chinese, 27.9% Indian, 21.2% Malay, and 9.6% of other ethnicities. Almost one third (32.7%) were married, 38.5% were single and 28.8% were either divorced, separated or widowed) and 46.2% were employed. The mean duration of education was 9.0 (SD = 2.7) years (mode = 10 years), with 94.2% completing primary school and 19.2% completing secondary school Mean age of addiction onset was 19.8 (SD = 6.6) years and mean frequency of substance use was 28.8 days per month. The mean ASI alcohol composite score was 0.64, with 80.8% of patients falling in the high severity range and mean drug composite score was 0.25, with 75.7% of patients falling in the high severity range. MMSE scores were negatively skewed, whilst MoCA scores were approximately symmetric (Table 1 for descriptive statistics). Approximately three-quarters,76.0%, exhibited CI on the MoCA with a mean score of 22.7 (SD = 3.6) whilst only 5.8% exhibited CI on the MMSE with a mean score of 27.0 (SD = 2.1) (Figure 1). In addition, 5.8% exhibited CI on both screens and 24.0% on neither screens. Of the 79 (76.0%) failing the MoCA, 92.4% passed the MMSE and conversely, of the 6 (5.8%) failing the MMSE, none passed the MoCA. The Kappa measure of agreement was 0.05 (95% CI = 0.015-0.085) indicating a significantly low level of agreement between MMSE and MoCA for classifying cases as CI.
Figure 1: Box plot (median and interquartile range of the MMSE and MOCA scores with recommended cut-offs for cognitive impairment.
(Note: Dotted lines represent recommended cut-offs).
View Figure 1
Table 1: Summary of the descriptive data for MMSE and MoCA scores. View Table 1
Several differences emerged when comparing MoCA and MMSE task errors (Figure 2). Substantially more patients had errors on the MoCA than MMSE domains; 76.9% versus 8.7% on the visuospatial domain, 96.2% versus 53.8% on the language domain, 73.1% versus 34.6% on the delayed memory domain, and 82.7% had task errors on the three executive functioning items on the MOCA (not assessed in the MMSE). Whilst MoCA had a moderate significant correlation with years of education (r = 0.43, p < 0.001) and only a weak significant correlation with age (r = -0.29, p < 0.01), the MMSE only a moderate significant correlation with age (r = -0.33, p < 0.01) and no correlation with years of education (r = 0.15, p = 0.14). T-test revealed no significant differences in performance on either the MMSE or MoCA between patients undergoing alcohol withdrawal and patients undergoing withdrawal from heroin, ketamine or benzodiazepines (Table 2). Similarly, chi-square analyses revealed no significant differences in the proportion falling in the impaired range on the two screening tools (Table 2).
Figure 2: Dot plot of the proportion (95% CI) of task errors made on the MMSE and MoCA.
View Figure 2
Table 2: Differences in MMSE and MoCA performance between alcohol and drug patients. View Table 2
In terms of patient acceptability of the MOCA, 15.4% found it to be "not at all" demanding, 64.5% found it "somewhat" or "fairly" demanding, and 20.1% found it "rather" or "very" demanding; 71.2% found the MoCA "not at all" unpleasant, 25.9% found it "somewhat" or "fairly" unpleasant, and 2.9% found it "rather" unpleasant with no reports of it being "extremely unpleasant".
In support of our hypotheses, the agreement between both the MMSE and MOCA was very poor, with the MoCA identifying more cases of CI than the MMSE, and deemed acceptable to patients. Our findings suggest that with Asian substance-dependent individuals, the MoCA is likely to be superior to the widely-used MMSE in the detection of cognitive impairment. Whilst the administration of a gold-standard neuropsychological assessment is needed to validate the MoCA with this population, the 76% impairment indicated by the MoCA is much closer to the impairment rates reported in the literature (i.e., 50-80%) as compared to the 6% indicated by the MMSE, and is very close to the 68% impairment reported by Alarcon et al (2015). These findings support those of several MoCA validation studies conducted on a broad range of clinical populations [12-16] including the one study on substance-dependent patients [17].
The findings provide further evidence that the routinely-used screening tool (MMSE) is inappropriate for use with substance-dependent patients. The MMSE was designed to detect moderate-to-severe impairment in dementia patients and has remained the predominant screening tool across the globe since its publication in 1975. In contrast, the MoCA was validated in 2005 specifically to detect mild CI, and so lends itself for use with a substance-dependent population. Confirming earlier literature, patients showed the poorest performance on the following cognitive domains: language, visuospatial, working memory and executive functioning. The MoCA adopts a more detailed approach to assessing cognitive domains including higher-order (frontal) processes with tasks such as the clock-drawing test, the 'trails' test which assesses mental flexibility, tests of verbal fluency and abstraction. Whilst some domain items are directly comparable with those of the MMSE (e.g. orientation to time, date, and place), the increasing difficulty of some items (e.g. recalling 5 words on the MoCA instead of 3 words on the MMSE) make it better at discerning mild verbal memory deficits. The finding that participants made errors on more cognitive domains on the MoCA relative to the MMSE suggests the former is likely to be better at detecting global impairment. Additionally, the finding that MoCA total score correlated with years of education supports other research findings [19] as this relationship is well established with detailed neuropsychological assessment [23].
Although it is suggested that the MoCA takes a few additional minutes to administer [24], its ability to detect more cases of likely CI, coupled with the broadly positive rates of acceptability, suggests that it should be the preferred tool. The MMSE's observed ceiling effect means mild-to-moderate cognitive impairments, that could potentially influence treatment responses, have a high chance of being missed. It is likely that its operational cut-off values require adjustment from those recommended over four decades ago given that, over time, populations perform better as a result of increasing environmental stimulation, technology and better general health impacting on brain networks [25]. Detailed neuropsychological assessment is time and resource intensive and requires specialist training and therefore must be reserved for those with indicated CI. However, such indications are not reliably driven by clinical impression alone nor self-report assessments of cognitive functioning, which have been proven to be inadequate [26], hence effective screening tools are needed. Although the 76% of MoCA identified CI cases may be an overestimate, it approximates rates reported in the literature. Moreover, high sensitivity can be prioritised over high specificity when screening procedures require little in terms of administration time and cost (note, use of the MoCA is free). On the basis of these findings, we recommend the use of the MoCA over the MMSE for clinical research trials when CI is an exclusion criterion with substance-dependent populations.
Whilst the findings of the study add to a sparse literature on the MoCA as a useful screening tool with substance-dependent populations, there are a number of limitations to be considered. With respect to the generalisability of the findings, being restricted to an inpatient sample, the proportion of patients identified as cognitively impaired is likely to be inflated, given their greater addiction severity relative to outpatients. However, at the same time, with the exclusion of patients with schizophrenia and bipolar affective disorder, which are associated with CI in the absence of substance use disorders [27], the prevalence of CI is likely to be underestimated relative to a more heterogeneous/co-morbid SUD population [28]. Whilst the proportion exhibiting CI on the MoCA aligns with estimated rates of CI in the literature, it was not possible to demonstrate that this tool is psychometrically more robust than the MMSE.
Research is currently underway to establish the psychometric properties of the MoCA among Asian substance-dependent individuals by using a gold-standard assessment of CI. This includes recruitment of demographically-matched healthy controls to inform optimal cut-off scores after controlling for loss of points due to the fact that English is often a second language among patients in Asian treatment settings. Meanwhile, the current results provide further support for the MoCA as an effective screening tool and suggest that it should replace the MMSE so that appropriate assessment, referral and treatment modifications can be made to optimise clinical outcomes.
We would like to acknowledge the funders of the study; The Lee Foundation, as well as the NAMS clinical staff for the identification and referral of suitable patients for this study, and the patients who participated for their time and assistance.
This study was funded by The Lee Foundation; a private charitable organisation in Singapore. No funding grant number is listed for it.