COUC515 Research and Program Evaluation

Author Qualifications

Runsen Chen is part of Beijing Key Laboratory of Mental Disorders of the National Clinical Research Center for Mental Disorders as well as the Clinical Psychology Center, both are at Beijing Anding Hospital, a part of Capital Medical University in Beijing, China. Chen is also a member of the Department of Psychiatry at the University of Oxford in Oxford, United Kingdom.

Assessment Critique

The authors utilized the Platinum Standard which was specifically designed to calculate efficacy in EMDR research. As such it appears that it would provide selected studies that favor EMDR and is therefore inherently biased.

Implications

The authors recommend the upcoming studies clearly determine and document the subjects’ history of treatment. Accordingly, my research would document the participants history of treatment.

Reference #2

Khan, A. M., Dar, S., Ahmed, R., Bachu, R., Adnan, M., & Kotapati, V. P. (2018). Cognitive behavioral therapy versus eye movement desensitization and reprocessing in patients with post-traumatic stress disorder: Systematic review and meta-analysis of randomized clinical trials. Curēus (Palo Alto, CA), 10(9), e3250. doi:10.7759/cureus.3250

Summary

The aim of this article was to compare the effectiveness of CBT and EMDR treatments in alleviating the symptoms of PTSD. The article’s systematic review and analysis is done in according to PRISMA guidelines. In the search for studies to include in this meta-analysis the following criteria were observed: (1) all participants were adults, adolescents, or children; (2) all articles were in English; (3) the articles detail comparative results from RCTs of CBT and EMDR; (4) a PTSD diagnosis, according to DSM-V, -IV, or -III; and (5) the articles must have data that was adequate to compare for calculation. Khan et al. found 14 studies that qualified. An average age of just over 30 years was calculated for 675 participants in 12 studies, as two studies did not provide such information.

Throughout these 14 studies multiple assessment tools were utilized for PTSD symptom appraisal, one was: the PTSD reaction index (PTSD-RI). The results of merging 11 articles in this meta-analytic study showed that CBT was less effective that EMDR in lessening symptoms of PTSD – the results were statistically significant (p = 0.006). Interestingly three months after treatment, four articles displayed that EMDR was not better that CBT in reducing PTSD symptoms. Regarding the reduction of anxiety, which is critical in PTSD diagnosis, five articles showed the statistically significant result that EMDR is better at reducing symptoms of anxiety than CBT (p = 0.005).

Source Evaluation

My research to focus on the efficacy of EMDR versus other interventions in treating PTSD is directly related to this meta-analysis which provides RCT data regarding a competing treatment, CBT, as well as data on EMDR interventions.

Author Qualifications

Dr. Ali M. Khan is a psychiatry resident at the University of Texas Rio Grande Valley in Harlingen, Texas. Dr. Ali M. Khan presented this paper at 2019's APA National Conference in San Francisco California (UTRGV School of Medicine, 2019).

Assessment Critique

Several of the assessment tools are clinician administered or clinical interview (CAPS and SI-PTSD). Clinician report risks biasing the report toward the clinician’s theoretical approach, possibly even unknowingly biasing results towards clinician’s desired outcomes. Anytime information from one person is filtered through another individual there is risk of altered information.

Implications

A limitation discussed by the authors is that there are so few participants in the RCTs reviewed, and a larger population RCT with extended follow-up assessments are recommended. My study would pull qualified participants from a larger numbered pool, by including multiple agencies and different localities, and post-treatment follow-up assessments would occur at multiple and successively more extended intervals.

Reference #3

Nordahl, H. M., Halvorsen, J. Ø., Hjemdal, O., Ternava, M. R., & Wells, A. (2018). Metacognitive therapy vs. eye movement desensitization and reprocessing for posttraumatic stress disorder: Study protocol for a randomized superiority trial. Trials, 19(1), 1-8. doi:10.1186/s13063-017-2404-7

Summary

This article aims to explicitly outline what is needed for a two-armed randomized controlled superiority trial to address the following: the effectiveness of metacognitive therapy (MCT) versus EMDR for PTSD by implementing a comparative RCT; discovering the possible moderators of the treatments’ outcomes, as well as the mediators; and examine the data regarding an kind of relapse. The number of individuals needed to participate in order to find statistically significant results is 58. To be sure that participants are qualified, the Anxiety Disorders Interview Schedule (ADIS-IV) will be used; for other measures, the authors provide a list of eleven more assessment tools. The steps to be taken to ensure blind assessors, who will not be part of the treatment process, is described; as well as plans to promote the retention of participants throughout the length of the study are given. Treatments are to be structured and done according to published manuals and specific protocols, and treatment adherence is critical. Finally, the authors outline a plan to analyze the data gathered. One part of this plan is to determine primary outcome, which is a contiguous longitudinal measure utilizing the Posttraumatic Diagnostic Scale (PDS) that would be subject to a linear mixed-model analysis.

Source Evaluation

This is an excellent source for my study as it lays out in detail the steps for running a trial comparing EMDR treatment with another intervention, which is precisely my study. It provides the hypotheses, study design, the measures to be used, the timetable for follow-up checks, the treatments utilized, and even a plan on how to analyze the data collected. 

Author Qualifications

Dr. Nordahl is professor in behavioral medicine at the Norwegian University of Science and Technology (NTNU) and employed at the Department of Mental Health in the unit for Adult Psychiatry and Substance Abuse. Dr. Nordahl is also researcher at the Regional Research and Treatment unit for Trauma and PTSD at St. Olav Hospital in Trondheim, Norway (NTNU, n.d.).

Assessment Critique

This article suggests the use of the Beck Depression Inventory (BDI-II) as one of the measures to be used to help provide insight into how the participants are feeling. This is a widely used assessment tool, and for good reason as the test-retest reliability is very high at 0.93. Other measures the authors recommend utilizing have lower test-retest scores, anywhere from 0.56 to 0.83. This attests to the powerful and consistent nature of the BDI-II.

Implications

The study protocol described within this article is relevant to furthering research in the area of comparing the efficacy of various treatment paradigms. And it is of particular use to this researcher as it provides so much guidance and specific information regarding the details of setting up, completing, and analyzing the outcomes of my intended study of EMDR versus other interventions for PTSD. Additionally, creating a protocol of how to carry out similar studies will promote homogeneity among research and will allow for better meta-analyses of such studies.

Reference #4

de Roos, C., van der Oord, S., Zijlstra, B., Lucassen, S., Perrin, S., Emmelkamp, P., & de Jongh, A. (2017). Comparison of eye movement desensitization and reprocessing therapy, cognitive behavioral writing therapy, and wait-list in pediatric posttraumatic stress disorder following single-incident trauma: A multicenter randomized clinical trial. Journal of Child Psychology and Psychiatry, 58(11), 1219-1228. doi:10.1111/jcpp.12768

Summary

This is a multicenter, single-blind, parallel-group study with three arms of EMDR, cognitive behavioral writing therapy (CBWT), and wait-list (WL). Participant criteria were as follows: 8-18 years of age, communicate in Dutch, and have PTSD as primary diagnosis. There were 103 participants which met these criteria, and 99 of these participants completed treatment. The authors’ hypothesis for this study is that both CBT and EMDR would be better than any delayed treatment in reducing PTSD symptoms even gaining remission from this disorder.

A primary outcome measure utilized was the Revised Children’s Responses to Trauma Inventory (CRTI). EMDR and CBT treatments had manuals on which to base the interventions, and they were all conveyed in up to six weekly individual 45-minute sessions. Findings showed that EMDR interventions were markedly briefer than CBWT. There were significant pre-to-post treatment decreases in PTSD symptoms for participants treated with EMDR and CBWT versus those WL. At 3- and 12-month follow-up assessments the improvements were maintained, but EMDR showed further improvements in these assessments than CBWT. Subsequent analysis of the data collected showed that the changes in symptoms of PTSD were more likely because of the impacts of treatment versus any measurement error. Interestingly , remission rates were substantially higher for EMDR interventions compared to WL. The interventions of EMDR and CBWT were relatively efficacious in this study.

Source Evaluation

My topic of interest being learning the impacts of EMDR and other interventions for treating PTSD is perfectly aligned with this study, as this research showed strong efficacy of both EMDR and CBWT in the treatment of PTSD symptoms as the intensity and presence of symptoms decreased significantly.

Author Qualifications

Primary author, de Roos, is employed at the Psychotraumacenter for Children and Youth in Leiden, The Netherlands. She is a clinical psychologist, psychotherapist, child and youth psychologist specialist, psychotrauma therapist, EMDR trainer and supervisor, and supervisor for the Association for Children and Youth Psychotherapy (Unknown, n.d.).

Assessment Critique

 Implementing the Subjective Units of Distress Scale (SUDS) allows for risks and has at least one or more limitations. Miscommunication, as a theme, emphasizes the need for clarity when using SUDs with children and young people (Kiyimba & O’Reilly, 2017). Clearly communicating the information in the scale and understanding responses is critical.

Implications

The authors point out that, while the EMDR and CBWT treatments were manualized, it is critical to be sure that outcomes are standardized. So making my study certify similar levels of training of the qualified experts completing the assessments as well as the interventions would be required.

Reference #5

Tran, U. S., & Gregor, B. (2016). The relative efficacy of bona fide psychotherapies for post-traumatic stress disorder: A meta-analytical evaluation of randomized controlled trials. BMC Psychiatry, 16(1), 266. doi:10.1186/s12888-016-0979-2

Summary

This study is a meta-analytic approach to the review of 22 journal articles which met the criterion of: (1) being of RCT design; (2) contains a minimum of two bona fide psychotherapies; (3) therapies are conducted in at minimum two sessions; (4) study participants are adults; (5) participants are DSM-IV and -III diagnosed with PTSD; and (6) the severity of symptoms of PTSD are assessed with self-report or a clinician rating. In the reported 24 direct comparisons, 1694 participants were studied. Study bias was assessed based on the Practice Guidelines from the International Society for Traumatic Stress Studies’ criteria.

Treatments were categorized into trauma focused (TF) or not trauma focused (NTF) to determine which was more efficacious, while the study reviewed the effectiveness of various types of interventions within these groups, such as EMDR, CBT, exposure therapies (EX), and present-centered therapies (PCT). Primary outcomes were calculated by direct measurements of the severity of PTSD symptoms utilizing an assessment such as: Clinician Administered PTSD Scale (CAPS). According to primary and secondary outcomes meta-analysis, no treatment reviewed purported more efficacious results than other treatments. Trauma focused interventions were significantly more effective regarding primary outcomes. After running the Wampold Homogeneity Test WHT 30 times, “the null hypothesis of no differences in treatment efficacy had to be rejected” (Tran & Gregor, 2016, p. 10). Interestingly, exposure therapies along with prolonged exposure therapies showed to be more effective regarding the severity and recovery of PTSD symptoms.

Source Evaluation

This study is focused on reviewing articles which utilized various treatments to lessen PTSD symptoms and speaks precisely to my research. Understanding which assessment tools were utilized and how subsequent data was analyzed will provide guidance for my research.

Author Qualifications

Primary author, Tran, is Senior Lecturer in the Department of Basic Psychological Research and Research Methods Faculty of Psychology at the University of Vienna in Vienna, Austria. His extensive history of research and publication goes back to 2004 (Tran, 2018).

Assessment Critique

The authors raise concerns about utilizing the Wampold Homogeneity Test (WHT) as there are two qualities that have the potential to compromise its validity: (1) WHT’s “Q test is known for its low test power when k is small”, and (2) “the WHT relies on only one run of random sign assignment” (Tran & Gregor, 2016, p. 6). These aspects of the WHT and only running it once for a specific dataset may “capitalize on random fluctuations who possible effect is greater for small k” (Tran & Gregor, 2016, p. 6). 

Implications

The authors point out that “distinguishing different types of recovery and improvement might accentuate differences in efficacy, particularly because improvement is defined by study authors in different ways” (Tran & Gregor, 2016, p. 14). My study would attempt to research various recovery definitions utilized in previous RCT studies and employ the most popular one in my article as to allow for increased efficacy.

References

Chrastina, J. (2018, January 1). Meta-synthesis of qualitative studies: Background, methodology and applications. ResearchGate. https://www.researchgate.net/publication/327114468_META-SYNTHESIS_OF_QUALITATIVE_STUDIES_BACKGROUNDMETHODOLOGY_AND_APPLICATIONS 

Kiyimba, N., & O'Reilly, M. (2017, March 21). The clinical use of subjective units of distress scales (SUDs) in child mental health assessments: A thematic evaluation. CORE. https://core.ac.uk/reader/151305816 

Norwegian University of Science and Technology (NTNU). (n.d.). Hans M. Nordahl. Norwegian University of Science and Technology. Retrieved July 19, 2020, from https://www.ntnu.edu/employees/hans.nordahl 

Tran, U. S. (2018). Ulrich S. Tran Curriculum Vitae. Persönliche CMS. Retrieved July 18, 2020, from https://ppcms.univie.ac.at/uploads/media/CV.pdf 

University of Texas Rio Grande Valley (UTRGV) School of Medicine. (2019, July 10). Ali M. Khan, M.D., Psychiatry Resident. Facebook. Retrieved July 18, 2020, from https://www.facebook.com/UTRGVSOM/posts/we-would-like-to-congratulate-dr-ali-m-khan-on-winning-the-american-psychiatric-/2268101619892117/ 

Unknown. (2017, December 21). Gemma WILSON. ResearchGate. Retrieved July 18, 2020, from https://www.researchgate.net/profile/Gemma_Wilson7 

Unknown. (n.d.). Carlijn de Roos. Wat is EMDR - EMDR Kind & jeugd. Retrieved July 18, 2020, from https://www.emdrkindenjeugd.nl/wie-zijn-wij/carlijn-de-roos/