Green, G. (2010). Training practitioners to evaluate evidence about interventions. European Journal of Behavior Analysis, 10, 223-228.
Reviewed by
Elisabeth M. Kinney, MS, BCBA, Behavioral Learning and Leadership
and
Zachary Houston, MS, BCBA, Quality Behavioral Solutions, Inc
Since 1998, the Behavior Analyst Certification Board (BACB) has provided guidance to consumers of applied behavior analytic services in determining whether practitioners demonstrate at least minimal competence in behavior analysis (www.bacb.com). The BACB clearly informs consumers that certification only guarantees that practitioners demonstrate the minimal behavior analytic competencies required to be eligible for, and then pass the certification examination. However, over the past 13 years, the BACB has convened Expert Panels and distributed and reviewed job analysis surveys that have resulted in revisions to the eligibility standards, supervision requirements, and the examination itself. Such updates provide further guarantee that those with BACB certification are meeting the minimal competencies as they evolve and/or are better understood.
Green’s article plants a seed for consideration of additional practitioner competencies in the scientific evaluation of interventions and evidence, especially for those providing services to individuals with autism. It recommends that the BACB considers the skill of examining scientific evaluation as an additional practitioner competency when the BACB next convenes its Expert Panels and disseminates job analysis surveys.
Green suggests that the BACB requirements “be expanded to explicitly include competencies in basic scientific reasoning and critical thinking” (Green, 2010, p. 223) for those providing services to individuals with autism. In doing so, Green further substantiates the role of the BACB’s Guidelines for Responsible Conduct by supporting a minimal competency toward ensuring consumers receive effective interventions. Current guidelines regarding responsible conduct include 1.01 Reliance on Scientific Knowledge, 1.03 Professional Development, 2.10 Treatment Efficacy, and 9.02 Scientific Inquiry.
Green is now upping the ante in making the case for the BACB and its affiliated university programs to teach and require practitioners of behavior analysis to chart a clear scientific course for consumers of autism treatment. Green discusses an expansion of the BACB Task List to include two new content areas: (1) Basic scientific concepts and reasoning, and (2) Evaluating claims about interventions. What exactly is Green asking that practitioners learn?
Green proposes that skepticism and other aspects of scientific inquiry are critical components of not only experimental but also clinical work. Competency in this area requires training in identification of valid re-search methods, bias, researching hype, and traps like logical fallacies in which arguments of logic are inherently flawed. This skill set becomes especially important in the identification of good science versus potentially harmful pseudosciences.
Although of concern in most disciplines in which behavior analysis is applied, Green argues that for ASD practitioners the competencies are paramount given the history of ASD interventions “touted as effective ; on the basis of anecdotes, testimonials, speculations, and poor research,” and that “many have been put into widespread use before they were tested carefully, or at all” (Green, 2010, p. 224). Over 10 years ago, Green & Perry (1999) asked readers to discriminate between science, pseudoscience, and anti-science, considering “… research has shown that many currently available interventions for autism are ineffective, even harmful,” and “… every moment spent on one of those therapies instead of effective intervention is a moment lost forever” (p. 5).
As such, an apparent void exists in the BACB curriculum with regards to evaluation of evidence for autism treatment, as the responsibility for critical reasoning should be on the shoulders of clinicians as well as consumers at all levels. Scientific inquiry and evaluation are critical to educational staff, clinical consultants, family members and all other involved parties. This is a tremendous responsibility. In the past decade much has been done to alleviate the burden on the consumer in scientifically evaluating interventions and evidence for autism treatment. For example, the following resources are helpful when considering the evidence for/against a specific treatment:
- The National Academies Press outlined their criteria for the designation of “Model Program,” in determining 10 representative models “in which empirically demonstrated strategies for addressing specific problems were gradually packaged as components of overall comprehensive models.” (http://www.nap.edu/openbook.php?record_id=10017&page=141).
- The Organization for Autism Research (OAR) created “a basic primer on autism research for use as a reference by parents,” titled A Parent’s Guide to Research (https://researchautism.org/a-parents-guide-to-research).
- The National Autism Center responded to “a growing number of unproven treatment methods,” with the National Standards Project for the purpose of helping “families, practitioners, policy-makers, and funders make informed decisions and choose effective treatments.” (http://www.nationalautismcenter.org/national-standards-project/phase-2/)
- Last, but definitely not least, ASAT itself has responded to the consumer’s need to separate the wheat from the chaff and summarized scientific research on intervention on autism (https://asatonline.org/for-parents/learn-more-about-specific-treatments/).
Considering these resources, the consumer can either evaluate a proposed treatment (as per Green & Perry, 1999) or assess the ability of service supports to evaluate a proposed treatment. Moving forward, however, Green makes the case that the burden must transition from the consumer to the practitioner. While the BACB has some standards that begin to address this, as in Conduct Guideline 6.03 Preparing for Consultation, which states that the behavior analyst must implement or consult on behavior management programs for which the behavior analyst has been adequately prepared, the standards, per Green, must be expanded to best prepare practitioners for skillful consideration of scientific evidence. The take-home point of this article is for the “buyer” to continue “to beware.” In this world of scientifically based and unscientifically based interventions, it is critical that we not only pursue new research on treatment options but also be required to learn how to effectively evaluate those that already exist.
References
Green, G. (2010). Training practitioners to evaluate evidence about interventions. European Journal of Behavior Analysis, 10, 223-228.
Green, G., & Perry, L. (1999). Science, pseudoscience, and antiscience: What’s this got to do with my kid? Science in Autism Treatment. Retrieved from https://asatonline.org/wp-content/uploads/NewsletterIssues/spring99.pdf
Citation for this article:
Kinney, E., & Houston, Z. (2011). A review of Green, G. (2010): Training practitioners to evaluate evidence about interventions. In European Journal of Behavior Analysis. Science in Autism Treatment, 8(4), 20-21.
