What makes an intervention effective? How are schools to decide what to fund or deliver? Gareth D Morewood explains what we can glean from the existing literature.
I've been involved in many discussions around how we evidence the effectiveness of interventions in improving learning outcomes, specifically with regard to effect sizes and meta-analysis.
As a researcher, I am fortunate enough to have the opportunity to work with colleagues on large-scale projects, in addition to the smaller studies staff my school undertakes. One of these larger projects was published in 2016 as a piece of work commissioned by the National Council for Special Education in Ireland (Bond et al, 2016).
It involved:
Evaluating the impact of research on different groups comes with a note of caution. As highlighted in the opening paragraph, the applicability of an intervention to a new context may not be as effective for a number of other factors.
(Professor Steve Higgins, Research Communication and Use for Schools, 2017)
We undertook three ‘strands’ of evidence gathering.
For the primary systematic literature review strand, a process was adopted in order to ensure rigour in the selection and review of studies. We identified 1,021 possible articles for consideration.
Following a process of applying inclusion and exclusion criteria to determine the quality of the evidence, methodological appropriateness and effectiveness of the intervention, we determined 85 studies to be ‘best evidence’.
The systematic review process was complemented by two supplementary strands.
Focusing the literature review on the evidence from 2008–13 enabled this study to reflect the most recent trends (at the time of writing) pertaining to autism intervention research.
However, the review could not provide a definitive or cumulative analysis of all interventions to support autistic children and young people in education settings over a longer period. It’s important to consult a wider body of literature when deciding whether or not to deliver or fund a particular intervention.
Our intervention rating scheme (all studies were from the 2008-2013 period) consisted of a mark out of four, identified as follows.
4. Most evidence
At least four studies providing positive evidence which either includes a positive RCT, quasi-experimental study (QES) or six or more single case experimental (SCE) studies.
3. Moderate evidence
Three or more studies providing positive evidence which either includes a positive RCT or QES or four or more SCE studies.
2. Some evidence
Two or more studies providing positive evidence which either includes a positive RCT or QES or three or more SCE studies.
1. Little evidence
One RCT/ QES or one or two SCE studies providing positive evidence.
In a pre-school setting, the following were all awarded four marks (most supportive evidence) as indicated by our rating scale above.
Interventions to increase joint attention skills (N=4)
Mostly one-to-one, play-based or turn-taking interventions with an adult (teacher or parent).
Comprehensive early interventions (N=10)
Interventions part of holistic learning experience and targeted a range of areas, for example social skills, behaviour, communication, attention and learning, often in a school setting. Measures sampled a range of areas of development.
Those rated three marks (moderate evidence) were:
Play-based interventions (N=3)
One-to-one and small group interventions focused on teaching play skills or peer mediated play activities.
Video modelling to develop communication skills (N=4)
Use of video of desired behaviour (or video as prompt) to encourage behaviour such as use of a Picture Exchange Communication System (PECS) or sharing information about the school day.
In a school-age setting, the following were given four marks.
Peer mediated interventions in mainstream schools (N=9)
Naturalistic interventions to enable peers to interact more effectively with autistic children. An example could be lunchtime clubs, or meetings in which researchers, peers and a child with autism work collaboratively to plan strategies.
Multi-component social skills interventions (N=6)
Interventions (such as UCLA PEERS, Children’s Friendship Training) included several elements such as a parent/carer group to support social skills/networks and a social skills group and/or social skills training for the young people.
Behavioural interventions based upon behavioural principles (N=7)
Behavioural interventions enabled the development of flexibility and tolerance for change (e.g. multi-element behaviour plans, environmental modification and prompting). These were often based on functional analysis of behaviour as opposed to a behaviourist approach.
The following were awarded three marks.
Social initiation training (N=4)
Pivotal response training or use of scripts to teach the child how to initiate social interaction.
Computer-assisted emotion recognition interventions (N=3)
The use of computer programmes and video modelling to improve emotion recognition.
Picture Exchange Communication System in special school (N=3)
Behaviourally based communication system beginning with the exchange of symbols for desired objects.
Discrete skills training using behavioural approaches (N=4) (this was also found to be the case for pre-school settings too)
Usually one-to-one skills training (e.g. model-lead-test and direct instruction) to support the acquisition of discrete skills such as letter or number recognition.
Narrative interventions (N=5)
One-to-one interventions such as social stories and power cards used to prompt particular behaviours.
You can read the detailed report and findings from the link in the references, but I did was to give a special mention to an intervention that was awarded two marks (some evidence): Lego therapy®.
This small group intervention with clear roles to enable group construction of Lego models is, in my view well worth considering as a greater investment than the rating of two would suggest – we just need to carry out more research into it.
Naturalistic interventions to enable peers to interact more effectively with autistic children
With budgets never having been so tight, it is important to consider any implementation carefully. What can actually be done? What does the evidence suggest is a good investment? As identified at the start of the post, this research does come with a ‘health warning’.
From this study, my personal experience and through working with many different schools and families over the last few years, I consider the following to be the hallmarks of good provision in schools. But don’t forget, a good strategy for an autistic child is most definitely a good strategy for all!
Here are my recommendations.
Explicitly teach rules and routines for play time, break and lunch times, assembly and so on. Don’t assume knowledge with regard to lining up, taking turns or understanding ‘unwritten rules’. Explicitly explaining and showing all children how to undertake different tasks, backed up with a visual support to provide prompts and reminders, is a good way to support an emotionally regulated environment.
Ensure there are clearly identified break, lunch, and before- and after-school clubs. Providing purposeful activities during unstructured time is important and in considering point one above, supporting pupils to understand how to access these clubs and activities and how to engage with them is also important.
Peer-mediated activities and interventions are really effective. Consider the clubs and activities starting from specific interests of young people themselves. Our Warhammer and Manga clubs are very popular, as is the coding club – all of which started from pupils’ suggestions and discussions.
Don’t underestimate the power of peer awareness and understanding. Providing an inclusive curriculum, where discriminatory views are challenged through debate and informed discussions can help to build a sense of community. Addressing misconceptions at the time (Brown, 1998) is important for developing understanding.
Our debate club, a popular identified activity that takes place at lunch times, is a useful addition to the curriculum.
Finally, don’t rush into implementing one of the interventions identified above without considering the broader needs of the school and how it may match with existing provision. Schools sometimes fall into the trap of seeing interventions as a ‘fix’ and process: ‘six weeks and you will be sorted!’
As with research in this area, there are many other factors – home life, community, self-awareness – that all interact. Something that constitutes a good investment or sound bet in one setting may prove ineffective in another.
Whatever your interest, reading the full report, the summary paper or simply considering the action points at the end of this post, simply considering the wider environment and how autistic pupils interact with it will make a difference.
Above all, remember that when you have met one autistic pupil, you’ve met one autistic pupil.
I’ll give one of our students, Bobby the final word.
‘Priestnall is really good because they have many autistic students and they understand us. I have a great time at Priestnall because they know how to deal with me and support me. Social time is great as I get to hang out with other students who like to share my interests.’
Peer support for autistic students: Bobby’s story
Autism and education: key points and resources
Girls and autism: what we need to know
Enhancing social skills one brick at a time
It's time to focus on what the latest evidence-based research tells us works to improve learning outcomes in the classroom.
Our Annual SEND Update conference will give you the skills to translate the latest research into effective practice, on everything from working memory to emotional regulation.
The event will take place in London on 22 May 2018.
Bond, C., Symes, W., Hebron, J., Humphrey, N., Morewood, G. & Woods, K. (2016) 'Educational Interventions for children with autistic spectrum disorder – a systematic literature review 2008-2013', School Psychology International, SAGE.
Calder, L., Hill, V., & Pellicano, E. (2013) '"Sometimes I want to play by myself': under-standing what friendship means to children with autism in mainstream primary schools', Autism: The International Journal of Research and Practice, 17, 296–316.
Hebron, J., & Humphrey, N. (2014) 'Mental health difficulties among young people on the autistic spectrum in mainstream secondary schools: a comparative study', Journal of Research in Special Educational Needs, 14, 22–32.
Humphrey, N., & Lewis, S. (2008) '“Make me normal”: the views and experiences of pupils on the autistic spectrum in mainstream secondary schools', Autism, 12, 23–46.
Humphrey, N., Bond, C., Hebron, J., Symes, W. & Morewood, G. D. (Eds.) (2015) Autism and education (Sage Major Work). London: Sage.
Morewood, G. D, Humphrey, N. & Symes, W. (2011) 'Mainstreaming autism: making it work', Good Autism Practice Journal 02.12.11, 62-68.