• Contact Us
• Sitemap
• UCD Home

Evaluation of the Effectiveness of the Childhood Development Initiative’s Mate-Tricks Pro-Social Behaviour After-School Programme (2012)

Study Number (SN): 0005-00

CITATION

ABOUT THE STUDY

This report presents the findings of an independent evaluation, undertaken by the Centre for Effective Education at Queen’s University Belfast, of the Mate-Tricks pro-social behaviour after-school programme. Mate-Tricks is an after-school programme designed to promote pro-social behaviour in Tallaght West (Dublin). Itis aimed at improving pro-social behaviour amongst children aged 9-10 years (Irish 4th class). The intended outcomes of this programme include enhanced children’s pro-social development, confidence and self-esteem, problem-solving skills and reduced anti-social behaviour. This evaluation primarily focused on assessing the impact of Mate-Tricks on children’s outcomes.

The programme is part of a wider 10-year strategy that began in 2003, led by the Childhood Development Initiative (CDI), which aimed to improve the health, safety and learning of children. Mate-Tricks is a bespoke intervention that combines elements from two pro-social behaviour programmes: the Strengthening Families Program (SFP) and Coping Power Program (CPP). The programme is a one-year multi-session after-school programme comprising 59 children-only sessions, 6 parent-only sessions and 3 family sessions, with each session lasting 1½ hours.

The evaluation of Mate-Tricks utilised a 3-year rolling cohort randomised controlled trial design. The rolling cohort nature of the study means that the randomisation was carried out in three calendar years independently and combined after the trial to produce a sample with sufficient size and study power to detect effects if they were present. The children were referred from 7 schools and were individually randomly assigned within each of these schools. Each school contributed children for each cohort and therefore there was an experimental balance. In short, the data is individually pupil randomised and clustered at school level. Overall, the trial had two arms i.e., children were randomly allocated to the intervention group (which received the programme), or the control group (which did not receive the programme). A total of 589 children randomised to the intervention (n=304) and control conditions (n=285).

MAIN TOPICS

• Extracurricular activities
• Social behaviour
• Schoolchildren
• Economic and social development
• Self-esteem
• Problem solving 

COVERAGE, UNIVERSE, METHODOLOGY

Population

The evaluation was carried out among children aged 9-10 years living in Tallaght West, their parents and teachers. The Mate-Tricks programme is a one-year multi-session after-school programme comprising 59 children-only sessions, 6 parent-only sessions and 3 family sessions, with each session lasting 1½ hours. In Mate-Tricks, it was proposed that 210 children would be referred to the programme each year for 3 years, giving a maximum proposed total sample of 630 children. In total, 592 children were randomly assigned to intervention and control groups over the three cohorts. 347 children and 122 parents completed both pre- and post-test questionnaires.

Observation units

Individual

Temporal coverage

Dates of fieldwork: 09/2008 to 06/2011

Time dimension

The current study utilised a 3-year rolling cohort design. A rolling cohort design ensures that the impact of the intervention (such as Mate-Tricks) is looked at in a longitudinal way (i.e. over the three cohorts) and provides a sufficient sample size for statistical comparisons and effect size calculations.

Geographical coverage

• Country: Republic of Ireland
• Region: Leinster
• County: Dublin
• Town: Tallaght (West)

Sampling procedures

The evaluation included a randomised controlled trial (RCT) to measure the effects of the programme on child outcomes and a process evaluation that investigated its implementation. The children were randomly selected to the intervention or control group. There were 15 places available in the programme for 5 settings and 30 places available in one setting. Three cohorts of children were necessary in order to achieve a large enough sample to rigorously assess the programme’s effects. In Mate-Tricks, children completed pre-tests at the beginning of the programme in September and post-tests at the end of the programme in June.  A rolling cohort design also ensured that the impact of Mate-Tricks was looked at in a longitudinal way (i.e. over the three cohorts), as well as providing a sufficient sample size for statistical comparisons and effect size calculations. In total, 592 children were randomly assigned to intervention and control groups over the three cohorts. 347 children and 122 parents completed both pre- and post-test questionnaires.

SPSS was used to analyse the data. Measures of means, standard deviations (SD) and significance of difference on outcome variables were carried out. Primary and Secondary Outcome variables, contributing scales and Cronbach’s alpha were also analysed. A sample size power calculation was calculated based on: identifying an effect size with a Cohen’s d in the range of 0.2-0.4; a statistical power level of 0.8; having a minimum of two predictors in the model; and identifying a probability level of 0.05. Descriptions of the weights will be included in the database.

Methods of data collection

PAPI (Paper and Pencil Interviewing) with interviewer (face to face, telephone or other)

PAPI (Paper and Pencil Interviewing) self-completion (distributed by post, email or other)

Focus groups with children and parents were also carried out.

Response rate

There was some attrition during the various stages of testing. The major reason for this was that some children were absent from school on the day of testing or left the school after referral was made. However, the pattern of attrition was similar in both the intervention and control groups, with similar numbers leaving the study at the various stages. Also, the final numbers in the analysis (61% intervention group and 57% control group) were similar. 

There was some attrition in teacher responses during the various stages of testing. The major reason for this was that a school withdrew from the teacher evaluation element and teachers did not return the questionnaires. The non-return of questionnaires was most prevalent during Cohort 1, particularly at pre-test, as there was initial scepticism about the programme. Therefore, the post-test return rates were better. However, the pattern of attrition was similar in both the intervention and control groups, with similar numbers leaving the study at the various stages.

There was some attrition in parental responses during the various stages of testing. The main reason for this was that parents did not return questionnaires and, furthermore, did not complete them after a second resend of the questionnaire and follow-up telephone calls (for Cohorts 2 and 3). However, the final numbers in the analysis (20% intervention group and 22% control group) were similar.

In total, 592 children were randomly assigned to intervention and control groups over the three cohorts. 347 children and 122 parents completed both pre- and post-test questionnaires. Attrition rates appeared to have no impact since participants in the final analysis were well matched in the control and intervention groups.

DATA AND DOCUMENTATION: FILES’ DESCRIPTION

Data (available through ISSDA application process)

Documentation (available for download)

LINKS

Mate-Tricks

O’Hare, L. (2014). Did Children's Perceptions of an After-school Social Learning Program Predict Change in Their Behavior? Procedia-Social and Behavioral Sciences,116, 3786-3792.http://www.sciencedirect.com/science/article/pii/S1877042814008593

O’Hare, L., Kerr, K., Biggart, A., & Connolly, P. (2012). Evaluation of the Effectiveness of the Childhood Development Initiative’s' Mate-Tricks' Pro-social Behaviour After-school Programme. CDI, Dublin.

O'Hare, L., Biggart, A., Kerr, K. & Connolly, P. (forthcoming) A Randomized Controlled Trial Evaluation of an After-School Pro-Social Behavior Program in an Area of Socioeconomic Disadvantage. The Elementary School Journal http://www.journals.uchicago.edu/doi/10.1086/683102

ACCESS INFORMATION

Accessing the data

To access the data, please complete a ISSDA Data Request Form for Research Purposes - Pseudonymised Datasets, sign it, and send it to ISSDA by email.

Data will be disseminated on receipt of a fully completed, signed form. Incomplete or unsigned forms will be returned to the data requester for completion.

For teaching purposes, please complete the ISSDA Data Request Form for Teaching Purposes - Pseudonymised Datasets, and follow the procedures, as above. Teaching requests are approved on a once-off module/workshop basis. Subsequent occurrences of the module/workshop require a new teaching request form.

Please contact us if you have any queries.

Acknowledgements

Any work based in whole or part on resources provided by the ISSDA, should  acknowledge: “Mate-Tricks Pro-Social Behaviour After-School Programme" and also ISSDA, in the following way: “Accessed via the Irish Social Science Data Archive - www.ucd.ie/issda”.

Citation requirement

The data and its creators shall be cited in all publications and presentations for which the data have been used. The bibliographic citation may be in the form suggested by the archive or in the form required by the publication.

Bibliographical citation

Evaluation of the Effectiveness of the Childhood Development Initiative’s Mate-Tricks Pro-Social Behaviour After-School Programme (2012) [computer file]. Belfast: The Centre for Effective Education at Queen’s University Belfast & Tallaght West: Childhood Development Initiative [producers], 2012. Dublin: Irish Social Science Data Archive [distributor], 2016.

Notification

The user shall notify the Irish Social Science Data Archive of all publications where she or he has used the data.