aes18 International Evaluation Conference

Duncan Rintoul (NSW Department of Education), Ben Barnes (NSW Department of Education), Ian Watkins (NSW Department of Education)

For many evaluators, quasi-experimental designs fall at the first set of hurdles, due to the absence of readily available data sets and the difficulties associated with identifying appropriate comparison/control groups. At the NSW Centre for Educational Statistics and Evaluation (CESE), we have been fortunate to clear these first hurdles on occasion, only to then hit the second set: the technical challenges of working with big data.
 
This paper is a chance for participants to get their hands dirty... or at the very least to hear the stories of people with dirty hands. The presenters are senior practitioners - the Director of CESE's evaluation unit and the Principal Data Analyst responsible for statistical modelling. The paper will lift the lid on this important (but uncommon) aspect of evaluation practice: the models they build; the data management challenges they face; the internal political challenges they face; the statistical methods that bear more - or less - fruit; and how they translate 'heavy quant' back into actionable insights for policy and program management. 

Through a set of case studies, the presenters will draw out practical lessons and tips for making these designs work - including what the team has needed in terms of skillsets, models, software, datasets, mindsets and other complementary elements of evaluation design that sit alongside the quant.