Welcome cocktail reception, Tuesday 18 September 6:00–8:00pm, open to all delegates! Venue: Penny Royal
Back To Schedule
Wednesday, September 19 • 11:00am - 12:00pm
Big data, big possibilities, big challenges: Lessons from using experimental designs in evaluation of system-level educational reforms

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
Duncan Rintoul (NSW Department of Education), Ben Barnes (NSW Department of Education), Ian Watkins (NSW Department of Education)

For many evaluators, quasi-experimental designs fall at the first set of hurdles, due to the absence of readily available data sets and the difficulties associated with identifying appropriate comparison/control groups. At the NSW Centre for Educational Statistics and Evaluation (CESE), we have been fortunate to clear these first hurdles on occasion, only to then hit the second set: the technical challenges of working with big data.
This paper is a chance for participants to get their hands dirty... or at the very least to hear the stories of people with dirty hands. The presenters are senior practitioners - the Director of CESE's evaluation unit and the Principal Data Analyst responsible for statistical modelling. The paper will lift the lid on this important (but uncommon) aspect of evaluation practice: the models they build; the data management challenges they face; the internal political challenges they face; the statistical methods that bear more - or less - fruit; and how they translate 'heavy quant' back into actionable insights for policy and program management. 

Through a set of case studies, the presenters will draw out practical lessons and tips for making these designs work - including what the team has needed in terms of skillsets, models, software, datasets, mindsets and other complementary elements of evaluation design that sit alongside the quant.

avatar for Keryn Hassall

Keryn Hassall

I'm interested in how people think - about evaluation, about policies and programs, about their work, and how people think about other people.


Ben Barnes

Director Evaluation, NSW Department of Education
I began in consultancy, and made the move to the public sector in 2012. I am now Director of Evaluation at the Centre for Education Statistics and Evaluation in the NSW Department of Education. We have a team of over 30 internal evaluators, data analysts and evaluation capacity builders... Read More →
avatar for Duncan Rintoul

Duncan Rintoul

Manager, Evaluation Capacity Building, NSW Department of Education
I have been working in social research and evaluation since 2000. My favourite things to chat about, apart from my kids and how good Wollongong is:* evaluation capacity building* design thinking and innovation* evaluative practice in education, particularly in schools * public sector... Read More →

Ian Watkins

Principal Data Analyst, NSW Department of Education
Ian has a background in psychology and statistics. He studied and taught at the University of New South Wales before moving to the public sector. He is now the Principal Data Analyst at the Centre for Education Statistics and Evaluation in the NSW Department of Education where he... Read More →

Wednesday September 19, 2018 11:00am - 12:00pm AEST
Chancellor 5