aes18 International Evaluation Conference

Gerard Atkinson (ARTD Consultants)

One of the challenges faced by evaluators is how to effectively determine the impacts of a program when a control group is not readily available. Sometimes the design of the program makes such groups impossible or unethical to create (e.g. mandatory or selective participation), or constraints on resources and scope make such investigations infeasible.

These challenges have led to the development of quasi-experimental and natural experimental approaches to evaluation.In parallel to the adoption of these techniques, the shift to policies of "open government" has enabled greater public access to data. Much of these data capture transformations in society over time, or provide records of how people have interacted with government and public services. In the right situations, these data can be used to augment impact evaluations through creating comparison groups for analysis.

This presentation looks at a variety of publicly available data sources, ranging from large scale longitudinal studies such as HILDA, geographic data such as the Geographic National Address File, or transactional data such as public transport journeys. These data sets can be used to enhance the robustness of quasi-experimental and natural evaluations. Through exploring example data sets and case studies, we consider the challenges of identifying and preparing such data, the privacy and ethical implications, and the value that such data can add to the evaluation process.