aes18 International Evaluation Conference

Timothy Carey (Centre for Remote Health, Flinders Univeresity), George Tremblay (Center for Behavioral Health Innovation, Antioch University New England, US), Jim Fauth (Center for Behavioral Health Innovation, Antioch University New England, US)

In 2017 the Australian-American Fulbright Commission funded a research project at the Center for Behavioral Health Innovation (BHI), Antioch University New England to investigate the important factors for initiating and sustaining ongoing monitoring and evaluation within an organisation.

The research was conducted by an Australian Fulbright Scholar who interviewed 15 people from different organisations with whom BHI had partnered at various times to establish evaluation procedures and protocols for a range of different projects. A surprising finding of the project was the potential to transform the role of external evaluators. Based on the data gathered from the research participants it appears that external consultants can offer important expertise and guidance in an ongoing way.

Rather than working with organisations for discreet periods of time to reach conclusions about a specific program's effectiveness, research participants described the value in having flexible and responsive mentors who were external to the organisation but available in an ongoing capacity. Transforming the way in which the role of evaluators is conceptualised enabled service providers to change their attitudes from fearing evaluation to embracing it as a learning process that is crucial to effective service delivery. While evaluation expertise remained an important aspect of the external evaluator's role, they were able to expand the support they provided and establish a different relationship with organisations. Participants described the value of having summaries of research evidence presented to them by the external evaluators as well as having resources such as powerpoint slides prepared. Importantly, having the external evaluators as an ongoing presence meant service providers were much more likely to maintain fidelity to the relevant model. Transforming relationships with external evaluators required reorganising mindsets concerned with the traditional role of evaluators, however, the benefits of this transformation appeared to be engaged, committed, and motivated service providers.