In November 2019, Professor Jeroen van der Heijden reviewed briefly the book Regulatory Delivery: Introducing the Regulatory Delivery Model, Graham Russell and Christopher Hodges, editors, (2019), Hart Publishing Oxford, 504 pages.
This article is the third of a series of brief chapter reviews from that book. The chapter reviews will outline the key themes of the chapter, with brief commentary about them in respect to New Zealand experience.
Chapter 13 – Outcome Measurement – Outcome measurement is one of three practice elements of a ‘Regulatory Delivery Model’ that the authors describe as being useful as a map, a diagnostic tool or a predictive model to guide thinking about regulatory delivery. The model overall is made up of three prerequisites: Accountability (ch 7), Culture (ch 10) and Governance Frameworks (ch 3), and three practices: Outcome Measurement (ch 13), Risk-based Prioritisation (ch 16) and Intervention Choices (ch 20). Each chapter ends with a set of ‘Key Lines of Enquiry’ that can be used to assess the current picture in a regulatory organisation.
The chapter settles on using the terms ‘outcomes’, which are defined as the intended and unintended results and consequences of your activities; and ‘impacts’ which are considered to be the long-term outcomes with a wider impact on the community or environment. The premise is that proper governance of a regulatory agency is dependent on the measurement of progress towards outcomes to enable appropriate monitoring, reporting and management of resources and performance. The ability to articulate outcomes and impacts is discussed as being important to provide clarity about where the agency is going and what progress is being made in order to support effective partnerships, credibility and public trust.
It is noted that, ‘despite the deceptive simplicity of the concept of focussing on outcomes, putting the concept into practice can be immensely challenging’. The particular challenges discussed include: how to specify desired outcomes effectively given they can include broad social, economic and environmental components compared to the specific laws and regulations that regulators tend of work with; and understanding the direct and indirect contributions regulators make to outcomes given factors such as the multiplicity of contributions and time lapses in results emerging. The challenges often push regulators to revert to measuring inputs and outputs or perhaps using counterfactuals and logic stories to try to explain their achievements. They certainly require regulators to give deep consideration to the measurement of progress using appropriate performance measurement frameworks supported by good data and analytical capacity – which are both things that are hard to access. The story that this chapter outlines resonates with the New Zealand experience of outcome measurement practice and development and is unlikely to provide any surprises to regulatory practitioners in this country. Nevertheless, drawing on the extensive experience and knowledge of the authors[1] it provides a sound discussion of issues to support us in our constant quest for outcome measurement frameworks that are meaningful and underpin public confidence in our work.
[1] Graeme Russell, Chief Executive of the Office of Product Safety Standards in the UK Department for Business, Energy and Industrial Strategy and Helen Kirkman, who is responsible for regulatory approaches across that Office
This post was written by Keith Manch, the Chief Executive and Director of Maritime New Zealand. He has worked in the public sector since 1977 and brings extensive leadership experience in a number of policy and operational senior leadership positions in regulation, compliance and response.
From the Regulatory Frontlines 