In his Report of the Auditor General of Alberta—July 2014 released today, Auditor General Merwan Saher includes the results of one new systems audit and seven follow-up systems audits, and a comment on the budget for financial reporting.
Results analysis reporting (page 15)
Why we did this audit
We examined the quality of the government’s results analysis reporting to Albertans, for two reasons:
- Public accounts committee members are questioning the quality of this reporting, finding that it lacks detail and precision.
- We want to highlight—and reinforce—the government’s results-based-budgeting initiative. This initiative reviews programs and services to ensure they will achieve results that Albertans want. Consistent with this, the government needs to tell Albertans whether it is achieving the results they want. Simply put, there is not much point in setting targets for results unless you analyze the results to learn what is working and what is not.
What we found
Public service managers and the government need to improve the quality of results analysis in many ways, but the payback will be worth the effort.
Ministries need to:
- identify and analyze results to improve their reporting
- present a clear analysis of results for significant matters, including business plan priority initiatives
- shift the balance in their annual reports—increase the reported analysis of results and decrease the number of programs and activities described without analysis
- develop standardized guidance and training to direct and support improving results analysis reporting by ministries
- monitor results analysis reporting compliance with the annual report standards within each ministry and across government
A results management framework is essential for progress. The framework shows that the whole purpose of results analysis reporting is to learn from what you do, so you can do better in the future. The framework has three inseparable elements: governance, oversight and accountability for results.
Governance—a process and structure that bring together capable people and relevant information to achieve desired results (for cost-effective use of public resources).
Oversight—the glue that holds everything together. If more people practiced proper oversight, the results could be spectacular. Ministers need to provide oversight—not management (managers do that). To provide proper oversight, ministers need to:
- be vigilant
- check that processes and systems, including the accountability-for-results system, are working well
- signal to managers preferred behaviour
Without proper oversight, achieving results is a game of chance.
Accountability for results—the obligation to show continually improving results in the context of fair and agreed-on expectations. For Albertans to receive value for money, all those who use public resources must:
- set measurable results and responsibilities
- plan what needs to be done to achieve results
- do the work and monitor progress
- report on results
- evaluate results and provide feedback (in other words, results analysis)
Managers have to feed the results analysis back into their planning. Should they continue a program at its current cost because it produces the results they want? Should they spend more because evidence shows that will produce more of what they want? Should they quickly abandon a program that is not working, or that costs too much for the results it achieves?
We stress that the framework’s three parts—governance, oversight and accountability for results—work together to produce results. You won’t get accountability for results without oversight. Invariably, good oversight improves results. And without the right people and information, you have a high risk of waste.
This results management framework can and should operate at three levels:
- the government level—The key being ministerial oversight of the public service
- the cross-government, deputy minister level—We see this level as a virtual entity. This is where evidence-based long-term planning, enterprise-wide risk management, resource allocation and government-wide results analysis has to be overseen.
- the individual organization level—a Crown agency or department, for example
Repeated Recommendations (beginning on page 37)
We followed up on 15 outstanding recommendations. We assessed nine as implemented and had to repeat six.
That a large number of recommendations are repeated should be a concern for the government and the public accounts committee. We repeat a recommendation when we believe not enough has been done to resolve the findings we had in our original audit. We draw particular attention to four of these repeated recommendations.
Recommendations 2 and 3 relate to climate change strategy. Without a clear process for monitoring progress, the government cannot know if the actions it is funding are those most likely to yield the results it expects. These repeated recommendations illustrate why results analysis reporting is not what it should be; the actions that should be reported on are not clear and reporting on actual results is not happening.
Recommendations 4 and 7 focus on enforcement of laws. There is a risk that taxpayers may have to bear the cost of remediating sand and gravel pits if systems to ensure operators meet their remediation obligations are not improved. And, there is a risk to public safety if timely and appropriate enforcement action is not taken on high risk commercial vehicle carriers.
Other Audit Work
Budget for financial reporting—other audit (page 79)
In early 2014 the Department of Treasury Board and Finance spent significant time and effort to prepare the constructed budget included in the province’s 2013–2014 consolidated financial statements, released publicly in June 2014. This constructed budget is necessary because the budget prepared and presented in accordance with the Fiscal Management Act is not suitable for comparison to the audited results contained within the province’s consolidated financial statements.
Now that the department has developed processes to prepare a constructed budget, the department needs to consider when the constructed budget should be prepared and made public. In our opinion, the constructed budget should be prepared and made public prior to the beginning of the period to which it relates. Ideally, this would be done at the same time the budget under the Fiscal Management Act is made public.