What results analysis means
Results analysis is a technical term. It is a process that managers use to check how well a program is working and to improve it. Managers examine program results and costs, and ask:
- What is working and what is not?
- Do actual results match the planned ones?
- Does the program achieve these results at the expected price?
If not, they adjust their efforts (what they do, how they do it, how much they spend) to improve results.
What we examined
Our audit objective was to assess the quality of the results analysis information publicly reported to Albertans through ministry annual reports. Results analysis requires public sector managers to integrate their financial and non-financial performance information into a clear and concise explanation of performance in relation to business plans. In identifying significant results (good, as well as any that are unsatisfactory) and through the analysis, there will be learnings for the next planning cycle.
What we found
The Alberta government has a well-established public performance reporting regime. Business plans and annual reports are prepared by each ministry. Standards exist for reporting both financial and nonfinancial performance information in ministry annual reports.
Through our audit, we found that:
- significant ministry matters (including priority initiative results) were not sufficiently identified or analyzed in ministry annual reports
- little guidance or training was provided to assist ministry management in their results analysis and annual report preparation activities
- results analysis reporting compliance against the annual report standards was not monitored
Much of the results analysis in each annual report’s commentary described a broad range of the ministry’s programs and activities. Given this, it was difficult to determine which aspect of operations senior management considered most important. Among the multitude of program descriptions, it was also difficult to identify insight from management that would explain the completion of priority initiatives and progress towards goals, which are essentially statements of desired results.
Financial information was generally not incorporated into the commentary. Financial information would help in assessing ministry results against plans and success in achieving results at a reasonable cost.
What needs to be done
Ministries need to focus on the identification and analysis of results to improve their reporting. Ministries need to clearly present an analysis of results for significant matters, including business plan priority initiatives. Ministries also need to shift the balance in their annual reports – increase the reported analysis of results and decrease the number of programs and activities described without analysis.
Standardized guidance and training should be developed to direct and support improving results analysis reporting by the ministries. In addition, results analysis reporting compliance with the annual report standards should be monitored within each ministry and across government.
Why this is important to Albertans
Albertans, members of the legislative assembly and the government need sufficient and appropriate results analysis information that will allow them to assess if ministry programs are contributing towards meeting goals and achieving desired results.
Without such information from management, ministry performance and progress in achieving success with priority initiatives cannot be evaluated. The ability to identify lessons for operations and future planning may be compromised. Without public reporting that identifies and explains results each year, the government’s accountability for results to Albertans is incomplete. Albertans need to be told of opportunities to improve performance.
A significant amount of staff resources go into the planning, preparation, review and approval of each ministry annual report, and the business plans that precede the report. Better value for this investment would be achieved by improving the analysis of results to help Albertan’s understand the performance of their government.