Authored by the OECD team: Natalia Nolan Flecha, Arnault Pretet, Simon Callewaert and the CAF team: Gracia Vara Arribas and Veronica Menegatto
As part of a new European Commission-OECD project, the OECD and EIPA have carried out a survey to public sector organisations using the Common Assessment Framework (CAF) model in EU Members State that showed that they have been widely impacted by the Covid-19 pandemic, especially on people and processes criteria. Public sector organisations had to adapt and found the CAF model useful to respond to the crisis, implementing innovative measures on digitalisation, remote working and simplifying procedures. Priorities looking ahead include digitalisation and workforce management to strengthen their resilience to future shocks and will be further explored in next project phases.
Introduction
The coronavirus pandemic has been a stress test for governments and their public administrations. The crisis has challenged like never before the capacity of governments and public sector organisations to deliver on their goals and priorities. While the crisis cannot be put behind us yet, public administrations are currently reflecting on the lessons learned and how to improve responses in regards to future shocks.
The OECD, the European Commission (DG REFORM), and 12 EU Member states (Austria, Belgium, Bulgaria, Croatia, Finland, Greece, Italy, Poland, Portugal, Slovakia, Slovenia, and Spain) have embarked on a joint research project to strengthen the resilience of public sector organisation in EU Member states, using the Common Assessment Framework (CAF) model, a total Quality Management tool designed for the public sector.
The research aims to explore on three directions:
- A backwards look, by taking stock of the responses in managing the Covid-19 crisis to generate and share case studies and practices;
- A forwards look, by promoting innovative and key public governance themes to make public administrations more resilient to future shocks and challenges;
- A look into quality, by analysing and strengthening the use of the CAF model in the post-Covid-19 period.
In order to understand how the public organizations have experienced and managed the pandemic and how CAF contributed to their organisational readiness and resilience, the research entails different phases with a combination of an online survey (that is now completed and the topic of this blogpost), case studies, (deep-dive) workshops and analysis. These phases will allow the development of different perspectives, analysing what went on in public service organisations during the crisis and how it may have affected their resilience and how it steered them to a more resilient future.
In this blog we present the results of the first survey phase of the project, stemming from the online survey conducted by the OECD with the support of the European CAF Resource Centre at the European Institute of Public Administration (EIPA) in 2021.
The survey process
As a first step, a questionnaire was designed to collect insights into the responses of public sector organisations to the crisis, their use of the CAF tool and its role in supporting greater resilience across its different CAF criteria – Leadership, Strategy and Planning, People, Partnerships and Resources, & Processes.
The extensive questionnaire was distributed among 2,000 CAF users. The reason to focus on organisations that have implemented (or are planning to implement) CAF is twofold:
- CAF is a free and user-friendly total quality management tool, designed to assist organisations across Europe in applying quality management techniques to improve their performance. The organisations implementing CAF have been using a structured self-assessment and performance evaluation process, with a focus on digitalisation, agility, sustainability, diversity and innovation. Our focus group is thus conducting a progressive journey towards excellence with a holistic approach with a model that has proven its value over 20 years of practice. As a consequence, important lessons could be derived from their results on performance evaluation.
- The online survey wanted to reach a very large number of public sector organisations, and the database for doing that was available at the European CAF Resource Centre, which hosts the data on all CAF users in Europe, and is sponsored by EIPA.
The survey tried to identify the biggest challenges related to the management of the Covid-19 crisis, the strategies that were employed to navigate it, the enablers, best practices, and lessons learnt, as well as the new processes that could potentially remain in place after the crisis. Several questions tried to identify the highest priority areas for embedding resilience post-Covid-19, the steps that organisations are taking to do this, and challenges and opportunities foreseen. Finally, the survey aimed to identify which aspects of the CAF model were particularly helpful in the Covid-19 crisis, and which were perhaps less, and how the model could be used to strengthen resilience in the recovery.
The questionnaire was distributed in the fall of 2021 and run until mid-November 2021. This lead to a good response rate of 174 respondents from 23 countries, shedding light on the key trends and good practices which can serve to strengthen their preparedness in the face of future shocks. The high response rate was achieved thanks to the efforts of the EU CAF resource centre at EIPA, the OECD and the CAF National Correspondents, who strongly supported the promotion among the CAF users community.
The survey results
Who are the respondents?
Of the 23 countries that participated in the survey, Bulgaria (52) and Italy (32) recorded the highest response rate out of 174 answers. After weighing the responses, this overrepresentation appeared not to have distorted the survey results. Furthermore, the respondents show a relatively good balance between central and local levels of government. Small-sized organisations (up to 50 employees) are best represented with 40% whereas medium-sized (51 – 250 employees)) and large organisations (above 25 employees) make out close to a 30% of respondents each.
The distribution of responses by sector shows that public sector management departments are overrepresented due to the nature of the CAF model which targets these organisations, that includes a lot of central and local administrations, followed by and Education, Research and Culture.
How has the Covid-19 crisis impacted CAF users?
The results show that the crisis has impacted CAF users rather homogeneous across levels of government on all dimensions (Leadership, Strategy and Planning, People, Partnerships and Resources, Processes), but especially regarding people and processes and slightly less on leadership and results.
The usefulness of the model is confirmed by the facts that 77% of respondents confirmed that the model was useful during the crisis and more of half of the respondents implemented it during the crisis. Previous use of CAF model and/or the implementation of other plans have helped too. On the other hand, non-CAF users reported to be slightly less prepared, were more impacted and were less able to adapt on most dimensions.
All organisations who used the model reported a high level of adaptation on all CAF criteria, but on people the most. The highest amount of new and innovative practices have been taking place in the areas of digitalisation, remote working and the adoption of emergency/simplified procedures.
What about the priorities of the respondent organisations?
The responses also allow for comparing public sector organisations’ top priorities before and after the crisis, which appeared to have shifted during the crisis. Organisational effectiveness, digitalisation and financial sustainability formed the top three priorities prior to the crisis. While digitalisation has remained a key priority, workforce management has now emerged as a top priority as well with the related challenges of remote working. Two-thirds of respondents also expect that the new work and delivery practices can have a positive impact on the sustainability and climate footprint of the organisation.
Except for few public management organisations, trust has not been reported among the top post-Covid-19 priorities – however it is important to stress that organisations could only indicate three priorities and that priorities on workforce management and digitalisation were overwhelmingly selected, followed by innovation, the need for new service delivery models and organisational agility. This overview on top strategic priorities, doesn’t change much if we have a closer look at the size of organisations and the level of government.
And what do the results by CAF enablers criteria tell us?
During the crisis, the main challenges for the leadership across European public administrations were ensuring a certain degree of safety and opportune working conditions for the employees, finding new ways to deliver existing services despite the restrictions, and keeping the staff motivated and willing to accept the new ways of working. To address these challenges, leadership had to step up fast and consider immediate and crucial priorities on safe employment. They also had to draw up innovative solutions to adopt new tools and coordination mechanisms.
From the point of view of strategy and planning, most of the respondents felt insufficiently prepared and revealed to lack the necessary tools to face a crisis situation such as the coronavirus pandemic. The main challenges in achieving strategic priorities were related to difficulties in organising workforce management, developing new strategies and plans – because of uncertainties – and delivering services.
As for the people criterion, workforce practices and allocations had to be adapted fast, but faced constraints. The immediate challenge has been the lack of technological infrastructure and digital tools to cope with the remote work requirements. People has showed up as the most impacted dimension inside the organizations, but it also shows the highest adaptation rate of new practices and has been the area where the CAF model helped the most. As the effects of the crisis fade, organisations are still reflecting on keeping hybrid work models, including a significant share of teleworking, or moving back to more office-based models.
The partnerships and resources criteria, suffers most evidently from the fact that restrictions have limited the capacity to reach partners or to meet new ones, mainly because of the already mentioned and rather widespread lack of technology and digital tools. Not many organisations implemented new partnerships with suppliers, civil society or other types of external stakeholders. Only one-thirds of respondents stated that they could implement new partnerships during the crisis.
Finally, the impact of the crisis was important on most internal processes, particularly those in need of digitalisation such as communications, IT and HRM departments. Service delivery models had to make a big effort to turn digital in order to communicate, collaborate, and deliver. As the digitalization has even further increased due to the pandemic, this shift should be maintained as it also helps to better face future shocks.
Conclusions and way forward
The results of the survey show the popularity and usefulness of the CAF model during the coronavirus pandemic as CAF users felt distinctively better prepared and more prone to adapt. The crisis has impacted all criteria of the model, but especially people and processes. People management and digitalisation turn out as clear focus areas during the pandemic, while new long term priorities indicate that they will remain, along with resilience, as the top three in the post-crisis forecast.
The results further indicate that the CAF model needs to evolve to better reflect new dimensions stemming from the crisis such as innovation, digitalisation and resilience. These are also some of the long-term strategic priorities identified by responding organisations. Therefore, in response to the crisis, a further CAF evolution could be considered in the coming months and years.
The next phases of the research project led by OECD and the European Commission will now analyse concrete cases of CAF users in order to identify the most innovative practices in handling the pandemic that can be shared across organisations. This will help map out the changes and new practices organizations will need to adopt in order to enhance their resilience to future shocks. It will also further identify areas for improvement in the self-assessment methodology of the Common Assessment Framework (CAF), so the model can continue best supporting public sector organisations in future crises as it did during the Covid-19 crisis.
The views expressed in this blog are those of the authors and not necessarily those of EIPA.
