• What is Civic Evaluation?
  • Definition of Civic Evaluation
  • Civic Evaluation as expression of Active Citizenship
  • Civic Evaluation and Participatory Evaluation
• The experience of Cittadinanzattiva in Civic Evaluation in Italy
  • The “Impararesicuri” (Learn safely) Campaign
  • The Civic Evaluation of “Urban Quality”
• Civic Audit in Healthcare
  • Purpose and goals of Civic Audit
  • The Evaluation Structure of Civic Audit
  • The Evaluation Process and the ways of Citizen Involvement
  • The Civic Audit Effects
• Obstacles and Development Prospects of Civic Evaluation in Italy
  • Obstacles
  • Opportunities

What is Civic Evaluation?

Definition of Civic Evaluation
Civic Evaluation can be defined as an action-research performed by citizens, through the use of established and verifiable methods, to issue reasoned judgements on realities that are significant for the protection of rights and the quality of life.
Therefore, citizens themselves, organized and provided with the appropriate evaluation tools and techniques, produce important information on fields that are deemed significant, such as services provided by public or private organizations (i.e. health, transportation, school, telecommunication, utilities, financial services, etc.) or public policies applied in given fields (such as welfare, environment, justice) at a national or local level.
Civic evaluation allows, in this way, to monitor and verify, for example, compliance with certain quantity- quality standards provided by contractual undertakings or Service Charters, the compliance of given policies to the expectations of citizens or, more, the compliance with specific regulatory obligations, sometimes widely ignored.
Civic evaluation is, therefore, a mainly “technical” activity. Citizens are not limited to the expression of subjective opinions, but they are able to issue judgements, based on data and information collected and processed according to specific methods and, where possible, judgements that are valid and meticulous from a scientific point of view. This technical dimension makes civic evaluation a process similar to other types of social evaluation and research.
The elements which make civic evaluation different from other types of evaluation are:

• in the first place, the “point of view”, from which reality is observed, which identifies, formalizes and makes measurable typical aspects of the citizen's experience, which cannot be interpreted from other observation perspectives;
• second, the fact that such activity is performed directly and autonomously by organized citizens, playing an active role in society in order to improve institutions and policy making.

Within the civic evaluation processes, citizens are at the same time:

• promoters of the process, that is subjects expressing the need to examine in depth and issue a judgement on a given problem;
• enforcers of the survey, because the data and the information regarding the problem are gathered and processed directly by them;
• users of the knowledge produced, because they have a direct interest in producing a change in the reality analyzed.

It is, therefore, not possible to divide the strictly "technical" activity of producing information, from the more properly “political” activity of exploiting it in order to influence society in a concrete way.
Civic evaluation integrates these two dimensions, because it:

• identifies and makes measurable the significant aspects from the citizen’s point of view;
• defines a set of technical tools for data collection and for information processing;
• allows citizens to assert their interpretation during policy making processes.

Summarizing, in civic evaluation processes, the evaluation action necessarily coexists with the mobilisation of the people concerned with a specific issue, the sharing of information and assessment of the problem and the involvement in finding and implementing solutions. The evaluating citizen is always - and in any case - an active citizen who is interested in changing society.

Civic Evaluation as expression of Active Citizenship
For the above mentioned reasons, there is no doubt that civic evaluation can be considered as a form of expression of active citizenship.
We mean, by “Active Citizenship”, the ability of citizens to organise themselves autonomously, to mobilize human, technical and financial resources, and to act within public policies, through different methods and strategies, in order to protect rights and attend to the greater good.
This is a wider concept of citizenship than the traditional one, which lists an assembly of rights and duties which asserts that an individual belongs to a national identity.
This new concept lays stress on exercising powers and on citizen responsibility in tackling the problems of public life that interest him as her directly. In other words, organized citizens offer themselves as political players. Their presence is related to governance in society and to the general interest, not only with the resolution of single issues, or with the mere defense of private interests.
Civic organizations, therefore, make citizens the primary players in the defense of their own rights and in the care of the common good, in a role that is not alternative but complementary to the role of democratic institutions.
For this objective, the ability to make citizens main players of the policy making process and to enhance their level of empowerment in the public arena becomes crucial. One of the most efficient strategies consists in enhancing the citizen’s level of specialization and knowledge of the single issues, through better analysis and information production skills. This is what Aaron Wildavsky defines as “analyst citizen”.
A citizen becomes an analyst when he/she is able to produce autonomously a knowledge of social phenomena or problems, through which he/she can direct his/her own actions within the system of relationships between the political actors and public policy.
According to the active citizenship approach, therefore, civic evaluation, as described above, is a vital empowerment tool offered to citizens and civic organizations. The autonomous production of information can allow the reduction of subordination, to or dependency on others or, in positive terms, to enhance the power of citizens and their ability to influence.
The knowledge coming from civic evaluation processes can produce actions of information, listening and assistance to citizens, interaction with institutions, participation in public policies or, in a simpler way, complaints, claims or legal action.
In any case, such knowledge is a potential source of original power, not derived from other authorities: the power to produce and spread information and judgements based on reality, the power to survey and verify the correct operation of institutions, the power to directly intervene to solve problems or to meet needs coming from citizens (for example through co-production of public policies or services).
Acting on the system of relationships, citizens can produce a social change and start collective learning processes. The result of such processes depends, obviously, on the relationship system itself and on its context.
This is not the right place to deepen the examination of the links between evaluation, organizational learning or social change. In any case, it cannot be doubted that civic evaluation, among the several forms that can be taken by policy evaluation, is specially linked to learning processes. The will to act is intrinsically present in the evaluation action itself.

Civic Evaluation and Participatory Evaluation
The last thought leads us to a further consideration regarding the distinction between Civic Evaluation and Participatory Evaluation.
A few years ago, the onset of participatory approaches in evaluation practice started a wide debate, which is still very active.
According to many authors, drawing on several concrete experiences, the involvement of stakeholders in evaluation processes can bring many benefits. First, it widens the perspective used to tackle a specific issue and improves the quality and depth of the issues on which the evaluation process is founded. Second, it makes the process clearer and more shared, encouraging comparison, communication and collaboration between subjects with different interests. Lastly, it helps the evaluation process itself and makes each stakeholder more conscious and capable in the evaluation.
Summarizing, a participative approach is considered an element which can make the findings more useful, significant and believable.
The civic approach to evaluation is partially different from the participative, although very close to it. Usually, participative methods develop the listening of the different stakeholders with sophisticated techniques, but they do not recognize them as subjects able to produce on their own structured evaluations. The point of view of citizens, in particular, is considered a survey object, one of the several points of view which may be taken in account while developing the different evaluation steps.
In civic evaluation, the citizen ceases to be a mere survey object and becomes the evaluating subject which analyses reality using his own tools, collects data, analyses documents, interviews managers, service directors, etc.
In this process, citizens seek confrontation and dialogue with institutions and other actors, possibly in a partnership and collaboration relationship, but they consider evaluation activity an autonomous and independent power. The main goal is, therefore, to make citizens more capable and efficient in their participation in public life and in their relationship with the institutions. It becomes possible, through the findings of the evaluation, to create pressure to share and to implement the improvement programs with the institution itself.
In conclusion, with respect to participatory evaluation, civic evaluation appears to grant citizens greater autonomy and to provide them with better possibilities of interaction at the same level with the institutions involved. At the same time, the civic evaluation process does not aim at guaranteeing that every point of view is expressed, as is in the participative evaluation processes, even if it remains open to other stakeholders and requires confrontation with external interlocutors.
This said, in the real world, the distinction between civic evaluation and participatory evaluation can be less clear and appear a pure academic exercise. The need positively and actively to involve citizens in the definition, implementation and evaluation of the public policies is still a very open and topical issue.
On the other hand, forms of concrete citizen participation can also vary in relation to the political context and the evolution of time. In Italy, notwithstanding some renovation processes started during the Nineties, there is still a weakness in the institutional processes of evaluation of the policies and performance of public administrations. Formal evaluation is still scarcely practiced, in the same way that inclusive and participatory decision-making processes are still limited to few local experiences. This entails that civic evaluation initiatives themselves, in some cases, flank evaluation processes performed by the institutions, but, in other cases, fill a blank, express a need for knowledge and play an almost controlling role over the actions of institutions themselves.
In this sense, indeed, civic evaluation in some way anticipates and stimulates the birth of forms of participatory evaluation and, more generally, better attention to the involvement of citizens in governance processes.

The experience of Cittadinanzattiva in Civic Evaluation in Italy
During the last ten years, Cittadinanzattiva has been the first organization that promoted and developed projects and methods of civic evaluation in different fields of activity of the Italian Public Administration.
In the healthcare field, in particular, through the Civic Audit method, during the 2001-2010 period, mixed teams of citizens and operators performed a full cycle of evaluation in more than 150 local and hospital health public authorities, gaining significant cultural and organizational returns.
The examination of the Civic Audit method will be deepened in the next chapter. Two evaluation initiatives are briefly shown herein: they were performed, respectively, with regard to school buildings and urban quality.

The “Impararesicuri” (Learn safely) Campaign
Starting from 2002, Cittadinanzattiva promoted a national campaign for the collection of up-to-date data regarding the condition of the Italian school building stock, through the monitoring of a significant number of buildings nationwide (during the first year the campaign covered 70 schools and gradually spread throughout the nation. In 2004 200 building s were monitored, 382 in 2005, 271 in 2006, 184 in 2007, 132 in 2008 and 82 in 2009 covering nearly all the Italian regions).
To this end, dedicated sampling tools were designed and revised. The evaluation teams are formed by volunteers coming from Cittadinanzattiva, but also by teachers, parents, groups of students, with the addition, in some cases, of school managers and directors of the School Prevention and Protection Service.
The teams, appropriately trained in the use of the tools, perform the monitoring within the schools which declared themselves ready to perform the survey. All the collected data are then sent to the national headquarters of Cittadinanzattiva, which proceeds with the preparation of a national Report and with the dissemination of the findings through a public presentation at a national level and a number of meetings in the different cities participating in the survey (the latest national report, presented on September 16th 2010, is available at http://www.cittadinanzattiva.it/imparare-sicuri/rapporto-imparare-sicuri-viii.html).
The evaluation structure of “Impararesicuri” decomposes safety in 4 components, 17 factors and 315 indicators. It is possible to couple each school building with a synthesis score, which shows the overall safety level.
The monitoring is performed by a pair of evaluators which have at their disposal two tools:

• the “structural observation grid” which is used to record the information collected during the inspection through direct observation or through simple questions to the personnel present (the topics are related to common paths, educational services, rooms, systems, general services and toilets, condition of the building, construction sites).
• the questionnaire for the Head of the Prevention and Protection Services aims at three objectives: to gather information on safety not detectable by direct observation, to gather useful elements to learn which knowledge persons in authority use to tackle safety issues; to gather general information on the monitored school.

The implementation of the “Impararesicuri” Campaign allows citizens to have at their disposal every year a wide-ranging and updated picture of the safety conditions of Italian school buildings. It also allows, based on rigorously collected and processed data, to address to the institution specific proposals for intervention in the fields of safety and quality improvement of the school premises, including specific undertakings and the performance measurable of concrete actions (the yearly monitoring activities are joined by other information and awareness initiatives, besides an award for good practices).

The Civic Evaluation of “Urban Quality”
The project is born from the joint efforts of Public Administration Ministry and Cittadinanzattiva in the framework of a Memorandum of Understanding entered in order to "develop innovative initiatives in the horizontal subsidiary field".
After an early evaluation testing performed by the citizens on the school services and the municipal front- office services, in 2009 a civic evaluation method of urban quality was ready (the workgroup included Public Function Department, Formez, Cittadinanzattiva, and Fondaca).
“Urban Quality” was intended as quality of the environments wherein citizens move, live, socialize, work. Components and dimensions of urban quality were identified through a national focus group including social research experts, civic organization representative and main figures of local authorities. The aim was to answer the following question: Which are the essential components with regards to “Urban Quality”? How can these components be declined in an essential way, so to identify accurate and measurable dimensions?
In this way, ten components of urban quality were shared (Safety, Connectivity, Sociality, Healthiness, Maintenance, etc.). For each component, quality dimensions were defined and for each of them some quantity/quality indicators were identified.
Experimentation involved 14 cities coming from four southern Italian regions: Campania, Puglia, Calabria and Sicily. In each city, it was decided to limit the observation scope through a “spatial” approach focusing the attention on a specific portion of urban territory so to allow the citizen to collect information in a direct, continuous and deep way through a given time (The spatial approach is generally used in the so-called safety audits, widely spread in Northern America. The approach provides that groups of citizens “walk” along a predefined urban path recording a number of factors which impact on the subject of the analysis).
Through a public solicitation, in each participant city a local team of evaluating citizens (around 10 members each team) was formed. The team identified its own coordinator who kept the formal contacts with the authority.
Each local team, after an appropriate training, was given the task to collect direct and indirect information. For direct collection a “grid” was prepared to be used as a real “notepad” along the survey course. The direct collection is subdivided in "one-off" observation, repeated observations and direct/older experience. The directly collected information were completed and integrated by an already available data set which was to be requested from the reference authorities and institutions.
To order and process the data, a Data Base was prepared, as well as the criteria for the consensual attribution of the score was defined.
The process provides, at the end, the preparation of a final report and the confrontation with each authority regarding concrete improvement actions (currently - September 2010 - the preparation of local reports is under way).

Civic Audit in Healthcare

Purpose and goals of Civic Audit
Civic Audit is a critical and systematic analysis of the performance of the health public authorities (With “health authorities” or “health organizations” we mean all public health organizations, including local health units and hospital authorities) promoted by civic organizations. It is therefore a tool at the disposal of citizens aimed at promoting the evaluation of the quality of the performance of the local and hospital health organizations.
The first experimental cycle of Civic Audit was started in 2001 aiming at defining and proving on the field the theoretical and methodological framework drafted in cooperation with 12 health local units. Starting from 2003, the possibility to endorse Civic Audit was extended to the universe of health organizations, and the number of involved public authorities constantly increased throughout the years (overall, the authorities involved, during years, amounted to 175, that is more than 50% of Italian health public organizations).
Today, it can be asserted that Civic Audit is established in the Italian healthcare service, as the main concrete form of evaluation in the public sector which features citizens as main players, in a framework of collaboration between civic organizations and health organizations.
The decision to provide active citizens with their own tool of evaluation of the action of the health authorities was born to meet three types of problems.
The first need was to give a concrete shape to the citizen centrality. Civic Audit was contemplating to overcome a reductive vision of the role of citizens as mere recipients of services. In fact, with respect to the more traditional methods or customer satisfaction surveys, in Civic Audit the citizen is no longer a pure subject of survey, but he becomes the evaluating subject who visits premises and interviews the people in charge. The centrality of citizen is exercised therefore in the concrete definition of specific criteria for the design and the evaluation of services and health policies.
The second need was to make the action of health public organizations more transparent and verifiable. In 1980, Cittadinanzattiva founded the Patients' Rights Tribunal, a network of offices throughout the country, in hospitals and health structures, aimed at collecting reports and inputs coming from citizens. This long experience was the base founding development of the concept of making the opinion of citizens even more significant in the view of changing the health system. From this point of view, Civic Audit is also one tool to increase the accountability level of health authorities and their directors.
The third essential reason was to create an evaluation procedure which was local but was also founded on homogeneous and comparable criteria, so that also the disparity and fragmentation degree of the National Health Service could be valued. We will see, indeed, that Civic Audit is based on a unified indicator system which makes the performance of the authorities easily comparable in a benchmarking perspective.

The Evaluation Structure of Civic Audit
The design of Civic Audit started from four questions, representative of as many aspects of the common citizen experience with regards to health services.

• The first topic focuses on the citizen as user of the services, involved in prevention, diagnosis, care and rehabilitation process. "Which were the actions promoted by health authorities aimed at concretely putting citizens and their need at the centre of their interest?"
• The second topic interests the citizen, ill and suffering from serious or chronic pathologies: “What priority was given to policies having a significant health and social relevance, such as risk management, pain management and the support to the chronically ill?”
• The third issue is related to the exercise of the citizenship rights and entails the need to ask if  “The participation of citizens is considered by health authorities to be an essential resource for the improvement of health services or is promoted (if any) only like a formality provided by some laws?”
• The fourth issue regards the citizen and the community in which he/she lives: “What answers were provided by the health authority to a problem deemed as urgent by the local community?”;

While the fourth question is related to local issues, the first three are valid for all the country and allow to organize the evaluation structure in 3 evaluation components and 12 evaluation factors, according to the following scheme.
Orientation to citizen:

• access to health services;
• protection of the rights and improvement of quality;
• customizing of care, privacy and assistance to patients;
• information and communication;
• comfort

Commitment of the authority in promoting some "policies" which are particularly relevant to health and society:

• patient safety;
• safety of premises and plants;
• chronic illnesses and oncology;
• pain management;
• prevention

Involvement of civic organizations in the policies of the authority.

• implementation and operation of the user participation practices;
• other forms of citizen/health authority participation and interaction.

Each of the twelve factors above is investigated through the detection of a set of indicators that are the same for all the organizations (overall, 380 indicators were detected).
The evaluation structure of the Civic Audit is completed with the definition of the application levels, the areas of the National Health Service wherein the detection of the indicators is performed. The planned and applied levels are three, until now:

• the authority area (the health organization area in the whole);
• the hospital assistance area;
• the primary care area, comprising:
• the fundamental health assistance (districts, family medicine, domicile care),
• the territorial specialized assistance (health centres),
• the territorial and semi-residential specialized assistance (in particular, mental health and drug addiction services).

For the collection of data, the team employed 5 types of questionnaires aimed at the directors of the authorities involved in the Civic Audit and 6 different check lists for the direct observation performed by auditors (citizens and health operators).
All the indicators are coupled to a recognized standard (The sources used for the standard recognition were the recommendations of the international institutions, the regional and national regulatory guidelines, the Patients' Rights Service Charters and the recommendations of the scientific societies) and it is therefore possible to calculate, with easy steps, for each level and factor, a "Standard Adequacy Index" (IAS). If the standard is completely met, the indicator has a 100 score, zero if opposite. The weighted average of the scores of the indicator groups is the IAS value. The difference between IAS and 100 measures the gap between the detected situations and the full achievement of all the standards.
The study of IAS is the base of the local evaluation and of the benchmarking and allows to perform system analysis having a particular value in national and regional application cycles.

The Evaluation Process and the ways of Citizen Involvement
As we have seen above, Civic Audit is not a mere collection of information, but a civic evaluation process promoted by a civic organization and approved by health authorities, which can be subdivided in four main steps: preliminary operations, preparation, execution and final actions.
Preliminary operations to the audit execution in a health authority are two: formalization of the participation to the program and the training of the people in charge.
In order to participate in the Civic Audit program, the partnership between the Authority General Direction and the Local Director of the Patients Right Tribunal is formalized. The parties (public and civic) proceed, then, to appoint their respective persons in charge who will have the duty to guide the realization of the Civic Audit program.
In some cases (currently eight) the Region itself is interested to the implementation of the Audit in all the health organizations and enters directly in a convention with Cittadinanzattiva.
The authority and civic people in charge attend the regional training course on the Civic Audit, wherein the methodological system, the tools employed for data collection, the operation cycle to be followed and the participation procedures are presented. Participation to the training course is compulsory for the access to the Civic Audit program.
The evaluation teams (a number between a minimum of 8 and a maximum of 20 people) will be composed by citizens and by service operators.
Citizens are selected through public solicitations, which give anyone interested the possibility of participation. The service operators are chosen by the agency Director of the authority.
The team members are prepared for Civic Audit with a training module cared by the people in charge who have participated in the regional training. The operative team makes use of the technical assistance from the national and regional headquarters of Cittadinanzattiva.
Each team defines a local Plan of Civic Audit which includes:

• the definition of the survey field, that is the detailed list of the structures that will be subject to the analysis, the names of the people in charge to be interviewed and the indication of the operators responsible for receiving the teams;
• indication of names of people in charge of specific data collection operations (distribution of questionnaires and direct observation);
• data collection calendar.

The execution stage comprises the operations of data collection and their return on electronic media.
Based on collected data, the team prepares a local evaluation Report divided in four parts: data analysis, brief evaluation report, plan for the elimination of the non-conformities and the plan for the corrective actions.
The national or regional headquarters of Cittadinanzattiva prepares a final report at the end each Civic Audit application cycle, comprising the analytic illustration of the benchmarking tables, discussion of main elements highlighted by data analysis and improvement recommendations addressed to different interlocutors.
Roughly a year after the conclusion, a verification on the actual results is planned and, in particular, a verification on the degree of implementation of the plan for the elimination of the non-conformities and the plan (In particular there were performed a focus group and a questionnaire with around 15 particularly significant realities, a research on the websites of the health authorities and Regions, an in depth examination about the Rome health organizations and a report of the Emilia Romagna Health Agency regarding the regional experience of Civic Audit) for the corrective actions. Usually, this verification is joined by a new cycle of Civic  Audit.

The Civic Audit Effects
In the past, Cittadinanzattiva performed a few analyses to evaluate the real effects and impact produced by ten years of experimentation and performing of Civic Audit in the Italian health authorities. These analyses gave four main results.

First, Civic Audit proved to be a participation tool itself, which opened new relationship channels between citizens and health institutions. The regional public solicitations which invited the citizens to be part of the enforcement of Civic Audit received hundreds of applications. In total, during the years, around 3.000 citizens could actively participate in the evaluation of health organizations. A direct involvement in the evaluation of the services which allowed also to revitalize or to improve other forms of consultation and participation already set forth by the law, but frequently reduced to mere formalities (mixed consulting committees, participation conferences, the Service Conferences, etc.).
A second impact is more cultural. The Civic Audit has in part contradicted the reductive visions which consider citizens lacking the required expertise to attend the public issues.

The most evident confirmation comes from the declaration of the European Charter of Patients' Rights in 2002 which was followed by a monitoring on the degree of implementation of the 14 rights performed in 14 countries employing a Civic Audit inspired method. This work won, in 2007, the first prize of the

European Economic and Social Committee as the best initiative of the civil society and became a benchmark, which found confirmation in the official documents of the same Committee and of the European Parliament and contributed to the decision to issue a European directive on patients' rights.

Moreover, the joint work of the citizens and of the operators in the area of local teams and regional coordination groups aided the updating of the respective cognitive models. The discussion of the evaluation reports and of the improvement plans, with the required sharing of the criteria of reading and evaluation of the data led to share a new way to consider health service.

During these ten years, a "critical mass" of around 100/150 people were formed (between local people in charge of the Civic Audit, head of the Public Relation Offices and Quality Services, Health Authority Directors) which are particularly active and involved in ensuring the Civic Audit implementation. Their activity significantly improved and enriched the procedure system of the Civic Audit, provided useful elements for the revision of the evaluation structure completed in 2009, promoted the circulation of good practices, gave its contribution to the definition of the criteria of interpretation of the data and identification of corrective actions.

A third significant element has been the progressive evolution of Civic Audit from a strictly local dimension related to the single local authority to a more regional vision. During the time, actually, thanks, above all, to the possibility of benchmarking, Civic Audit was used to support the regional policies as well. For example, it was joined to the structure accreditation processes, to the evaluation of the General Directors of the authorities, to the establishment of the Public Relation Offices and of the Quality Services or to the revision of the Service Charters.

Lastly, from single authorities’ point of view, the Audit is a tool starting from which concrete and verifiable improvement actions have been started in many realities. In general, they are interventions of adjustments to the proposed standards starting just from the critical points arisen from the evaluation. This is the area where the effects of the Audit can be more tangible and immediate. The ability to translate the civic evaluation in real improvements in the quality of health services depends, in wide part, on the ability of the health agency to exploit the Audit results in its planning and management processes of the services and from the collaboration relationship established with the civic organizations.

Obstacles and Development Prospects of Civic Evaluation in Italy

The Civic Audit represents, even now, the most complete experience of civic evaluation realized in Italy. In spite of this, as already seen, there are other Public Administration intervention ambits affected by direct evaluation processes by the citizens. In general, we can confirm that currently in Italy many national and local institutions are taking an interest in the civic evaluation approach.
What generally interests them is not the instrumental and technological apparatus of civic evaluation, which is necessarily not particularly sophisticated, given that it must be easy for the non-professional evaluation team to use.  The institutions instead appear to be interested in the innovative approach and the active role given to citizens during the different phases of the process. It is necessary, however, to carefully analyze the reason for this interest in the specific Italian context.
Italian Public Administration is characterized by a wide-spread weakness in systems for evaluating and measuring performance, and by evaluation of public policies that are not yet consolidated.
Certainly, important experiences of innovative public administrations exist, but mainly at a local level, where very advanced evaluation systems have been consolidated for some years (public policy evaluation, service quality evaluation, personnel evaluation, etc.). These are generally those administrations that have also developed the best planning, programming and control systems, those that are the most transparent and most interested in allowing their accountability level to grow, and those who are more careful in involving stakeholders. In these ambits, civic evaluation can bring greater completeness, making it possible to better integrate the point of view of the citizen in the local governance processes.
Where the administration does not have evolved government systems, however, the attention to civic evaluation processes can have an ambivalent meaning. On the one hand, it can indicate a new opening and desire to put the citizen at the centre of change. On the other hand, it can be a form of compensation for the absence of other instruments, as if the administration was delegating a duty of its own to the citizen, being that of assessing reality and formulating proposals for action.
Looking at it in another manner civic evaluation is, for a conscious public administration, an additional resource for pursuing a model of good government. Public administration that is weak in the governance and management processes involving the citizen can risk becoming an end in itself, a more ideological or formal option rather than a substantial contribution to the production of knowledge that is useful for deciding and achieving a real change.
The theme, in any case, is particularly complex and requires the realisation of ad hoc surveys to evaluate the concrete impact of civic evaluation processes on the operation and performance levels of the public administrations.
In this text we will limit ourselves to indicating in more general terms the factors which, in our opinion, can obstruct or, on the contrary, favour, civic evaluation in Italy as a useful instrument for changing and improving institutions.

Obstacles
The difficulties that civic evaluation meets in Italy can be divided into factors that are inside and outside the evaluation process itself.
The first internal factor is of a purely technical nature. Carrying out an evaluation process presupposes both the willingness and the resources (material and immaterial) that are not always easy to find, even from an economic and financial point of view. Civic organisations are largely based on the voluntary and spontaneous activity of citizens who only occasionally transform themselves into “analysts”. It is therefore necessary to find a balance between the need to maintain this voluntary and spontaneous characteristic of participation and the need to make the evaluation activity “more specialised”, defining procedures, instruments, rules and organisation. Should the investment be too low, the risk arises of the evaluation activity being “weak” from a technical and methodological point of view, and thus not very efficient. If the investment is very high (for example in terms of planning, citizen involvement, training, communication, etc.), the civic organization that supports it must also be able to support it. This implies the choice of a “technical quality threshold” for evaluation, below which it is opportune not to fall and over which it is difficult to go with the resources that are available.
A second internal limit to evaluation can be of the perceptive type. Organised citizens normally feel that their action has an effect on reality much more than is really true. In a health ambit, for example, the teams that carry out the Civic Audit can easily feel themselves at the heart of health system operation and overestimate the impact of their activity. What they concretely generate on the services analyzed is probably lower than that expected or perceived.
This cognitive distortion can be in part resolved by reinforcing the evaluation of the evaluation itself, in other words dedicating special attention to measuring the efficiency and real impact of the civic evaluation processes.
There are also external obstacles to civic evaluation development, these being essentially of the political and technical type. The relationship between institutions and citizens (therefore also the availability of the citizens to participate in the evaluation) in Italy still seems to be dependent on the electoral cycle. The changes made to government organs, following an election, can have an important and at times unpredictable effect on the continuity and development of civic evaluation projects. They can be interrupted, suspended, or on the contrary promoted and reinforced, more according to the sensitivity of individual figures (for example a politician, manager or services manager) than on respect for the principles of institutional continuity. In reality, the activity of civic evaluation should not be affected by the electoral cycle, proposing itself instead as an instrument of knowledge that supports institutions, and not this or that political party.
The last obstacle, previously mentioned, and which is probably the most important, is the weakness of the management and evaluation systems present in Italian Public Administration, which inhibit the development of systems that are more evolved and transparent in measuring and comparing the results. The data and information are not easily accessible, and in many contexts, evaluation is seen as a type of control rather than as an opportunity to learn. It is a cultural factor, overcame only partially by the reform processes that have taken place during the last twenty years.
In reality, what happens today is the fragmentation and differentiation of accountability levels and managerial strata of Public Administration. As already mentioned, more evolved contexts express a more informed demand for citizen participation and are open to civic evaluation. Indeed, in these contexts, it is necessary to guarantee better technical quality of the evaluation itself, to ensure that the information produced has a true additional value opposed to the information already available and generated by the administration.
On the contrary, the less advanced ambits are less sensitive to civic evaluation or are not able to absorb the knowledge produced, even while making themselves available for evaluation. These are the contexts where ideological drift and legitimisation of citizen participation as an end in itself is easier than making a solid mark on reality.

Opportunities
In the face of these difficulties, it is necessary to highlight that all the most recent Public Administration Reform interventions in Italy have evidenced the need to measure performance, transparency, reporting and participation.
The most recent is Italian Legislative Decree no. 150/2009, which deals with the adoption by public administrations of new instruments, such as the Performance plan, the Performance report, and the Three- yearly programme for transparency and integrity.
In practice, this decree indicates that the evaluation, reporting and transparency of public administration actions are no longer voluntary and sporadic initiatives of the individual body, but obligatory phases that shall be integrated into a performance management system on which the governing action of the same administration rests.
This new “prescriptive space” for evaluation and transparency seems to be an additional answer to the continuing request for accountability, change and governance in Public Administration that has matured during recent years in Italy. Indeed, even though in a general frame of institutional delegitimization and mistrust in politics, the demand for public services (above all in essential services such as health, education, and justice) remains to the fore in the mind of Italians. The rapid development of some civic evaluation processes and the involvement of thousands of citizens bear witness in part to this desire for change and participation. In Italy, in short, citizens still seem to have faith and expectation in change, from which it is possible to follow roads that improve politics and public services.
From this point of view, civic evaluation has in itself a great development potential, because it mixes the production of information with its use for action. It is a type of social research that has installed within itself the objective of producing a change in reality and the means to achieve it, given by the desire to participate and become protagonists on the part of the citizens. Faced with a request for change and a weakness in the replies of the institutions, citizens, in partnership with the institutions themselves, can supply useful knowledge for learning, in other words for the construction of a new model of social relations and governance of complex problems.
For this purpose, the final positive element is the specific competence that some civic organisations have themselves developed. The easy access to and circulation of information can today favour, even inside civil society, a more rapid spreading of knowledge and the learning of new tasks. Organised citizens carry knowledge and qualified experience and they no longer correspond to the old image of passive beneficiaries of interventions that are decided on and realised by others. Citizens, if organised, are always more able to analyze and judge reality, with their own instruments. Civic evaluation, from this point of view, can be an instrument for channeling and finalizing new forms of citizenship.

Document written by:

• Angelo Tanese, Director, Civic Evaluation Agency, Cittadinanzattiva (Italy) This email address is being protected from spambots. You need JavaScript enabled to view it.
• Alessio Terzi, President, Cittadinanzattiva (Italy) This email address is being protected from spambots. You need JavaScript enabled to view it.

(Updated in September 2010)