How to measure the impacts of participatory innovations on the quality of democracy - the framework
Frameworks for evaluating democratic innovations are rarely spelled out. Although Sewell and Philips called for a ‘concise research agenda’ as early as 1979, few authors followed their call (Abelson and Gauvin 2006; Rowe and Frewer 2004: 521 ff.). These authors applied different approaches: some of them developed criteria by relating to the perspectives of the participants: ‘success must be defined and judged by those involved’ (Moore 1996: 168). Accordingly, criteria refer to impacts on the participant and the group, for example the improvement of political knowledge and enhancement of civic skills (Carnes et al. 1998: 390; Moore 1996: 156; see also Fishkin in this volume).
In the European context, the criteria ‘legitimacy and effectiveness’ attracted major attention in the wake of Scharpf’s work. In his book Governing Europe: Effective and Democratic? Scharpf (1999) argues that the quality of democratic systems can be measured first by input-legitimacy and second by effectiveness. Several European authors refer to these criteria in their evaluation of democratic innovations (for example, Holtkamp et al. 2006).
Other scholars follow a different line, applying criteria such as openness, acceptance, quality of deliberation, publicity, representativeness, access to resources, early involvement, accountability, context, process, transparency, accountability, outcome or resolving conflicts (Papadopoulos and Warin 2007: 455 ff.; Abelson and Gauvin 2006; Dalton et al. 2006: 14; Rowe et al. 2004: 93; Rowe and Frewer 2004; Beierle and Cayford 2002: 6; Renn et al. 1995). Similarly, in this volume, different frameworks and criteria have been applied (see the chapters of Budge, Fishkin and Smith).
Not all, but many of these criteria, however, face five crucial problems: first, they are either too abstract to be used empirically; second, they lack any theoretical background. Whereas, for example, Renn et al. (1995) discussed theoretical questions at length, they addressed practical, operational questions only very briefly. Other authors, such as Moore (1996), seem to have little interest in theoretical reflection. Third, some of the criteria are problematic. For example the criteria ‘openness’ (Papadopoulos and Warin 2007) is tricky. ‘Open’ innovations, in which participants recruit themselves, might provide less inclusive participation than ‘closed’ innovations with selected participants. Or, as Dalton et al. (2006: 262) put it: ‘Equality of access is not sufficient if equality of usage is grossly lacking.’ ‘Publicity’ as a criterion is another example. Why should a high level of publicity enhance the quality of democracy (see Smith in this volume)? These examples lead to the fourth problem: ex ante and ex post impacts are often not differentiated properly. Impacts that are predisposed by design a priori and impacts that can only be measured after the end of the procedure are grouped together. For example, the fact that consultative procedures are just consultative without decision-making competency is part of the design and not an ex post impact - it was mostly determined from the beginning that they will have no effect on public policies. Fifth, and this might be the most important problem, criteria for the evaluation of success and prerequisites for success are often confused: for example, criteria such as ‘access to resources’ or ‘early involvement’ (see Rowe et al. 2004) might be favorable conditions for a successful participatory procedure. However, they cannot serve as criteria for the evaluation of the impacts of a participatory innovation. Similarly, ‘context’ (see Abelson and Gauvin 2006) is necessary to identify favorable conditions for successful innovations, but cannot be a criterion to measure success. It is fundamental to distinguish between impacts (or success, in most studies) and prerequisites for success (see Smith in this volume). Without doubt, impacts and prerequisites are linked, but for a clear analysis, both must be scrutinized separately.
Thus, the question arises as to how a framework can be developed without stepping into these pitfalls. It might be helpful to consider not only research on democratic innovations, but more general research on the quality of democracy, since the question of how to measure the quality of democracy has recently gained popularity (for example, Diamond and Morlino 2005). In the debate on the quality of democracy, three lines of argument can be identified. One line in the literature examines the procedural aspect, namely input-legitimacy and democratic process. The second line in the literature adds the outcome of democracy. These authors agree on the notion that democratic input and procedure is not enough and that a democracy must also fulfill collective goals of a constituency (for example, Scharpf 1999). However, a third, often forgotten line, has already been mentioned by Thomas Jefferson (1776). He, like many others, insisted that democracy should also generate an informed, enlightened democratic citizenry (see also Fung and Wright 2001; Dahl 1992).
This classification provides the outline for the framework of this study. I suggest a framework based on the following four dimensions: (1) input-legitimacy, (2) democratic process, (3) effectiveness, and (4) civic education. All aforementioned topics coalesce and relate to these four criteria in one way or another. However, all four criteria are complex concepts and require further explanation, as outlined in the following paragraphs.
168 Brigitte Geissel Input-legitimacy
Input-legitimacy refers to the input-side of the political system (‘government of the people’). In representative democracies, this means that citizens are equally and significantly involved. It was the main promise of representative democracy to provide political equality, expressed as ‘one man, one vote’. However, more and more citizens are drawing the conclusion that equal political input via election of representatives is a myth and is not reflected in reality. As mentioned before, many participatory innovations have been established to counter these feelings of political inequality, which are often combined with political distrust and alienation.
In fact, the main arguments for and against participatory innovations focus on the question of inclusive participation: proponents argue that participatory innovations would attract a multitude of citizens who do not get involved in elections and other forms of traditional participation. Opponents reason that only the politically active strata of society engage. So, the questions to be answered are whether participatory innovations guarantee de facto inclusive and equal involvement, or whether participants stem from the ‘well-off’, already politically active strata of society (see Smith and Talpin in this volume).
Legitimacy can, however, also be measured by a second criterion: legitimacy, which refers to citizens’ political support (‘perceived legitimacy’). Several proponents of democratic innovations argue that citizens would accept their representatives, the political system and political decisions with more enthusiasm if they were involved in the political process. Thus, one criterion of evaluation should be whether participatory innovations improve ‘perceived legitimacy’.