Democratic quality indicators

Indicator system to measure quality indicator in Decidim.

Introduction

Democratic quality is a core principle of Decidim. To uphold this, we have developed a system of indicators that help evaluate the quality of participatory processes on the platform. This article explains how these indicators are calculated.

How Are Indicators Calculated?

We have defined two methods for collecting quality indicators:

1. Automatic Metrics: These are calculated directly from Decidim platform data.
2. Self-Assessment Questions: These are based on a form filled out by an administrator.

Each indicator is expressed as a percentage (n/total). This percentage is then converted to a score on a 1-to-5 scale:

• 1 = 0–20%
• 2 = 20–40%
• 3 = 40–60%
• 4 = 60–80%
• 5 = 80–100%

Automatic Metrics

These metrics are automatically calculated from data available in Decidim. They are evaluated per participatory process and can be made public by administrators, so all participants can access it.

We have defined four main indicators: Citizen Influence, Hybridization, Responsiveness and Traceability:

Citizen Influence

This indicator measures the level of citizen influence on the accepted proposals and the outcomes of the process.

• Citizen influence on proposals
  • Percentage of proposals accepted (n/total)
  • Percentage of approved citizen proposals over total of citizen proposals (total citizen influence on proposals approved)
  • Percentage of approved citizen proposals over total of approved proposals (relative citizen influence on approved proposal)
• Citizen influence on outputs/results
  • Percentage of results linked to citizen proposals out of the total of results (total citizen influence on outputs)

Hybridization

This indicator measures the level of combination between online and offline spaces during the process.

• Is a meeting component active? (Yes=5, No=0)
• Are there face to face meetings? (Yes=5, No=0)
• Are there online meetings ? (Yes=5, No=0)
• Are there hybrid meetings? (Yes=5, No=0)
• Are there proposals created from meetings? (Yes=5, No=0)

Responsiveness

This indicator measures the level of responsiveness of the organization to participant input. It reflects their engagement through official responses and implemented decisions.

• Percentage of proposals with answers (n/total) – Proposal institutional responsiveness
• Percentage of results completed at 100% (n/total)

Traceability

This indicator measures the transparency and traceability of the participatory process. It reflects how easily participants can follow the evolution of proposals and verify institutional actions through published records.

• Number of Proposals with history (n/total)
• Percentage of results linked to proposals (n/total)
• Percentage of meetings closed with minutes published (n/total)
• Activation of the open data module (dichotomic: 0 or 5)
• Percentage of proposals related to budgets (n/total)

Self-Assessment Metrics

In addition to automatic metrics, we propose the use of a self-evaluation form to assess the qualitative dimensions of each participatory process. This form will be completed by platform administrators at the end of each process, and the responses will be publicly displayed in the indicators content block.

These questions aim to capture aspects of democratic quality that cannot be assessed through quantitative data alone. It is important to note that the responses to this survey are biased by the opinion of the platform administrators.

Self-assessment questions

The questions that make up the self-assessment form answered by the administrators, as well as the values associated with each answer, are set out below.

Informativeness

1. Does the published information provide clarity on the decision-making mechanisms (e.g., voting, consensus, direct negotiations)?
   • No clarity = 0
   • Low clarity = 1
   • Sufficient clarity = 2,5
   • Full clarity = 5
2. Has information been provided regarding the participants' capacity to influence the topic under discussion (both in the communication of the process and invitations, as well as at the beginning of each debate session)?
   • Never = 0
   • Almost never = 1
   • Usually = 2,5
   • Always = 5

Citizens influence

1. Assessment of the degree of participation in the process
   • Participants received information from the organisation. = 1
   • Participants were asked in a consultative manner or gave feedback to the organisation. = 2
   • Participants were involved to share decision-making power. = 3
   • Participants were involved decisively in the decision-making. = 4
   • Participants were involved in the decision-making and the management of resources or the implementation. = 5
2. Percentage of phases with 0-5 citizen decisional intervention.

   Out of all phases in the process (from selecting issues to supervising execution), in how many does the public have the ability to make decisions or take executive action?

   • 0-20% = 1
   • 20-40% = 2
   • 40-60% = 3
   • 60-80% = 4
   • 80-100% = 5

Inclusiveness

1. How is the degree of diversity of the agents and participants assessed with respect to the population of reference in the process?
   • None = 0
   • Low = 1
   • Sufficient = 2,5
   • Good = 5
2. In how many languages has the process been communicated?
   • One language = 1
   • Less than half of the languages spoken in the community = 2,5
   • More than half of the languages spoken in the community = 5
3. Were the venues of the face-to-face meetings accessible to people with reduced mobility?
   • No = 0
   • Partially = 2,5
   • Yes = 5
4. The participation sessions have been scheduled at varied times (morning, midday, afternoon, evening, and weekends).
   • No = 0
   • Partially = 2,5
   • Yes = 5
5. Has digital support been offered to the participants?
   • No = 0
   • Partially = 2,5
   • Yes = 5