Introduction Section

Citizen participation is instrumental in promoting transparent and accountable government, and in fostering the norms that underpin a functioning democratic system. Democracy develops and deepens when citizens are active and exercise their right to be heard, and when they use organized, informed action to hold government accountable for making real improvements in people’s lives. 

Citizen-led political process monitoring can be used to promote more open and accountable government, improve government performance and increase citizen engagement. By systematically gathering information and reporting on political processes, such as campaigning for elected office, public budgeting and procurement, lawmaking, or follow-through on policy implementation, civil society organizations (CSOs) and activists can offer assessments, provide data-driven recommendations, and lay the foundation for greater citizen influence over political processes and outcomes. 

Political process monitoring initiatives can be implemented by diverse civic actors, ranging from national-level watchdog organizations to local-level groups concerned about quality-of-life issues. These initiatives can focus on a range of processes for different purposes depending on the identified governance gaps. This guide supports five types of monitoring: campaign monitoring; parliamentary monitoring; monitoring government follow-through; budget monitoring and expenditure tracking; and shadow reporting.

Monitoring the Information Space

Monitoring misinformation and disinformation in political processes is increasingly important, particularly in campaign-related monitoring. Both social media and traditional media, online and offline, are vectors for spreading misinformation or disinformation narratives. Social media in particular is a significant source of information for citizens, where unfiltered and unverified information and minority opinions can become disproportionately spread and incorrectly accepted as factual. While many of the most successful and reliable fact-checking initiatives have been undertaken by researchers, independent media or trained journalists, CSOs can also play a critical role, often complementing fact-checking initiatives1 with firsthand knowledge of an issue, community or geographical area. While monitoring misinformation and disinformation is outside the scope of this guide, several tools and guides, particularly for campaign-related monitoring, can be found below.

Drawing from NDI’s research and experience supporting local partners engaged in political process monitoring, this guidebook provides practical approaches that civil society actors can use to initiate or enhance monitoring activities. It also focuses on the fundamental knowledge and skills needed to conduct a strategic and politically astute monitoring initiative. 

This NDI guidebook takes a fresh look at political process monitoring by incorporating technology approaches and recognizing the digital era’s emerging opportunities and threats. The rapid rise in the use of technology among citizens and civil society organizations is increasingly complementing citizen activism and challenging institutions and paradigms in ways that directly affect the relationships between governments and citizens. New technologies offer the possibility of strengthening citizens’ voices in politics and governance, creating political spaces for activism, and promoting increased government transparency and accountability. Over the last decade, the use of information and communication technologies for social and political purposes has rapidly increased, and as a result programs can engage those citizens who are online, taking advantage of the powerful capabilities of the devices for new forms of monitoring, communications and analysis.

Acknowledgements

NDI prepared this updated guidebook with the assistance of civil society leader Jetmir Bakija, and includes significant contributions by Daniel Arnaudo, Aaron Azelton, Priyal Bhatt, Chris Doten, Jesper Frant, Mario Mitre, Madeleine Nicoloff, Jessie Steinhauer, Evan Summers and Moira Whelan. NDI’s Equal Rights in Action Fund team and staff in Albania, Kenya, Moldova, Myanmar, and Ukraine were essential in helping to identify relevant case studies. A special thanks also goes to Eden Beck for her brilliant copy edits. 

The guidebook builds on two earlier resources by reflecting emerging lessons and responding to practical challenges and opportunities, especially those posed by changing technology. 

  • Kourtney Pompi and Lacey Kohlmoos. Political Process Monitoring: Activist Tools and Techniques. Washington, D.C.: National Democratic Institute, 2010. 
  • Political Process Monitoring: Considering the Outcomes and How They Can Be Measured. Washington, D.C.: National Democratic Institute, 2012.

Introduction: Section 1

Political Process Monitoring Defined

Political process monitoring involves a methodical effort to track the actions of government actors or institutions and objectively assess alignment with established democratic principles, obligations, commitments or standards. Typically, political process monitoring initiatives gather and track information over an extended period of time to draw trends, gauge progress, and elucidate findings and recommendations.

The Purpose of Political Process Monitoring

Political process monitoring can be used to promote more open and accountable government, improve government performance and increase citizen engagement. By systematically gathering information on political processes, such as campaigning for elected office, public budgeting and procurement, lawmaking, or follow-through on policy implementation, civil society organizations and activists can offer assessments, provide data-driven recommendations, and lay the foundation for greater citizen influence over political processes and outcomes. Political process monitoring can raise public awareness of governance issues and fuel broader organizing efforts to influence government behavior. For example, political process monitoring can serve as an integral tactic used by those advocating for public-sector reforms or improved service delivery. Although some CSOs might concentrate only on conducting monitoring activities, many organizations and activists treat monitoring as a means to an end, rather than an end in itself. Typically, monitoring initiatives open up other opportunities and entry points to create change through further, sustained action. In this respect, it is important to determine how different forms of political process monitoring fit within the broader civic organizing and activism ecosystem, and how monitoring may complement other efforts.

Political Process Monitoring Model

There is no “one size fits all” model for political process monitoring initiatives. Countries have different systems, structures, legal frameworks and political norms, as well as different democratic histories. Therefore, monitoring strategies, techniques and timelines need to be tailored to local contexts and the identified governance gaps, with a realistic sense of the political circumstances. There are a number of key considerations and steps that surround each type of political process monitoring listed below. For instance, there will always be a need to determine if monitors have access to the respective political process or to requisite information that would allow for a fair assessment of the process. Likewise, all forms of monitoring require the development of objective metrics that can be used in the assessment. These could include anything from whether a parliamentarian regularly attends plenary sessions to whether a public procurement process resulted in multiple competitive bids. Monitoring groups also need to share the metrics and present the purpose of their monitoring, as a means to build credibility and create appropriate public expectations. In this regard, monitoring groups should strive to build working relationships with politicians and government officials, so that monitoring is viewed as a constructive contribution to democratic functioning and a legitimate activity for civil society.

Theory of Change

Political process monitoring involves methodical efforts to track the actions of government actors or institutions and objectively assess alignment with established democratic principles, obligations, commitments or standards. When civil society actors undertake such monitoring, they open the processes to public scrutiny and input. In new and emerging democracies, the monitoring can help establish the imperative for citizens to have an active voice in shaping political processes, enlarge and maintain the space for political engagement, and foster stronger accountability relationships with public officials and political leaders.

NDI’s political process monitoring theory of change is illustrated in the diagram below, which depicts a dynamic relationship between monitoring and developing democratic governance processes that work in the public interest.

Theory of Change

Preconditions for Political Process Monitoring

What are the preconditions for political process monitoring? In many countries, government and public institutions are closed to the public, organizations and anyone outside of government officials.

There may be various reasons for this. It may be because there is no tradition, or at least no recent tradition, of an open society. Government or institutions may have concerns about opening up to citizens, such as how officials would appear if they were more open to their citizens, how this will affect their work, how they will be judged, and whether this will make the work of monitoring quite difficult.

At a minimum, there should be some public information to work with, some citizen interest in the political process, and some level of government response to CSOs’ work, at least to respond positively to a request for a meeting to tell them about the anticipated work. Ideally, there should be full transparent access to public documents and responses to CSOs’ issues and requests.

Illustrative Outcomes

The following chart of development outcomes represents what can be achieved through different types of political process monitoring initiatives. The chart organizes the illustrative outcomes along three dimensions: voice, space and accountability.

DESIRED OUTCOMES
TYPE OF MONITORING VOICE SPACE ACCOUNTABILITY
Campaign- Related Monitoring
  • An electorate more informed of candidates’ backgrounds, campaign platforms and voting records
  • A public record of campaign promises
  • Constructive relationships between citizens and elected officials
  • Citizen priorities taken into account throughout the campaign season
  • Elected officials held accountable for their campaign promises post-election
  • Policy priorities of both candidates and elected officials more accurately represent citizen interests
Parliamentary Monitoring
  • Citizen access to reliable information on the functioning of legislative processes at the national and local government levels
  • Citizen access to reliable information on legislators at the national and local government levels
  • Direct, constructive interaction between citizens and legislators at the local or national levels
  • Direct, constructive interaction between citizens and political parties at the national or local levels
  • Increased government and political party accountability to citizens
  • Improvement in legislative capacity and individual legislators’ performance at the national or local levels
Government Follow-Through
  • Increased public awareness of the extent to which policies or laws are implemented and enforced
  • Increased public awareness of the extent to which public officials are in compliance with a government agreement
  • Improved ability to influence government based on monitoring findings
  • Enhanced understanding of how government follow-through on decisions can affect democratic governance
  • Increased constructive cooperation between governments and CSOs concerning the implementation of government decisions
  • Increased compliance with government agreements
  • Increased implementation and enforcement of public policies and laws
Budget Monitoring / Budget Advocacy / Expenditure Tracking
  • Increased citizen capacity to hold governments accountable
  • A citizenry more engaged in political processes, especially at the local level
  • More transparent local budget processes and expenditure management
  • Improved communication between civil society, government and citizens
  • Improved service delivery and public infrastructure projects
  • Decreased levels of corruption
  • Improved service delivery and public infrastructure projects
Shadow Reporting
  • The United Nations (U.N.), national government and the partner have more accurate and in-depth information on government compliance with international treaties, conventions and accords
  • Increased collaboration between governments, CSOs and political parties on policy development and implementation
  • Increased political party and government accountability and responsiveness
  • Gaps in government policy development and implementation identified and addressed

Types of Political Process Monitoring

Public officials and institutions are involved daily in various processes that affect the country and its citizens. These processes are political in the sense that various stakeholders may want to influence the functioning or outcome of a process, and citizens want their views and needs represented and included. Political processes begin from electoral campaigns and extend through legislation and the implementation of government policies, decisions and programs.

Surveys and Selection Bias

Surveys, focus group discussions and online forms are useful for data collection and can be leveraged for advocacy. Simple surveys can provide basic information about citizen needs and interests, and national surveys using representative sampling methodologies can capture snapshots of public opinion. Legislators can use that information to shape policies that reflect the respondents’ priorities. Additionally, the information can become the basis for dialogues or public forums that foster more direct citizen-legislator and citizen-political party interactions. However, there is a risk that selection bias might skew the results of these tools. Surveys that don’t use random, representative samples might not represent the interests of all citizens. For example, a survey that uses a public webform to collect data will only represent the opinions of those with access to the internet. Similarly, women and marginalized groups are often unintentionally excluded from surveys because targeted outreach is required for these voices to be represented. Thus, findings might be useful to spur conversations but may carry risks if they are used as a basis for new policy. It is critical that the data collection methodology avoid selection bias and ensure inclusivity by surveying all available data and by making certain that surveys are representative of the broader population, e.g., with a public opinion poll,* or door-knocking with the original survey technology — pen and paper.

* Polling firms are usually needed to conduct public opinion polls, which are time consuming and involve complex statistical analysis.

 

The remainder of this section explores the five types of political process monitoring by broadly describing each type’s purpose and approach.

Campaign-Related Monitoring

Definition

Campaign-related monitoring uses elections as an opportunity for civil society to encourage political parties and electoral candidates to take explicit positions on issues and make commitments about the work they will undertake if elected. This information is then used by civic groups to monitor the performance of elected officials during their term of office.

Monitoring Purpose

Campaign-related monitoring can be used by watchdog organizations to record and publicize the different positions and commitments made by parties and candidates, and then monitor and report on their record while in office. The information produced by these efforts can help increase public scrutiny of politicians. At the same time, this type of monitoring can be used by issue-based organizations to gain a specific campaign promise that then becomes the target of post-election monitoring with the purpose of applying pressure to ensure the promise is kept.

Campaign-related monitoring initiatives commonly aim to achieve the following outcomes:

Pre-election period

  • The electorate is more informed on a candidate’s background, campaign platform and voting record.
  • Citizen priorities are taken into account throughout the campaign period.
  • A public record is kept of campaign promises.

Post-election period

  • Elected officials are held accountable for their campaign promises post-election.
  • Citizens and elected officials develop constructive relationships.
  • The policy priorities more accurately represent citizen interests.

Select Monitoring Tools

To help undertake campaign-related monitoring, there are a few methods for obtaining and communicating the positions and commitments of political parties and candidates. These include:

  • Voter guides
  • Community manifestos
  • Pledge drives

All three of these tools help shape what citizens should expect from candidates and political parties and create metrics that can be used to gauge the performance of elected officials. A monitoring initiative can involve one or more of these methods.

Voter Guides

A voter guide compiles electoral candidates’ biographical information, policy positions and priorities, and any public commitments made as part of their campaigns. The guide is shared widely with citizens in the lead-up to an election.

Purpose: Groups have developed and disseminated voter guides in the pre-election period to help voters make choices by providing comparative details about political parties and candidates. The guides also serve as a public record that monitoring groups can use to benchmark performance once an elected official takes office.

These guides can be produced either as online resources or as print copies. Traditionally, voter guides were printed publications physically distributed to the voters, but in today’s digital world, they are more often distributed via websites or mobile apps that voters can access online. While moving voter guides online can increase the scale of distribution, internet access must be widely available, and websites sometimes exclude populations, especially people with disabilities or with other factors that limit access. Therefore, accessibility standards7 are important to consider when implementing websites or mobile apps. Voter guides have also been used in the form of briefings or infographics for the media to disseminate to voters and to challenge candidates during television debates in the election campaign period.

Pre-election

Groups have created voter guides by:

  • Organizing public forums or dialogues in order to collect information on citizen priorities and identify the most pressing of those issues;
  • Creating a “voter guide information packet” that includes questions regarding candidates’ positions on citizen issues;
  • Distributing the voter guide information packet to the candidates to review and answer the questions;
  • Recording interviews with candidates about their policy positions and then posting them online;
  • Collecting candidates’ responses to the voter guide information packet questions; Posting the candidates’ biographical data and their responses to the voter guide questions online or in print; and
  • Publicizing the voter guide through postings on websites, email listservs, social media platforms, newspapers, radio and public forums.
Post-election

Monitoring groups commonly follow up on their pre-election voter guide activities in two ways:

The first is post-election monitoring and reporting based on the information collected for the voter guide. Citizens and CSOs can use this information as benchmarks or indicators when monitoring elected officials’ behavior post-election, advocating for change, or working with newly elected public officials to enhance their accountability to constituents. For example, environmental organizations can use voter guides to monitor elected officials’ follow-through on environmental campaign promises to inform their advocacy campaigns and maintain public pressure on elected officials in both the majority and opposition parties.

The second way groups have followed up on a pre-election voter guide is to continue producing such guides over a series of elections in order to monitor how political party and candidate positions, as well as citizen priorities, change over time. This has helped citizens and CSOs identify trends. The continued use of voter guides can also foster more systemic change. As voter guides become a part of the political landscape, candidates and political parties will begin to expect that groups will monitor and record their actions and promises, and that they will be held accountable if elected. When operating under a constant spotlight, it is more likely that public officials will alter their behavior to become more accountable to citizens.

Civic groups and journalists increasingly use websites to track adherence to election promises, issue periodic progress reports, and, when public officials campaign in additional election cycles, keep those officials accountable to their previous campaign promises.

The Center for Research, Transparency and Accountability (CRTA) monitors public statements and promises in Serbian elections to assess if they are based on factual information. Their aim is to hold candidates publicly accountable to their word in elections and prevent misinformation in election campaigns. Statements made by candidates in election campaigns are published on their portal istinomer.rs (Truth-O-Meter) and are assessed on their factualness.

Community Manifestos

A “community manifesto” is produced collectively by citizens who deliberate and decide on a set of common priorities. It is typically presented to candidates and political parties during a campaign period to secure their commitment to addressing the community’s priorities. In some cases, communities may organize candidate debates to hear from candidates about their plans to address the priorities.

Purpose: Because the community manifesto results from an act of community engagement, it serves to unite the community around shared concerns. In this regard, it can be a powerful tool in influencing the priorities and follow-on actions of candidates and political parties. The heightened expectation that a community manifesto creates can also enliven more citizen interest in the election process and the subsequent performance of elected officials. The manifesto then serves as the basis for ongoing monitoring and community influence over government policymaking.

Considerations: Community manifestos are often considered a viable tactic used by geographic communities at the local-government level. However, manifestos are also created by sectoral interests groups, such as those concerned about health or education. Similarly, manifestos are created by women and youth networks to elevate their priorities and focus political parties and electoral candidates on relevant policy positions at the national level.

Pre-election

Groups have created community manifestos by:

  • Developing a questionnaire that asks citizens what they believe the most pressing issues are within their community;
  • Disseminating the questionnaire to as many members of the local community as possible, ensuring the inclusion of women, youth, older adults, people with disabilities and other traditionally marginalized populations;
  • Collecting the questionnaires and using the findings to identify community priorities;
  • When possible, putting the issues up for deliberation and a vote in the community to identify three to four priorities;
  • Compiling the priorities into a community manifesto in a format similar to that of a candidate’s platform;
  • Presenting the community manifesto to candidates and political parties during individual meetings, public forums and debates; and
  • Encouraging the candidates or political parties to respond to the manifesto.

Gathering information about a community’s priority issues can be challenging. Opinions can vary widely, and some issues may be polarizing, whereas other issues might enjoy broad consensus. Simple online survey tools like Survey Monkey8 or Google Forms9 can help to better understand what issues community members agree on and what issues are of highest priority. Data collected through surveys can help ensure community platforms represent the opinions of the community in a way that interviews or roundtables might not fully capture. For more information, see the Section II chapter on data collection and analysis.

Post-election

In the post-election period, monitoring groups can use community manifestos and public forums to hold elected officials accountable to citizen priorities. It is also a means to continue the dialogue between citizens and public officials to build more constructive relations and establish a participatory culture where issues and ideas can be openly discussed. For example, groups have organized public forums and participated in city council and municipal meetings to discuss progress on citizen priorities outlined in the community manifesto and ways to improve performance, and to incorporate citizen priorities into budgeting decisions. Groups have also organized one-on-one meetings with mayors and municipal representatives. Informed by the community platforms and public forums from the pre-election period, these post-election initiatives foster citizen participation and maintain or increase political space. Though this type of engagement between citizens and public officials usually occurs at the local level, some groups have been able to discuss these issues with national-level government officials.

During Liberia’s 2017 elections, NDI assisted CSOs in organizing their communities to identify a set of priority issues, capture these priorities in a community manifesto, and facilitate meetings with candidates to share the priorities.

Pledge Drives

A pledge drive is a pre-election initiative where citizen groups ask candidates to publicly commit to upholding a set of principles or to take specific actions if they are elected. The candidate’s commitment is made by signing the pledge and, thus, campaign pledges often hold more weight than campaign promises.

Purpose: Typically, groups use this tactic to get candidates to commit to good governance principles once in office, such as measures to reduce corruption, increase transparency and improve government responsiveness. However, pledge drives can also be used by issue-based organizations to get candidates to commit to a specific policy solution that the group deems important. In the post-election period, groups can use pledge drives to monitor elected officials’ behavior and accountability to their campaign pledges.

Considerations: On their own, pledge drives are not monitoring initiatives, but they do set the stage for post-election monitoring during which citizens can hold newly elected officials accountable for the pledges they signed. Because pledge drives tend to be highly publicized and officials often feel the pressure of the spotlight, it can be a good tactic for holding elected leaders accountable at the national level.

Pre-election

In order to be meaningful, pledge drives require candidate, political party, media and citizen buy-in. As such, these drives are often conducted by national-level CSOs or a coalition of CSOs with a wide geographical reach, strong connections with political parties, and a high capacity for managing media relations.

To be fair and balanced, all candidates or parties contesting the elections should be presented with the opportunity to sign the pledge. In some instances, only a few candidates may sign the pledge. However, even those candidates who refuse or do not respond to a group’s pledge campaign send a message to voters. Sometimes none of the signatories to the pledge are elected, and thus there is very little to track after the elections. In these cases, the pledge drive serves more as a voter guide by signaling to voters which candidates were ready to act on the issues. In any scenario, pledge drives have the power to encourage competition among candidates in terms of their readiness to abide by democratic principles and adopt good policies to meet citizen needs.

Groups have created pledge campaigns by:

  • Developing a pledge document outlining the desired post-election behavior. This may be a pledge to adhere to good governance or anti-corruption standards, or it may be a pledge to agree to solving a specific priority issue if elected;
  • Meeting with candidates to explain the pledge drive and encourage them to sign on;
  • Organizing a highly publicized (preferably televised) event where key candidates sign the pledge;
  • Developing and disseminating posters, buttons and stickers with the pledge drive logo;
  • Continuously meeting with political party representatives and other candidates to explain the pledge drive, gain support for the campaign, and encourage candidates to sign the pledge; and
  • Continuing to publicize the pledge drive through TV, radio interviews and press conferences.

Candidates Pledge Image

Post-election

Follow-through on the pledge drive must be tracked in the post-election period for the monitoring initiative to be effective. Groups can do this by:

  • Developing a monitoring form;
  • Identifying communities where newly elected officials signed the pledge;
  • Establishing a monitoring group of 10-15 active citizens in each of the identified communities;
  • Meeting with local government officials to explain the project and inform them they will be monitored;
  • Training the monitoring groups on how to evaluate local government officials and complete the monitoring form;
  • Holding pre-assessment meetings with the local government officials and monitoring groups so that the officials can talk about their accomplishments and ongoing efforts;
  • Providing any necessary support as the local monitoring groups observe and analyze local officials’ performances;
  • Collecting the local monitoring groups’ findings and regularly compiling the information into performance report cards;
  • Holding regular meetings with the monitored local officials to share the performance report cards, discuss the results and provide feedback; and
  • Holding an event to recognize the achievements of the highest-scoring officials.

In Albania, during the post-election phase of the “Good Governance Monitoring Campaign,” citizen groups participated in city council and municipal meetings to determine the extent to which newly elected representatives were upholding their pledges. ACAC used a monitoring form to engage citizen groups and communes in monitoring and assessing the performance of the mayors and local representatives who had signed the good governance pledge in the pre-election period. ACAC concluded the initiative with a roundtable honoring the top mayors who scored highest on their good governance report cards.

Recommendations for Program Implementers

  • Identify reformers within political parties and involve them to the extent possible in campaign-related monitoring initiatives.
  • Because the election period is limited, there is generally not a lot of time for in-depth discussions on a wide range of issues. Citizens and CSOs should systematically prioritize the issues they want candidates to address.
  • Consider that a large part of campaign monitoring is to make candidates and parties more concrete in their promises and commitments and more realistic to context.
  • Civic groups can initiate small-scale campaign monitoring by focusing on only one issue of interest and one or two regional elections. Campaign monitoring does not have to cover all the issues or the whole country.
  • Campaign monitoring will most likely become useful for any other type of political process monitoring and advocacy campaigns, thus it should be considered by all civic groups.

Parliamentary Monitoring

Definition

Parliamentary monitoring is an organized effort to scrutinize the functioning and performance of legislative bodies and the elected members. A parliamentary monitoring effort can be undertaken at the national, subnational or local level, as long as there is a directly elected body that has legislative powers.

Monitoring Purpose

Parliamentary monitoring can fulfill a variety of purposes that may vary depending on the country context and the monitoring organization’s objectives, interests and capacities. Generally, though, this type of monitoring is carried out to increase public transparency of parliamentary processes, help the institution function more effectively, and improve the accountability of members of parliament.

Issue-based groups may choose to monitor the legislative process through the lens of a specific issue or cause (e.g., healthcare reform) to ensure that the legislative work is focused on the public interest and done in a transparent and responsive manner. In contrast, impartial watchdog organizations may choose to monitor the entire proceedings with the aim to improve democratic practices and performance. Through sustained monitoring, these types of organizations, commonly known as parliamentary monitoring organizations (PMOs), can stimulate demand for improved parliamentary functioning and nurture a culture of openness and responsiveness.10 In more open political environments, PMOs can reinvigorate citizen engagement by developing new platforms for political expression and policy discussion.11 Groups can decide to be more or less aggressive in pushing legislative bodies to comply with national laws or international standards. For instance, a group might use litigation to compel a legislature to release information. PMOs can also feed issue-based organizations information and findings that help inform issue-based monitoring or advocacy initiatives.

Civic groups have conducted monitoring initiatives to achieve the following outcomes at national- and local-government levels:

  • Increased citizen access to reliable information on the functioning of legislative processes;
  • Increased citizen access to reliable information on the activity of individual legislators and political blocs;
  • Constructive interaction between citizens and legislators;
  • Improved legislature capacity to communicate about its work and cooperate with civil society; Increased individual legislators’ responsiveness to their constituents; and
  • Greater legislative autonomy within the governing system.

Evaluating the impact of parliamentary monitoring efforts

In 2024, NDI and Directorio Legislativo (DL) published a report, Parliamentary Monitoring in Tough Times: Lessons Learned for Building Networks and Achieving Results, that re-examines parliamentary monitoring and networks, and includes national and regional case studies from Latin America, Africa and Asia. The report reviews existing evidence on the impact of parliamentary monitoring efforts, delves into the role of PMO networks and draws lessons from real-world initiatives in order to understand the evolving landscape of parliamentary monitoring and its place in sustaining and enhancing democratic governance. The report finds that, in the face of great adversity, PMOs are adapting, collaborating and innovating to navigate the challenges and to continue bolstering legislative effectiveness and integrity. 

Monitoring Openness

With a focus on transparency, accessibility and inclusiveness, monitoring for openness relies on examining legislative bodies using an established set of principles. For example, in 2012, NDI and the World Bank Institute facilitated a conference of civic groups that resulted in the Open Parliament Declaration. The Declaration identifies 44 actions that signify parliamentary openness. More than 80 civic groups in 55 countries have signed on to the Open Parliament Declaration, and Open Parliament is now the second pillar of Open Government Partnership.12 Various civic groups have designed tracking mechanisms to follow the progress of their country’s legislature on implementing open parliament actions.

Monitoring Functionality

In general, legislatures have three roles crucial to a country’s progress and the lives of its citizens: making laws, overseeing executive institutions and representing citizens. Civic groups may adopt parliamentary monitoring techniques to improve a legislature’s functionality by monitoring how effective and efficient the institution functions in performing these roles: whether official rules and procedures are followed, oversight functions are performed, or efforts are made to engage citizens. Because legislatures are typically large institutions, it can be valuable to monitor the work of committees and other bodies as well as plenary sessions in order to present an accurate picture of the institution’s functionality. In such initiatives, civic groups monitor legislative procedures and practices, often as it relates to a set of legislative standards, and use their findings to work with legislators and parliamentary officials to improve weaknesses or to advocate for changes in the laws and regulations governing legislative processes.

Monitoring for MP Performance

Another reason that groups engage in parliamentary monitoring is to increase legislators’ performance and accountability to constituents. Publicizing a fair and objective review of a legislator’s performance can put additional pressure on them to account for their work and has the potential to encourage competition among legislators and political parties, which can, in turn, improve performance. It also gives citizens information they can use the next time they vote.

In Germany, a nonpartisan and nonprofit organization, known in English as Parliament Watch,13 aims to promote parliamentary openness, transparency and accountability as well as dialogue between citizens and elected representatives. Parliament Watch developed an online platform that publishes information about members of parliament and their activities and enables citizens to directly ask their members questions. Parliament Watch then tracks and monitors the member’s responsiveness to citizen queries.

Since 2009, Al Hayat Center for Civil Society Development has been collecting information and publishing scorecards on the activities of Jordanian members of parliament (MPs), legislative committees and political blocs.

Monitoring Preparation Stage

After deciding what dimensions of the parliamentary process to monitor, an initial preparatory step involves determining what type of information is needed and whether the information is accessible. One consideration, for instance, is whether parliamentary sessions are open to the public and can be observed. Another is whether there is a timely record of official parliamentary proceedings available in print or digital form.

Based on decisions about what to monitor and the information to collect, a next step entails developing the appropriate monitoring tools. When it is time to pick a tool, conducting an ecosystem analysis — for example, of what information is already available and what is missing, as well as of what the target audience’s needs and motivations are — is a critical step toward choosing the right type of tool and developing effective targeted questions and indicators of analysis. More information on how to develop clear and effective monitoring objectives and methodology can be found in Section II of this guide.

During the preparation stage, groups frequently seek legislator support for monitoring initiatives by meeting with ranking members and sending out formal letters describing anticipated monitoring activities. Groups are less likely to experience push-back during the data collection stage if legislators are aware of the initiative from the beginning of the process and do not feel that they are being attacked unfairly. The letters and meetings are preliminary measures to gain the respect and buy-in of legislators; they should not be used to collect information for the initiative. The legislator interview forms should be applied later in the process during the data collection stage.

Al Hayat Center for Civil Society Development in Jordan laid the groundwork for its parliamentary monitoring initiative early in the process. After publicizing the initiative among Jordanian MPs, Al Hayat Center strategically engaged the Speaker and Deputy of the House to explain the initiative’s objectives and how it could benefit his work by helping to improve the overall functioning of parliament and the professionalism of MPs. Al Hayat Center secured the Speaker’s support and commitment to help resolve any issues that might arise with legislators, as well as access to information. Al Hayat Center also organizes and facilitates capacity-building sessions for MPs.

Common Monitoring Tools

Groups have created petitions or questionnaires to collect information on citizen priorities and perceptions of political parties, individual legislators and legislative bodies. These can be developed either as interviews or as forms to be filled out individually. Though the response rate is higher for forms completed during an interview, individual forms require less staff and volunteer time. For issue-based groups, petitions are also a great way to demonstrate broad support for a particular issue or piece of legislation. Paired to a Contact Management System like CiviCRM14 or MailChimp15, petitions and surveys can also serve to build a list of supporters that can be targeted with further communication, such as the release of a monitoring report. Surveys do have, however, important limitations; for details, see the Surveys and Selection Bias sidebar in the Types of Political Process Monitoring section.

Organizations commonly develop parliamentary monitoring forms that volunteers and staff use to capture information on legislative processes or legislators’ performance during legislative assembly and committee sessions. Depending on their capacity for monitoring and the purpose of the monitoring, groups have designed the forms as checklists, questions to be answered with written narratives, or a mixture of both.

Groups also use legislator interview forms to capture legislator perspectives on their responsibilities and performance. Such forms can be administered by monitors or filled out individually by legislators. Once all the tools are complete, groups conduct training for volunteers and staff to learn how to administer the tools while observing legislative sessions and interviewing legislators.

Data Collection Stage

Groups can collect data at national and local levels depending on the monitoring purpose and strategy. When groups monitor to increase citizen access to information, improve legislator performance, or reduce corruption, data is primarily collected through examining public documents and direct observations of legislative sessions and committees. This examination can provide monitoring groups with a better understanding of legislative processes and official government positions on issues and policies. It also allows groups to supplement, and sometimes validate, the data collected through observations.

Through direct observation of legislative sessions and committees, monitoring group staff and volunteers can collect information such as:

  • Legislator attendance;
  • Legislator participation;
  • Legislation or amendments introduced;
  • Responsiveness to formal CSO and citizen requests;
  • Amount of time devoted to meetings with citizens;
  • Communication between legislators and citizens;
  • Number of requests for information received by legislators and how the requests are handled;
  • Accessibility of public information;
  • Legislature and committee functions;
  • The role of the opposition;
  • How CSO and citizen initiatives are conveyed to the assembly;
  • Follow-through on campaign platforms and promises; and
  • Debate issues.

When the purpose of the monitoring is focused on accountability and increasing direct interactions between citizens, legislators and political parties, groups tend to also use questionnaires or surveys, which helps capture citizen and legislator perspectives. When groups use interviews and questionnaires to interact directly with MPs and assembly members, they get a more nuanced sense of the opportunities and challenges legislators face when attempting to fulfill their responsibilities.

Monitoring Communications Stage

Tools and strategies commonly used to communicate the findings and recommendations with the relevant target audiences include:

  • Monitoring reports that outline findings and issue recommendations for improvement;
  • Scorecards that outline the performance of MPs;
  • Websites or mobile applications that regularly publish information about parliamentary proceedings and performance and provide a space for citizen queries to MPs;
  • Infographics that make findings from monitoring digestible to the wider public;
  • Social media pages that regularly report on parliament proceedings;
  • Public town hall meetings and roundtables informed by the monitoring reports and scorecards; and
  • Awareness-raising and advocacy campaigns based on the monitoring reports and findings.

Recommendations for Program Implementers

  • Involve legislators from the beginning of the parliamentary monitoring project so that they understand the project and do not feel threatened by monitoring. Legislators will then be more likely to give their support and cooperate more fully during project activities.
  • When seeking legislators’ buy-in to monitoring initiatives, provide them with an incentive for participation, such as recognition for good governance.
  • Ensure that citizens have physical access to legislative sessions and committees. Parliamentary monitoring is most successful when formal and informal mechanisms create enough political space for citizens to collect the necessary information.
  • Identify areas where the legislature is making progress, so that the monitoring is not only about finding out what is not working or what is not open to the public.
  • Much of the changes for openness and functionality of a legislature are in the hands of administrators who are civil servants. Civil servants also run the logistics and much of the access to the parliament. Thus, developing a good relationship with committee support staff is important for monitoring activities.

Monitoring Government Follow-Through

Definition

Monitoring government follow-through involves tracking the extent to which policies, programs, laws or specific commitments are implemented. The focus is typically on executive branch institutions and actors responsible for providing public goods and services that satisfy citizen needs and interests.

Monitoring Purpose

This type of monitoring is a method for civil society to draw attention to the government’s responsibilities and provide information about whether progress is being made toward reaching the government’s stated objectives. It can also focus on the extent that “open government” practices are being followed or public-sector reforms are being instituted.

Examples of common monitoring government follow-through objectives include:

  • Increase public awareness of the extent to which governments implement and enforce laws, policies and programs;
  • Pressure governments to comply with and enforce decisions and policies;
  • Increase inclusion and participation of marginalized groups in policies and programs;
  • Increase implementation effectiveness and responsiveness;
  • Increase transparency of government decision-making;
  • Increase government accountability to legislators and citizens; and
  • Strengthen good governance.

Some groups may only monitor government follow-through, while others may combine this type of monitoring with advocacy or awareness-raising campaigns focused on a sector (e.g., health or education) or an issue area (e.g., climate change).

Watchdog organizations may choose to monitor specific democratic governance dimensions, such as the level of transparency in decision-making at the national level, or the political space for civil society voices to participate at the local level. These initiatives can be an effective oversight method for civil society to prompt more responsive and accountable governance by determining if official decisions and actions have resulted in the promised changes. Issue-based organizations may choose to monitor government follow-through to track an implementation process and be prepared to advocate, if a desired change is stalled.

Monitoring government follow-through can work in tandem with other forms of monitoring, such as budget monitoring, expenditure tracking or shadow reporting. All of these efforts can also help build better working relationships with public officials and can foster more collaborative forms of governance.

Monitoring Dimensions

Government follow-through initiatives can target a wide range of local or national-level government actions, such as the execution of power-sharing agreements, post-disaster reconstruction programs, job-creation policies, environmental protection programs, local development plans, or constitutional reforms, among others.

Monitoring Compliance

One dimension these types of initiatives may monitor is compliance. Specifically, groups can monitor how well government agencies comply with the legislation and policies that regulate a process. A good example is an initiative that aims to monitor how well and how quickly the government is abiding by or upholding a right-to-information act, also known as “freedom of information” legislation.

Monitoring Priorities and Responsiveness

Monitoring the priorities of the local government, a government agency or the national government is another dimension of this type of monitoring. In these kinds of initiatives, civic groups gather the citizen or community priority issues and present them to the relevant government institutions. The civic groups then monitor the level of responsiveness of the target institution to the priorities and whether and how the priorities are addressed, e.g., priority issues are included in budget plans, target issues are made into public policies, and government actors provide explanations for why a specific issue has not been addressed yet.

Monitoring Effectiveness and Efficiency

Issue-based advocacy groups, in particular, are inclined to monitor the government’s performance as it relates to their priority issue or cause. These types of groups tend to monitor the effectiveness and efficiency of government follow-through. Initiatives that monitor these dimensions are also common after a national disaster or economic recession, where a large-scale government program is planned that involves significant public funds and diverse beneficiaries.

Because there are so many different government policies, programs and decisions at different levels of government, this type of monitoring is also the most diverse of all other types of political process monitoring.

Civic groups in Nepal, some supported by NDI, monitored government follow-through on the earthquake reconstruction program after the 2015 earthquakes that devastated the country. They focused on the effectiveness and efficiency of the national program based on performance indicators. Similarly, the country’s COVID-19 economic recovery programs drove many civic groups to initiate monitoring government follow-through of those programs.

In 2020, Chapter One Foundation (COF) launched an initiative to monitor how the Zambian government’s COVID-19 response impacted human rights. COF recognized the need to apply a “human rights lens” to the pandemic response to ensure that critical human rights were being preserved and no one was excluded from the response, and to mitigate the long-term effects of the pandemic. COF developed a monitoring framework for a specific set of human rights to track their status or violations and the degree to which the Zambian government was complying with obligations. Due to social distancing regulations, most of the data was gathered using secondary sources. After analyzing the data, COF developed a monitoring report, sharing it with government officials and citizens. To raise awareness, COF also held a series of webinars on human rights and COVID-19 topics that were broadcast nationally on television and livestreamed on Facebook.

Monitoring Preparation Stage

During the monitoring preparation stage, groups should:

  • Determine the goals, objectives and results for the monitoring project;
  • Develop a strategy, work plan, timeline and reporting mechanisms to guide the project’s activities;
  • Develop benchmarks based on the government’s outlined plan for implementing the decision. If there is no implementation plan, develop a reasonable one based on the political context and government capacity. These benchmarks will guide the coalition’s monitoring activities;
  • Create standard monitoring tools, such as a monitoring worksheet, so that information will be collected uniformly; and
  • Assign specific roles and responsibilities for collecting and analyzing information.

During this stage, it is important to conduct research to determine the procedures, timelines, and background information of the target government process to develop an effective monitoring methodology and plan, design monitoring tools, and consider entry points for monitoring. Civic groups must also clearly define the target audience and how to engage them to design the monitoring tool and types of targeted questions that support the monitoring goal. You can find more information about developing clear and effective monitoring objectives and methodology in Section II of this guide.

When the target government decision, policy or program extends across issue areas and geographic regions, monitoring coalitions are often better suited to monitor government follow-through. Coalitions combine the strengths of civil society organizations with varied interests, expertise and geographical reach. As such, coalitions are more likely than a single organization to be able to mobilize the expertise and resources needed to ensure that information can be collected over a sufficiently large geographical area and encompass the entire scope of a target political process.

Data Collection and Analysis Stage

Depending on the government decision, policy or program being monitored, CSOs can use a variety of methods and information sources to gather data. Some groups have collected data by:

  • Examining government press releases and other public information sources;
  • Submitting requests for information to government officials;
  • Monitoring media coverage of the decision and its implementation;
  • Conducting key informant interviews with citizens;
  • Organizing informal dialogues with experts on the decision’s issue areas; and
  • Holding meetings and roundtables with legislators to exchange information.

During the preparation stage, CSOs should develop standardized data collection tools for use in all areas of the data collection stage. This ensures that they can uniformly capture information from a variety of sources. Whether managed by a coalition or a single organization, the volunteers and staff members collecting information should use the same data collection tools to ensure that the same type of information is collected through the same methods — even if the issue areas are different. This makes it easier to sort and analyze the data once it is compiled and adds rigor to the research, therefore increasing the legitimacy of the findings.

If working as a coalition, member organizations may choose to carry out monitoring activities independently, periodically compiling and analyzing the data based on the implementation benchmarks developed during the preparation stage. These findings can then be presented to the rest of the coalition during regularly held meetings in order to compile information and share experiences across member organizations. This information can then be organized and analyzed by a smaller group of each organization’s staff.

Monitoring Communications Stage

In most cases, individual groups and coalitions have used the gathered information and the analysis to produce monitoring reports. Based on the data collected from various sources, these reports comment on how well the government has been implementing its decisions, highlighting both successes and implementation gaps, along with recommendations for improvement.

While these reports provide a useful picture related to follow-through, they can also serve to inform advocacy campaigns and awareness-raising at the local, national and even international levels. For example, groups may have wider campaign strategies designed to influence particular outcomes, such as reducing unemployment or improving services at health clinics. Groups have publicized reports through press conferences, the internet, newspapers, private discussions and roundtable discussions. These awareness-raising campaigns can apply the pressure of public scrutiny to influence the government to do a better job implementing the target decision or program.

Recommendations for Program Implementers

  • Because policies are often complex and may encompass a variety of issues, groups should be realistic about the scope of their initiatives to monitor government follow-through. If they are not able to monitor the implementation of the government decision in its entirety, then the group should choose specific aspects on which to focus. Otherwise, a coalition of groups might be needed to monitor broader government undertakings.
  • Monitoring government follow-through can be challenging when executive branch institutions and public service providers are reluctant to share information and answer directly to citizens. Groups have to be strategic in finding entry points, establishing relations and accessing useful information that can be used to draw objective conclusions about implementation processes.

Budget Monitoring and Expenditure Tracking

Definition

Budget formulation and public expenditure are central fiscal responsibilities of national- and local-level government. These functions are cyclical and have different moving parts, involving both legislative bodies and executive branch institutions. Annual budgets reflect government priorities and programs, and should also provide details on revenues, spending and debt. Budget monitoring opens the budget-making process to public view and helps ensure that it follows the established rules, including provisions for public review and legislative approval. Public expenditure tracking uses the approved budget to determine if public funds are allocated and spent in accordance with the budget and with other laws governing public spending. Monitoring public procurement is an aspect of expenditure tracking that focuses on processes surrounding the government’s decisions to purchase specific goods and services, such as solid waste disposal.

Generally, there are four stages to the budget process.16 First, in the formulation stage, also known as the drafting stage, a budget is put together and made ready for proposal to the legislature. Next, in the approval stage, the budget is reviewed and changed before it is approved by the legislature. In the implementation stage, a budget is put into action to be implemented by the executive branch. Lastly, in the oversight stage, the budget is overseen by the legislature and the auditor’s office to check if it is being implemented as approved (by the legislature) and following the relevant financial laws (by the auditor’s office). Budget monitoring takes place during the formulation and approval stages of the budget cycle, while expenditure tracking, including public procurement monitoring, takes place during the budget implementation and oversight stages.

The four stages of the budget process cycleThe four stages of the budget process cycle

Some groups may move from monitoring the budget process into budget advocacy, which involves organized citizen efforts to influence budget allocations or reform the process.

Monitoring Purpose

Civic groups may monitor the budget and expenditures processes to make them more transparent, inclusive and efficient and to prevent corruption. Budget monitoring is generally done to ensure that the public has access to budget proposals and that budgeting rules are followed. Expenditure tracking helps ensure money is reaching the intended destination and that it is accounted for through the procurement processes. This helps mitigate mismanagement and contributes to resources better serving the public interest.

Civic groups can monitor budgets and track expenditures for a variety of intended outcomes, including:

  • Increased citizen scrutiny and involvement in budget decisions;
  • Improved transparency and fiscal responsibility;
  • Decreased levels of corruption;
  • Ensured value-for-money of public purchases;
  • Improved service delivery and public sector projects; and
  • Improved engagement between civil society, government, and citizens.

As monitoring groups become more experienced in budget monitoring and form relationships with public officials, they may choose to participate more actively in the budgeting cycle. Some groups may start with a focus on budget transparency and then move into expenditure tracking and budget advocacy initiatives.

Monitoring Openness and Outcomes Throughout the Budget Cycle

Monitoring Budget Openness

Because the budget enumerates the priorities and policies of a government for an entire cycle, the process of drafting and implementing a budget must be transparent and open for public input. However, complete budget information is often not accessible to citizens at the national or local levels. In addition, budgets and the technocratic steps in the budget cycle can be complicated for ordinary citizens to understand and easily engage in without support. For these reasons, some civic groups focus entirely on monitoring the openness of the budget process. Such groups monitor the level of space provided for citizen input and whether the budget process follows transparent practices at various stages, and then aim to make the process more open through ongoing monitoring and advocacy initiatives. Monitoring budget openness is typically done by watchdog organizations. Groups can develop a “Citizen’s Budget,” a concise, illustrative budget summary designed to be easily understandable by citizens, as a tool to help make the budget process more transparent. Organizing budget consultations is another tactic to support a more open and transparent budget process.

Monitoring Budget Priorities

Budget monitoring initiatives can also focus on tracking budget proposals and the enacted budget allocations and spending, in terms of investments in specific sectors and budget categories. This dimension of budget monitoring is centered on ensuring that the government is setting the right priorities and doing it in a manner that is fiscally responsible and responsive to citizen needs.

Using information collected through budget hearings, budget reports and citizen questionnaires, groups can determine the extent to which citizen priorities are included in the final budget. These findings have often led groups to conduct follow-on activities that continue their engagement in the budget cycle beyond the formulation and approval stages, including expenditure tracking and budget advocacy.

Issue-based organizations may monitor the budget trends within a sector or focus on a particular issue to raise public awareness about funding levels and to inform advocacy campaigns designed to influence spending decisions. Watchdog organizations continually monitor different aspects of the budget process against certain indicators related to openness and fiscal obligations. These types of monitoring initiatives can help civil society and citizens get informed about the budget process and the priorities and policies that will be funded in that fiscal year. Such monitoring can also be useful for policymakers and public officials seeking to involve informed citizens who understand the budget process.

Expenditure Tracking

Even when citizen groups actively monitor the budget-making process, there is often a gap between allocation decisions and how public funds are disbursed and spent. As agencies initiate spending through payrolls and procurement processes, citizen groups have kept records of local expenditures. As the projects are rolled out and payments are made, groups have observed the implementation of the projects and conducted citizen-based service delivery evaluations to ensure quality.

Expenditure tracking initiatives can take place during the budget execution and oversight stages of the budget cycle, but the majority of the activities typically focus on tracking spending during the budget execution process. Groups can use budget documents and findings from budget monitoring to engage in expenditure tracking. For example, information on the final budget allocations captured through a budget monitoring initiative can be used to provide points of comparison for the information collected through an expenditure tracking initiative. Civic groups may compile monthly reports based on their expenditure tracking findings. These reports can then be compared to the local government’s record of expenditures and yearly reports and statements. During the budget oversight stage of the budget process, some citizen groups have even audited local budgets to reveal how the government’s records compare to their own. In addition to ensuring the quality of public projects and services, these findings can help reduce corruption. Groups can use them as a basis for working with local governments to improve the budget execution process or to raise citizen awareness of discrepancies in expenditures.

To enhance transparency, accountability and integrity in Zambia’s COVID-19 response and its donations management system more broadly, Transparency International-Zambia (TI-Z) launched a monitoring initiative tracking the use of COVID-19 monetary and in-kind donations. TI-Z adapted an existing transparency and accountability tool — the CODOT System17 — to capture relevant data on the monetary value, source and disbursement status of the donations. Following the publication of their monitoring report, TI-Z engaged key government institutions to present their findings and share recommendations. TI-Z also developed a media presentation for press briefings and packaged their findings for social media and longer-term advocacy campaigns. TI-Z plans to continue to monitor the donations management system and to link their work to other government oversight processes, such as a COVID-19 government audit report, to amplify their voice and impact.

Public Procurement Monitoring

Government spending on projects, goods and services is done through public procurement from market sources. Around a third of all government spending around the world is done through public procurement; the rest includes salaries, unemployment benefits, interest payments, subsidy programs, etc. Public procurement is prone to inefficiencies and deficiencies that can often facilitate corruption. Watchdog groups interested in tracking expenditures to prevent and fight corruption, as well as to push for strict compliance of procurement laws and necessary reforms, monitor public procurement. Groups that monitor a specific government project, national program or public policy may integrate monitoring of public procurement in their area of interest to track how the funds are being spent.

Moving procurement procedures online has proven to be one of the best ways to increase transparency in public procurement. E-procurement platforms built by governments, usually after much civil society advocacy, are portals where government agencies publicize their notices for procuring goods and services and private-sector entities can apply and compete for selection. In countries where such portals are built and mandated by law, civic groups can use the information found on the platforms to monitor procurement procedures.

But even after competitive procurement processes are finished and contracts awarded, monitoring the implementation of those contracts is important to ensure that the work is conducted as completed and of high quality, and that it reaches all the intended areas and communities. It is also a time when corrupt practices, like kickbacks, can be detected. Civic groups have employed social accountability methodologies, such as Public Expenditure Tracking Surveys (PETS), to monitor the implementation of a government program or service and to track expenditures. PETS help evaluate the delivery, quality and impact of a public work or service procured by the government. That information is shared with the government agency that procured that work or service to enforce contracts and improve service delivery. In many cases, government agencies partner with civic groups to use PETS for monitoring a contract.

Other Resources

  • A Citizen's Guide to Making Public Accountability Work: a manual for
    every citizen in Nigeria, Africa, and the Global South who wants to ensure effective public oversight on government spending. Although the manual includes examples specific to the Nigerian polity, it can be easily contextualized to the setting in other countries.

Monitoring Preparation Stage

To conduct budget monitoring and expenditure tracking initiatives, civic groups need to understand the budget cycle and what can be monitored during the four stages. Civic groups should first do an analysis of the political context and existing budget practices to determine what gaps exist, including the amount of information available to citizens. In order for citizen engagement in the budget cycle to be successful, the government must be willing to allow citizens access to budget meetings and publicly share budget documents. If there is not a formal law or policy requiring that budget cycles be transparent, then efforts need to be made to open the processes; otherwise, there will be very little space to gather sufficient information and to effect any concrete changes.

After determining if there is adequate political space to implement a meaningful budget monitoring initiative, civic groups should examine their own capacities. Due to its highly technical and complex nature, it can be challenging to understand the budget cycle, fiscal requirements and standards, budgetary proceedings, and the materials produced.

In order to prepare for effective data collection, civic groups should aim to understand:

  • The different stages of the budget cycle;
  • How citizens can engage in the budget cycle at different stages;
  • How to analyze budgets;
  • Budget advocacy methodologies for specific contexts;
  • How to collect and analyze budget monitoring data for advocacy;
  • How to track specific expenditures;
  • How to collect and analyze expenditure tracking data for advocacy; and
  • Citizen-based public service delivery evaluations.

For budget monitoring initiatives, the methodology and action plan will need to align with the annual budget cycle. For budget monitoring or budget advocacy initiatives, groups need to wait for the beginning of the budget cycle to carry out activities. Similarly, for expenditure tracking, groups would have to wait for the budget execution and oversight stages. Meaningful monitoring of allocations and expenditures requires that the budget itself is transparent and fully published, ideally in a machine-readable form. Along with setting monitoring objectives, groups need to clearly define the audience for monitoring findings and consult the end users during development to make sure that tools are user friendly and accessible.

Data Collection and Analysis Stage

After determining the appropriate point of entry into the budget cycle, groups should then develop the monitoring tools. They can create monitoring forms to collect data while observing budget hearings and obtain budget documents during the budget formulation and approval stages. If the political space allows, groups can monitor both executive and legislative hearings. In some cases, budget documents are disbursed during hearings, while in others, groups must request them from government officials. The key documents that monitoring groups should request are the executive budget proposal, supporting budget reports, documentation of any budget laws, and reports developed by legislative budget committees.

Groups can also collect information on citizen priorities through interviews, questionnaires and public discussions. Using information collected with these tools can determine the extent to which citizen priorities are included in the final budget, and during which stage of the budget cycle priorities are cut.

Groups have also used investment tables to track expenditures on public projects and service delivery throughout the execution process. These simple spreadsheets inventory the objects of your monitoring efforts. The data collected in investment tables will depend on the availability of information, the legal requirements in your country and the goals of your monitoring efforts. Seeking advice from a data scientist or person with experience in your type of monitoring is highly recommended.

Tip: In the case of public tenders for construction projects, an investment table could list the following information as column headings with each tender on its own row:
  • Tender Name or Reference Code
  • Tender due date
  • When did the tendering procedure start?
  • Have the opinions and needs of stakeholders been considered?
  • What was the planned cost in the procurement plan?
  • What was the budgeted cost?
  • What was the contracted cost?
  • What tendering procedure was used? (open, limited or other)
  • Has the online public procurement system been used?
  • Are any tendering criteria discriminatory or eliminating competition?
  • Was there a prior notice on the official government website for the tender?
  • Have the terms of reference been published in the call?
  • Who was the winner of the contract?
  • When was the winner announced?
  • What type of contract was awarded?
  • Was the winning company selected according to the legal conditions?
  • Have there been official complaints?

Online tools can ease the process of data collection, especially when trying to quickly ascertain the shared priorities of a large group of people. More information on practical steps and best practices for online data collection and analysis can be found in the Data Collection chapter of Section II of this guide.

Monitoring Communications Stage

Civic groups can publish regular monitoring reports on their budget monitoring and expenditure tracking analysis, findings and recommendations and then use these reports to conduct follow-on advocacy activities. For example, based on their findings and analysis, some groups have developed proposals for specific changes to the budget or budgetary practices. In some instances, this might look like a “budget platform” that lays out a set of citizen priorities for budget committees to consider while developing a budget. Groups can also use monitoring reports and other communications products to raise citizen awareness of the budget process and discrepancies in expenditures and to encourage government officials to make budgetary information more open and accessible for citizens. Because of the complexity of the budget cycle and documents, it is important to present the findings and analysis in a concise and easily understandable manner. Some groups often build websites that publish information on budgets and expenditures to show trends in a visual, easy-to-understand manner.

Advocacy tactics to ensure the desired change to budget or budgetary practices is considered include:

  • Proposals or budget platform presentations during budget meetings;
  • Face-to-face meetings with officials;
  • Letter-writing campaigns; and
  • Public forums for local public officials and citizens to discuss budget priorities.

Recommendations for Program Implementers

  • Implement budget monitoring, budget advocacy and expenditure tracking initiatives through longer-term projects of at least 18 months. Successful budget monitoring initiatives are planned around the budget cycle and fiscal year of the host government. A longer time frame allows for citizen or CSO engagement throughout the entire budget cycle.
  • Budget monitoring and expenditure tracking is extensive, and groups must have clear goals for monitoring and achievement. That will determine whether to monitor for budget openness or alignment of government priorities with public priorities, or to push for efficiencies in a public project.
  • Civic groups must learn the budget process and spending procedures in their country to monitor the budget and track expenses, particularly for monitoring public procurement, where many laws and regulations can create a large bureaucracy.
  • Civic groups interested in monitoring for budget openness should first refer to various indexes and monitoring tools developed by other organizations around the world, as many of the same practices apply in any context.
  • Issue-based groups should partner with watchdog organizations monitoring the budget process to use their data and methodological experience as the basis for their own issue-focused monitoring.

Shadow Reporting

Definition

As part of their compliance with ratified international treaties, conventions and declarations, governments are typically required to submit a report to the international body associated with the agreement to which they are a signatory. These reports describe the progress made toward various requirements outlined in the signed agreements. To monitor and raise awareness of the progress, civil society groups conduct research on how well their respective government is complying with an agreement and produce a supplement or alternative “shadow” report of the national government’s “official” report.

Monitoring Purpose

Shadow reports allow groups to use the international agreement’s principles and standards to hold their government accountable for enhancing public service delivery and strengthening laws and policies addressing the relevant issue.

Shadow reporting is typically part of a broader effort to influence change within an issue area. Developing and publishing a shadow report creates an opportunity for groups already active in an issue area to raise awareness and establish relationships with both national and international stakeholders.

Typically, shadow reporting initiatives have been organized around holding governments accountable, specifically for promises to end discriminatory practices that most often affect traditionally marginalized populations. Other example objectives for shadow reporting initiatives include:

  • Identifying gaps in government policy development and implementation;
  • Increased collaboration between governments, CSOs and political parties on policy development and implementation; and
  • Increased political party and government accountability and responsiveness.
Monitoring Preparation Stage

A coalition of groups almost always develops shadow reports because the data collection process is time-intensive, and the report-writing process requires specialized expertise in both technical writing and the issue area. Shadow reports are also more powerful when a variety of stakeholders are involved, because the different perspectives lead to a more nuanced and representative report.

When forming a coalition, it is important that all of the coalition members are aware of the potential risks involved with participating in a shadow reporting initiative — such as negative reactions by governments — as well as the benefits. To ensure that all of the coalition members are in agreement concerning the purpose of the coalition and how it relates to the shadow report, groups should first identify the coalition’s values, mission, vision and governing structure. Developing and signing a memorandum of understanding has also helped coalitions clarify and confirm each of the members’ roles, responsibilities and expectations.

Once the coalition is established and the members have agreed on expectations, members can develop a monitoring strategy and plan by:

  • Determining a monitoring objective that relates to the coalition’s purpose;
  • Outlining the steps in the shadow reporting process;
  • Developing an action plan;
  • Determining a division of labor; and
  • Identifying the possible internal implications for the coalition, including follow-on initiatives.

When developing the methodology and action plan, special consideration should be given to timing. Typically, governments are required to submit reports to the relevant multilateral agency according to a set schedule — usually every two to four years depending on the agreement. Under most circumstances, the aim should be to submit the shadow report at the same time that the government submits its official report.

At this stage, the monitoring coalition also identifies or develops the tools needed to conduct the research and gather the information needed for the shadow report. Commonly, monitoring groups develop questionnaires for public officials that focus on the relevant national policies and laws in order to gauge the government’s compliance. Questionnaires, interview forms and focus group guidelines can also be created in order to collect information from relevant citizens about their experiences with public services and perspectives on how well the government is ensuring their rights.

Data Collection and Analysis Stage

In order to rigorously collect information on the government’s compliance with an agreement, monitoring groups must engage government officials and citizens in the data collection process. Typically, coalitions have collected information through examining government documents, administering questionnaires, and conducting interviews with public officials. Groups often will need to request government documents in person or by phone with the relevant government agency or submit formal written requests. In some instances, groups have used official government websites to request public documents or submit monitoring-related questions. This can be a useful data collection method for coalitions that want to avoid taking a confrontational approach or in political contexts where interviews with public officials would be difficult to arrange. Analysis of public documents and information from public officials can provide groups with a better understanding of the full range of government policies, programs and laws aimed at fulfilling the obligations laid out in the relevant international agreement. The United Nations and other multilateral institutions, national governments and local groups may also have more accurate and in-depth information on government compliance with international treaties, conventions, and accords.

Coalitions can also administer questionnaires to citizens affected by the relevant international agreement. These questionnaires, as opposed to those developed for public officials, gather information about government performance. They are used to provide insight into how well government initiatives have delivered on the international agreement. The questionnaires can directly ask citizens to comment on their satisfaction with public services or the protection of rights, for example, or they can collect information on citizens’ experiences. The first approach is most appropriate in countries with relatively open political space, and the second is most appropriate in environments where citizens might be hesitant to openly criticize the government.

Questionnaires can be sent out to the appropriate population groups in the form of a survey, or they can be completed during an interview conducted by coalition activists. The first method of disbursal involves less effort from the coalition but may result in fewer responses. The second method requires that the coalition recruit and train activists to conduct interviews, but this may lead to a better response rate than a mail-in or online survey. See the Data Collection chapter in Section II of the guidebook below for more information on online distribution mechanisms for surveys.

Once the coalitions have collected enough qualitative and quantitative data to have an unbiased view of the extent to which the government has implemented the international treaty, they organize and process the data accordingly. Data entry systems can make sorting and organizing information more manageable to support data analysis. Some groups choose to use volunteers to enter the data into the system. The benefit of using volunteers is two-pronged: the volunteers have sorted the data at low cost, and knowledge of the issue addressed in the shadow report has been spread to a wider audience.

Monitoring Communications Stage

The primary product of the shadow reporting process is the report sent to the appropriate multilateral organization for review, such as a committee of the United Nations. After analyzing the data, groups develop recommendations and compile them into a carefully formatted report, organized according to the way the international agreement is divided into articles or sections.

A shadow report usually includes:

  • The text of the treaty, convention or accord;
  • The government’s stance on its compliance; The coalition’s findings on the government’s compliance; and
  • The coalition’s recommendations for improving government compliance.

The findings should also identify the government’s successes in meeting the requirements of the agreement, as well as the gaps in policy, laws and services. Taking this approach provides a more complete picture of the government’s progress and can also help in making local public officials feel less threatened by the shadow report. The recommendations should put forth practical, concrete ways the government can improve its compliance with the international agreement if necessary. Framing shadow reports to be constructive rather than combative allows coalitions to use them as awareness-raising tools and mechanisms for change.

The timing of the publication is also dependent on when the government presents its progress report. Shadow reports generally have the most impact when multilateral organizations receive them at the same time as the official government report.

The monitoring coalition should present their findings to the appropriate committee or body within the relevant multilateral organization for review; presentations should be in a format that aligns with the needs and interests of their target audience. Shadow reports should be viewed as tools for creating change instead of the final step of the monitoring initiative. Groups can strategically use them to support their awareness-raising and advocacy initiatives, as well as other initiatives that support the coalition or the individual member organizations’ goals and objectives. For example, coalitions have publicized their shadow reports in order to raise awareness of their focus issue at both national and international levels — often using the findings from the report to garner citizen support for the issue and to increase the report’s legitimacy in the international arena and get the attention of international organizations and donors. As they publicize the report and their findings, coalitions should also ensure that they communicate the results to the stakeholders from whom they collected the data. This demonstrates respect for the stakeholders and contributes to the transparency of the shadow reporting process.

The large amount of data collected during the shadow reporting process can also be used to strengthen new or ongoing evidence-based advocacy campaigns conducted by either the coalition or its individual members. If the shadow report was framed to be constructively critical, it can create a basis for conversations with key decision-makers on the gaps in the implementation of the international agreement, as well as ways to address those gaps. Coalitions can use these conversations to facilitate better relationships with government officials, bring civil society more meaningfully into decision-making processes, and create real change to address target issues.

Recommendations for Program Implementers

  • Establish a coalition of CSOs to develop the shadow report. It is too large a task for one organization to tackle on its own.
  • There should be at least one person dedicated to work full-time on the shadow report. This person should have connections with CSOs and government officials at the local and international levels, as well as expertise in human rights, shadow reporting and the issues addressed by the report.
  • The coalition should seek out external funding so that it can hire the necessary full-time staff.
  • While establishing the coalition, ensure that the CSOs are interested in developing a shadow report and willing to work together with other organizations to do so.
  • While establishing the coalition, consider how the local government and other local and international CSOs view the potential coalition members.
  • Before beginning work on the shadow report, the coalition should map the context in which they are working, specifically what other organizations are doing in their issue area and if other organizations are developing shadow reports.
  • Often, shadow reporting conducted at an international level will have set data collection and presentation standards. Sharing information with entities in other countries can reduce the burden of identifying methodologies, tools and strategies.

IMPROVE Steps

Effective monitoring requires proper planning and clear goals. “IMPROVE” is an acronym for each of the key steps in effective political process monitoring initiatives:

  • Identify your goals.
  • Mobilize your resources and your people.
  • Plan your activities and monitoring tools.
  • Record or gather data.
  • Outline what you’ve recorded.
  • Voice your findings and recommendations.
  • Evaluate how well you achieved your monitoring goals.

Some effective monitoring initiatives use human-centered design (HCD), a methodology to design user-friendly and effective digital products, services and campaigns. HCD can help you define your objectives, identify your target audience, and understand their needs and wants to design digital tools that solve real problems. An HCD process also starts with an analysis of the relevant ecosystem of actors and information. This approach is beneficial to understanding how your monitoring initiative can have the most impact in the communities and context you are working in, and how you can engage with your target audience throughout the design and implementation of the monitoring initiative. Through low-cost, do-it-yourself–style activities, NDI’s Co/Act toolkit18 can help you adopt this approach for your monitoring initiative.

Improve Steps

Identify the target political process, your goals and the stakeholders

In the first step, identify the target political process (government program, target institution or officials) that you want to monitor, including the gap or need in the process or something that could be improved. Ask questions about the current level of transparency, how responsive the institutions and officials are to citizens, and the level of access to the process, etc.

Also identify the goals you want to achieve through monitoring. Envision how the monitoring initiative can improve the political process and what civic participation in the process will look like. Identify the overarching questions that need to be answered during the initiative. It is recommended at this step to survey citizens and stakeholders to determine their interests, priorities, needs and expectations around the monitoring initiative.

You should also identify the stakeholders: those who would be most affected by the process, and those who would be interested to know more about the process or who may support you in your work. Because significant time is often needed to show results or improvements, it is important to establish a clear goal with objectives for the short term, midterm, and long term.

Tip: Perform a strengths, weaknesses, opportunities and threats (SWOT) analysis for monitoring the target political process. Through the analysis, explore your organization’s strengths and weaknesses in terms of human capacity, technical capacity and financial resources, as well as in terms of your organization’s relationships that could benefit or hinder the initiative. Identify the opportunities to engage strategically, as well as the potential risks or threats from this type of work or sectors of likely resistance. Explore what is already being done in the space and which efforts are complementary or competing.

Mobilize resources, funding, stakeholders, and your organization and community

The next step involves mobilizing resources, activists and the community to be part of your monitoring. It is important to think about how to engage people in your work and how to get a community’s support. At a minimum, you should inform the community in which you will work to start gaining their support and garnering their suggestions. Spending some time to mobilize your resources will mean that you may have to do some fundraising activities, and may require you to reorganize and reassign your organization’s team. Going through this step is helpful for setting the scope of your monitoring initiative and to ensure you are better prepared for the planning of the initiative.

Plan your monitoring approach and tools

In this step, develop an action plan, where you list goals, activities, monitoring tools, and the people who will run those activities and monitor using those tools. If data is collected on different dates and in different places, the monitoring approach should take these nuances into account to avoid inconsistencies in the data. Identifying ways to standardize data collection and to be transparent and intentional about how data is collected will increase the results’ credibility. This is also when you plan how you will monitor, manage the monitoring initiative, and protect the privacy and security of people and information throughout the monitoring process.

Tip: Start small. You can start by observing only a part of a process, or only one feature within the process. For example, you may start by observing how much transparency exists in the process. This can be done fairly easily and may give you a good understanding of how easy or challenging it will be to monitor the target institution. Perhaps start by observing and tracking the decisions and actions of only one department of a larger public institution, or with one township rather than a larger geographical area. Pilot initiatives are a practical way to start monitoring. In this way, you can build your monitoring capacities and get to the evaluation step more quickly, while requiring fewer expenses.

Another essential part of the planning step is organizing the team responsible for managing the project, gathering and analyzing the data, and communicating the monitoring work. Assign concrete responsibilities to the monitors who will use the tools to gather the required information and report that information.

In particular, it is a good idea to think about who will process and analyze the data and whether this should entail hiring an external data analysis expert or providing training for someone from your organization in data analysis and processes. While monitors should be closely involved in the analysis part, in terms of giving feedback, they are usually focused on observing and recording a very specific part of the process, and therefore will likely have the most insight in that area only rather than in the broader situation.

Record using monitoring tools

The record step, the longest one in the monitoring initiative, is data collection, where you observe and gather the data you need through your monitoring tools. The observation methodologies and tools you use to do this will be covered more in later sections on “Monitoring Tools” and “Data Collection.” There you can find details on different tools you can use to monitor a political process as well as an overview of the data collection process. Because public opinion and budget numbers can shift over the duration of a survey, you must be transparent about the data collection timeframe to ensure the accuracy of inferences gleaned from resulting data.

Tip: Performing this step without clearly defined goals and some level of awareness and support from the community will make it difficult to generate clear results.

Outline your findings

In this step, develop your findings and recommendations and determine how best to share them. Essentially, this is the reporting step. This is when you process the data into information, analyze the data, draw conclusions, and organize the information in a way that can later be communicated to stakeholders and target institutions or officials.

Tip: Avoid speculation. If data has proven inconclusive, it is better to say this than to speculate.

Voice your findings, conclusions and recommendations

This step is where you determine how best to communicate your findings and recommendations to the relevant decision-makers and stakeholders. No matter the communications tool you choose to use, such as a monitoring report, infographic or social media campaign, your core message needs to be communicated clearly and consistently in order to effect change. Developing communications tools and strategy is covered in more detail later in this section.

Evaluate your impact and approach

Finally, evaluate and assess the effectiveness of your initiative and think about ways to improve future iterations of the initiative. This step involves reviewing how well the initiative achieved its objectives and how well the tools worked. Based on this evaluation, the monitoring organization can determine how to improve the tools, better tailor the activities, or organize the team, etc. Generally, it is best to conduct the evaluation when the target political process has come to a recess.

Opportunities and Entry Points

Certain circumstances create particularly good opportunities and entry points for monitoring initiatives. Some examples include:

  • When a government signs onto an international agreement, it is making a very public statement that it will pursue a specific set of objectives. Signing such an agreement makes the government accountable to a multilateral institution for complying with the terms set out in the agreement. This provides an opportunity for CSOs and civil society to supplement or present alternative information to the reports that governments are required to submit under international obligations or commitments. Shadow reports are usually published after or in response to the governmental report.
  • Because elections are mechanisms for holding government officials accountable and are arguably the most visible manifestations of democracy, they can present opportunities for citizens to engage in political processes beyond voting. Taking advantage of the political space created by an election, you can conduct campaign-related monitoring by gathering, analyzing and publicizing information on party platforms, candidates’ follow-through on campaign promises, or compliance with pledges signed during a campaign. These types of monitoring activities allow citizens to establish a set of expectations that can be used to hold public officials accountable for actions before and after an election.
  • Sunshine laws demand that government decision-making processes, as well as the decisions made, are accessible to citizens. Public meeting laws require that governments give citizens advance notice of government meetings, that the agenda is made public ahead of time, and that the meeting minutes are made public afterward. Freedom of information acts (FOIAs) provide citizens with the right to request access to records reflective of governmental decisions and policies. These types of laws open up the necessary political space for citizens and CSOs to effectively monitor political processes. The passage of such laws and policies creates a particularly good opportunity for legislative and budget monitoring.
  • Periods of political transition offer opportunities for CSOs to become involved in forming new constitutions and restructuring government systems and structures. When transitioning to a more democratic system, governments are sometimes more willing to include civil society in planning committees or working groups, thus better positioning them to monitor policy development and implementation.
  • In order to gain membership into intergovernmental organizations, such as the European Union, governments must meet a set of criteria that often includes the protection of political freedoms and a greater voice for citizens. This provides CSOs with increased political space to engage in any of the five types of political process monitoring initiatives.
  • When awarding funding to host country governments, foreign donors often require that the government officials report on how they are spending those funds. This provides CSOs with an opportunity to monitor budgets and track expenditures, even if their country has not adopted any sunshine laws or freedom of information measures.
  • When a CSO’s advocacy campaign has successfully led to the passage of a law or policy, that organization is uniquely situated to then monitor the extent to which it is implemented. During the advocacy process, the CSO might have developed a certain level of expertise in the issue area, created relationships with decision-makers and mobilized a grassroots base. All of those resources can be used to engage in policy implementation monitoring, budget monitoring or expenditure tracking.
  • Many countries have laws requiring transparency and access for hearings and public statements. Partnering with media organizations can increase access to this information in a format digestible by a larger number of people. Recording hearings and publishing statements online can further improve transparency by making it easier for journalists, civil society and members of the public to research and track government activities.

Monitoring Preparation Stage

There are a number of variables and considerations to take into account when preparing to undertake a monitoring initiative. At this stage, it is important to clarify how monitoring will help further your organizational mission and goals and to identify the change you seek to make through monitoring. This will help you more clearly design the types of actions to take during your monitoring initiative.

During the preparation stage, it is also helpful to gather relevant information about the political power environment, the target political process, and the stakeholders and target audience. This can be achieved through a variety of methods, including conducting a political context analysis or power-mapping exercises, and conducting a survey of citizen perspectives and experiences. Depending on a country’s political atmosphere, you may decide to conduct other activities before undertaking a monitoring project. For instance, you may decide that it is necessary to first push for freedom of information laws in order to conduct a meaningful political process monitoring initiative.

Understanding the Political Power Environment

Having a solid understanding of the political power environment is critical to the success of monitoring initiatives: what power looks like, who has power and influence, and how civil society might be able to increase their influence. It is important to understand these power dynamics to mitigate risks and maximize success.

In Maldives, where NDI supported civic groups’ efforts to monitor progress toward commitments made by the government in the Government of Maldives Strategic Action Plan 2019-2023 and elsewhere, power mapping helped prioritize key audiences. Not only did the power map inform communications strategy, it also factored into the design of a website that could be used to more effectively communicate the results of monitoring efforts with these key audiences.

Therefore, it is important for you to conduct a careful power mapping of the political process and environment to identify the institutions and public officials that have some sort of decision-making power over the targeted process. This planning method involves determining the political actors at play, their chain of command and their level of influence.

Tip: The best way to determine the level of influence is to actually do some monitoring. What may seem on paper as the responsibility of one institution may actually be much more dependent on another institution.

Power mapping is also crucial in identifying potential supporters and opposers to the civic group’s objectives and actions. A power map like the template below can help you brainstorm the various stakeholders and the power, needs and opportunities of each.

Power MappingPower Mapping

To analyze the political power environment, you can also consider the following questions:

  • Who has the power and authority to provide the necessary information as well as absorb the information in a way that can bring about positive change? For example, if the monitoring initiative is targeting a specific political institution, can that institution absorb and respond to the results of the monitoring project?
  • What relationships do you currently have with these stakeholders? What power dynamics exist?
  • What is the political context that you are operating within? How open or closed is the political space?
  • How might the current political environment impact your political process monitoring initiative?
  • What legal frameworks currently exist that would either allow or hinder a political process monitoring initiative? What barriers or issues exist to accessing the necessary information?
  • Can the institutional infrastructure respond to the demands or requests presented in the findings from the monitoring report?

Understanding Stakeholders’ Perspectives and Experiences

It is also important to have a solid understanding of the perspectives and experiences of the monitoring initiatives’ stakeholders. Involving your key stakeholders in the design of your monitoring efforts will yield better results, because it makes their needs, behavior and motivations central to the design of your intervention. This approach will also help you stay focused on the problem you’re seeking to solve throughout every phase of your monitoring initiative. Ask questions to identify what some of the citizens’ key interests are. For example, you can conduct a survey of citizens in a target community about what they would like to know about the target political process, where they would like to be involved, and what information they would like to receive. This can also help you identify interest in participation, awareness and gaps that can help you define your monitoring goals.

Consider the following questions:

  • Who are the primary stakeholders (such as government officials, political parties, community members, business owners, private sector actors, civil service workers, etc.)?
  • What interests do stakeholders have in the results and outcome of the monitoring initiative?
  • Are there any implications related to making the monitoring initiative findings public? Could this expand or shrink political space?

Clarifying Your Goals

After gathering information about the political context and identifying the governance gaps and community needs and concerns, clarify your objectives and desired outcome from monitoring.

Tip: Be realistic about the scope of the initiative and the expected outcomes or results for democratic development. Recognize that most changes will take time and have to be negotiated. Expect that any change will involve a complex process requiring an interface with public officials. One of the most common outcomes of monitoring initiatives is changes in or new relationships with public officials and the monitoring group or citizens.

As discussed in the IMPROVE steps, monitoring organizations should develop and follow a clear, concrete monitoring strategy based on clearly defined objectives. You are more likely to carry out program activities effectively and efficiently when your actions are informed by a strategy based on clearly defined objectives. Consider the following questions:

  • Which governance dimension are you targeting through monitoring?
  • What is the desired change you seek to achieve by undertaking a monitoring initiative? Specifically, how can monitoring help to increase citizen voice, expand political space, and/or increase government accountability?
    • Are you seeking to increase citizens’ voice by providing them with information on the roles, responsibilities and performance of their elected officials or other government institutions?
    • Are you seeking to occupy existing political space or further expand it?
    • Are you seeking to hold government accountable?
  • What are the short-, mid-, and long-term objectives for the monitoring?
  • What skills or capacities need to be strengthened for your organization to be successful at implementing a political process monitoring initiative?
  • How does political process monitoring fit in with your organization’s existing mission and goals?

Once you have clearly outlined your goals and the questions you need to answer to achieve those goals, you can design the appropriate monitoring tools.

Monitoring Tools

In monitoring, data collection is key. Thus, it is crucial to have the right monitoring tools that will enable you to gather data in a systematic manner. When adopting monitoring tools, pick the one appropriate for your particular methodology. Prior to selecting or designing a monitoring tool, think about how and where you will use it. Will you go into official meetings of a public institution regularly and record what is and is not happening? Or will you go out in the field into communities where a program is rolling out and record what is and is not happening? What questions are you looking to answer with the information you collect from your initiative? Understanding the needs and motivations of the target audience will inform the approach to the tool you use, questions you develop, and the way the audience will ultimately access and engage with the data. Your monitoring tools enable you to set the scope and direction of your new monitoring initiative.

Some of the main monitoring tools used by civic groups in political process monitoring are outlined below.

Observation Checklists

Observation checklists are tools to record actions and behaviors of public officials in a political process. While they are mostly used to check actions taken or not taken in a public meeting or official gathering, they are often designed to record qualitative information, such as what an official said, the questions they asked or how they responded. Such forms are administered for every meeting or gathering attended by officials under monitoring and recorded in a database for processing and analysis.

Observation checklists are mostly used in parliamentary monitoring and campaign monitoring, but monitoring groups also use them for monitoring government follow-through to check if a program is following the guidelines or legislation of the program, as well as in shadow reporting to record government actions or inactions.

Citizen Report Cards

Citizen reporting cards are used for initiatives exercising social audits as a methodology to monitor a government project or program. They are administered by civic volunteers and enumerators and are conducted with service users, intended beneficiaries, or community members affected by the government project or program.

A common form of a citizen report card is the Public Expenditure Tracking Survey (PETS), which tracks a program’s expenditures. However, citizen report cards can also be used for monitoring various public services.

Tracking Sheet

Tracking sheets are spreadsheets or simple pieces of software with certain preset indicators to monitor government performance. They can be used to monitor institutional transparency by tracking publication of public information and documents online, as well as to track responses to access-to-public-information requests and other citizen requests. They are sometimes combined with observation forms, or the two can complement each other.

The Open Budget Survey19 that aims to monitor a country’s budget transparency may be monitored through a tracking sheet. Civic groups also use tracking sheets to monitor if the government is following all the steps of a political process as written in the legislation that regulates that process, such as the budget process or the law-making process.

Online Crowdsourcing Platform

Online crowdsourcing platforms are pieces of software developed to gather citizen reports or complaints for the purpose of monitoring government responsiveness and/or political accountability.

Platforms such as Fix My Community20 are designed to allow citizens to report a problem in their community or a corruption activity, and the citizens themselves, along with the administrator, monitor the government institutions’ response to those reports. They are also used by parliamentary monitoring organizations to gather citizen questions for MPs and monitor MP response rates. ​​Crowdsourcing can be an effective method of tracking government failures or crimes, but they can also further undermine political institutions and come with a host of problems of either malicious use or reprisals against whistleblowers. Tools like this are most appropriate when used to help governments monitor and follow through on concrete service delivery issues, like fixing potholes. Here too, crowdsourcing should be used with caution. For example, enabling citizens to report failures of service delivery without connecting those reports with the government authorities capable of resolving them could serve to undermine citizen trust in government.

As with any technology used in monitoring initiatives, crowdsourcing platforms should be designed and deployed with a human-centered design approach. Consider the needs and constraints of those who will use the platform to report, as well as the individuals who will receive the report. The profile of these audiences will help you determine what type of platform is best suited for your monitoring effort, as well as what information should be collected.

Data Mining Software

Data mining software is a tool designed to extract, or “scrape,” data from official online sources to monitor public officials’ actions. Such software is used by organizations to scrape data from official public procurement portals to expose potential conflict of interests and raise other red flags. It is also used by parliamentary monitoring groups to scrape MP votes and other actions from portals of parliaments. Data mining software is also becoming popular with groups monitoring social media posts of public officials and public institutions.

As a monitoring tool, data mining software must be integrated into an online platform that displays the information in an organized and structured manner. Thus, using this type of monitoring tool requires extensive IT skills. Nevertheless, once designed properly, this tool requires only some maintenance because it automatically processes and displays the scraped data.

Data Collection and Analysis Stage

There are five steps of moving from data to information: 1) data collection, 2) data organization, 3) data cleaning, 4) data analysis, and 5) data visualization. If using technology tools to facilitate data collection, it is important to ground your approach by considering your data collectors’ and target audience’s needs, motivations and limitations. Human-centered design exercises in the Co/Act toolkit21 can help you do this. The monitoring tools you use and surveys you develop will set the direction for how the monitoring initiative engages with its target audience, and will inform not only the indicators of data analysis but the communications strategy around the findings.

Data collection is what your monitoring team will do on a regular basis, using the monitoring tool that you design. After defining the target and parameters of the data collection and designing the research tools, as outlined in the monitoring preparation chapter above, the monitoring team will use those tools and the determined data sources to record the information and input it to a software program. This could be a simple spreadsheet solution like Google Sheets22, or it could involve more sophisticated software, such as SPSS Statistics.23 For most monitoring initiatives, though, at least in the early stages, sophisticated programs like SPSS are not required.

Tip: Monitoring groups sometimes use national surveys, simpler local surveys, or focus group discussions to collect data. CSOs and legislatures can use that information to shape policy and encourage direct citizen engagement. However, these tools entail some risk; see the Surveys and Selection Bias in the Types of Political Process Monitoring chapter above.

In adopting a monitoring tool, you should understand the difference between qualitative information and quantitative information. With qualitative data, you track behavior and practices. In quantitative data, you track satisfaction, frequency and availability. The analysis of qualitative data is more somatic, whereas analysis of quantitative data is more statistical.

Monitoring tools to gather qualitative information are made up of open-ended questions rather than just yes/no or ranking questions. Such tools can provide you with longer and more subjective answers. Focus groups or interview guides, which lay out planned questions with space for interviewers to record notes, are helpful to facilitate qualitative data collection. In particular, nondirective interviewing can be a helpful qualitative data-gathering technique. In nondirective interviewing, the interviewer does not frame questions in terms of right or wrong answers or limited sets of options and avoids leading the interviewee to answer in particular ways or within particular value systems. Instead, the interviewer uses an open-ended approach to explore the interviewee’s thoughts, attitudes and beliefs. Monitoring tools to gather quantitative data are made up of closed-ended questions, typically those with answers of yes, no or maybe, as well as rankings.

You should also understand the difference between primary data sources and secondary data sources. Primary data includes direct observations, whereas secondary data includes previously conducted research studies. You must gather primary data yourself with questionnaires and interviews, whereas secondary data can be gathered from sources such as reports produced by the government, international organizations or other non-governmental organizations; official statistics; or media articles.

Your approach to data collection will depend on the type of data you collect and how it is collected. Only once you’ve determined this will it make sense to look at what technologies can be used to facilitate data entry, management and analysis. Some tools are more useful with collecting structured data, but would not be helpful with free-form data that you might collect through qualitative interviews. Other tools — such as data mining tools — are more helpful in getting information from secondary sources. In some cases, digital tools might not be needed at all.

Data collection tools can be divided into three categories: data repositories, online forms and data mining tools. First, data repositories like Airtable24 or Google Sheets25 — basically, spreadsheets — are used to organize structured data. Information can be entered directly into the data repositories in a tabular format. However, this is not very user friendly and can lead to messy datasets, especially if more than one person is working on the database or there is no standardized format.

Second, online forms make it easier to input data into the repository in a structured format. More structured data facilitates analysis and visualization. Simple online forms, like Survey Monkey26 or Google Forms,27 can be used to limit human error by validating entries, by, for example, only allowing valid email addresses to be entered into an email field. Advanced survey tools that take advantage of modern technologies, like smartphones, can even collect data passively to validate data or simply collect additional data for analysis. For example, smartphones with geolocation services can collect data about the location of the person collecting the data and the time they collected it, in a way that would be hard to falsify.

Sometimes it is necessary for political process monitors to collect information directly from their research subjects, such as with opinion surveys, crowdsourcing or through the work of trained field-based observers. Crowdsourcing or crowdmapping tools like Fix My Community28 can help improve government service delivery through citizen reporting. Monitoring efforts that involve field-based observers can take advantage of structured data collection tools like Apollo29 to collect observation reports in real time. In both cases, it is important to consider how the medium might impact the collected data. For example, an online survey of a community will be biased toward individuals with internet access (see Surveys and Selection Bias in the Types of Political Process Monitoring section for more information). If field-based observers will be in areas without internet access, phone calls or SMS may be preferable to online forms.

Third, data mining tools take publicly available data and use it for analysis. Some platforms and websites enable direct downloads of structured data via application programming interfaces (APIs) or really simple syndication (RSS) feeds. Pulling data from APIs is in most cases legal and ethical, since the API data is deliberately regulated by platforms and structured so as not to violate users’ rights. In cases where data is only available in formats that cannot be readily downloaded and analyzed, such as information on websites or in PDFs, you might need to make use of data scraping tools. Web scraping tools like 0archive,30 for example, can be used to extract data from websites or social media platforms and convert it into structured data sets. Web scraping is not regulated by platforms and may violate their terms of service or even be illegal in some contexts. Tools like DocumentCloud31 or Adobe Acrobat32 can help extract data from PDFs or image files. Amazon Textract33 can also extract relationships or structure, which can help automate the process of organizing data extracted from PDFs into a structured data set. While these tools are helpful when dealing with massive amounts of data, for smaller monitoring projects, manual data entry might actually be more efficient.

Tip: Take precautions to ensure the security of the data collection tool and the privacy of the data, particularly when using online tools to collect data that involves sensitive or personal identifiable information. In some contexts, as with the European Union’s General Data Protection Regulations,34 data protection or localization laws must be followed when ​​collecting and processing this personal data. Investigate the applicable legal framework and collect only the minimum data necessary for your monitoring efforts. Regardless of your monitoring plan, take some time to assess and mitigate potential physical, digital and information risks that could arise as a result of your work, and reference resources like the NDI’s Cybersecurity Handbook for Civil Society Organizations’ chapter on “Creating your Organizational Security Plan35 to work through a risk assessment process and access security resources and tips as needed.

Most software-as-a-service platforms have Terms of Service and Privacy Policy pages that explain users’ rights and how the company handles personal information. The section on Communicating and Storing Data Securely36 in NDI’s Cybersecurity Handbook For Civil Society Organizations has additional information on this subject.

Data organization and cleaning occur together and are about categorizing and structuring your information, to, e.g., rank or compare the data, etc. After you have adopted your monitoring tool and collected data through observation and other means, you have to organize and clean that data to turn it into useful information. This is done by finding and highlighting patterns that are analyzed from a contextually relevant perspective. Especially when working with secondary data, multiple datasets may need to be collected from more than one source in order to get a full understanding of the political process you are monitoring. Merging these datasets involves matching and reconciling related fields and reconciling duplicate references. It is highly recommended that you work with a data scientist who has experience working with the type of data you collect. How you organize and clean the data will impact the types of questions you are able to answer during the data analysis process.

It may take hundreds or thousands of data items gathered through monitoring tools and access to existing data to generate information that is useful as evidence of whether a political process is working as expected. Data may come from institutions, actions and behaviors, as well as from citizens’ perceptions. You may have both quantitative and qualitative data. That data must be organized and cleaned to generate information that can be useful for your purposes.

While quantitative data is inherently structured and lends itself to analysis, the organization of qualitative data is best performed through coding: going through the data, creating themes, and then analyzing that data through themes, without preconceived biases.

Data analysis is not only about aggregating information and producing statistics. Analysis is most helpful when you interpret what the information means and draw conclusions that can lead to actionable, evidence-driven recommendations. For example, does the analysis show some transparency? Does it indicate an institution’s lack of will or capacities? Is the public voice included in the political process, and have institutions created political space for citizen participation? Is there accountability to citizens and other institutions?

Data visualization is important in political process monitoring, just as it is in research, because it makes information easier to understand and can present a lot of information in a visual, concise way. Moreover, good visualization can be an effective storytelling tool to communicate what your data means in a compelling way. Charts, infographics, dashboards with analytics, and video clips with illustrations are all types of data visualization techniques. However, poorly done visualizations of data can actually do more harm than good, presenting data in a misleading way. For more complex datasets, graphic designers and data scientists may be needed to represent the data both accurately and intelligibly. For more simple visualizations, available tools make creating simple graphics possible for individuals without specific expertise. Tableau,37 Carto38 and Mapbox39 are particularly suited for datasets with location data. Infographics creation tools like Infogram,40 Canva41 and Piktochart42 make creating simple visualizations a snap. Google Sheets43 and AirTable44 (using extensions) even have some basic visualization features for creating charts and graphs from within the data repository tool.

Recommendations

  • Data collection can take longer than expected, so allow plenty of time to set up good systems and processes for data collection, entry, storage, management, analysis and reporting.
  • Consider outlining any assumptions about the target audience, their needs and motivations, and how they will engage with your monitoring findings. From there, build research questions to evaluate these assumptions and ground the monitoring initiative design and communications plan in an approach that will have the most impact on the target audience.
  • Consider doing an independent review of your methodology to check if your tool can provide quality data and to check that the information you are gathering will be essential for your goal.
  • Provide early feedback to your target audience on the initial findings and allow an opportunity to discuss these results. You can always tweak and revise your monitoring process as you go if it makes your initiative stronger.
  • Monitoring is often a long-term process, and the people involved can change over time, especially in the political sphere. The process should, therefore, be documented and filed in a location accessible to the appropriate people in your organization.

Communicating Your Monitoring Work

Developing Strategic Communications

Communicating and disseminating your findings and recommendations to stakeholders is key to making improvements in the target political process. Effective communication requires developing successful communication strategies and strong messages that can engage key decision-makers as well as other important stakeholders. A communications strategy is rooted in an understanding of the desired change, the roles and incentive structures of key actors you are trying to reach to make this change, and the information environment.

Both your communications of monitoring initiatives and the channels you use to communicate should be targeted to your intended audience and presented in an engaging and accessible format. Important questions to consider include:

  • Where do these groups currently go to find this type of information, and under what circumstances? How can you meet them where they are?
  • What barriers do they face to accessing or engaging with such information, and how will this communications strategy overcome those?
  • What motivations do they have for engaging with your monitoring initiative and findings, and how can you leverage those?
  • How can your communications motivate key actors to engage over time in a sustainable way, whether through ongoing awareness, advocacy or implementation of recommended changes?
  • How can you build an approach that is inclusive, transparent and accessible?

If you’re not communicating your findings and recommendations effectively, the information you gathered and analyzed likely won’t have a significant impact, and your monitoring initiative likely won’t generate the change you seek.

A critical foundation of an effective communications strategy is trust. Building trust in your initiative and findings among the public and among your target audiences in particular starts with embodying transparency around your process and your goals. Presenting your methodology clearly and communicating it to your audience in an accessible and intelligible way, particularly doing so as early as possible in the process before you have collected your data, will allow the public to trust your intentions and process, so that your monitoring does not become politicized. Particularly in the field of political process monitoring, where various social and political pressures and dynamics are in play, it is important to think through the impact, both intended and unintended, that your findings may have. For example, think through any preexisting social or political tensions that could be exacerbated by the results of the findings. If the findings have the potential to alienate the government further, how can you communicate them in a way that encourages the government to make a change? If the findings have the potential to add to public dissatisfaction with government services, how can you present them in a way that instead inspires the ability to make a change?

One of the best ways to build trust in and engagement with monitoring findings, or any communications goal, is to build relationships with your target audiences. As previously mentioned, an inclusive and human-centered design approach in your communications strategy and tools development enables co-creation and ongoing communication with your target audience from an early stage. Bringing your target audience into the design process not only ensures the end product will be more user friendly and engaging, but also establishes a mutual understanding of your process and goals as well as your audience’s ownership and buy-in for a more impactful uptake, as they have invested time and effort, and have had their voices represented in the development process.

Developing messages

Effective communication depends on focused and persistent messages. Messages are short, straightforward, and easily understood pieces of information that raise an interest for more information. Messaging is the act of communicating your messages. Messages are useful tactics for opening the door to more widespread discussion of your monitoring findings and recommendations. Use the data and evidence produced from your monitoring to develop messages that articulate a call to action. Effective communications should take that data and tailor the messaging to address the interests and perspectives of different audiences, such as the media, government officials and civil society.

There are six attributes to powerful messages. These can be thought of as STRONG messages,45 which must be:

  • Simple, because the whole idea must be easy to grasp;
  • Touching, so that it can raise interest and/or elicit a reaction;
  • Relevant to the monitored process;
  • Optimistic, so that it seems like a necessary action;
  • Not exaggerated by going beyond what the monitoring evidence shows; and
  • Get repeated, because the key to effective messages and messaging is repetition.

There are two types of messages: positive messages and negative messages. Negative messages highlight what is not being done or what is being damaged. Negative messages can often strike strong emotions that garner attention. As such, some organizations choose negative messaging because they believe it will generate a faster response from the government; however, it is more confrontational and may hurt future opportunities for collaboration.

In contrast, positive messages highlight what needs to be done and improved. Because monitoring work can be sensitive and because access to that government process can be crucial to your monitoring, positive messaging is recommended. A communication strategy that uses positive messages seeks to uplift and empower government to use your monitoring as an opportunity to build trust and improve government services.

For example, your monitoring initiative may reveal evidence that the budget is not as transparent as possible, and citizens want more transparency into public finances. A positive message could be “transparent budget brings trust in government.” In another example, you may have monitored appointments in government ministries and found that women are less represented. A positive message could then be “women should have an equal voice in government.” Once you have developed strong messages that align with your key recommendations, you are ready to effectively communicate your monitoring work.

Advocating Change

Advocacy can be defined as a set of organized, strategic actions taken over a period of time to influence decision-makers and bring about a specific desired change. However, too often civil society organizations narrow down advocacy to focus primarily on meeting with and persuading decision-makers. To be effective, advocacy should aim to also build support among society for key issues and to mobilize citizens to demand action and change. A citizen’s ability to advocate for change is a fundamental method of holding leaders accountable.

In advocating your recommendations to decision-makers, you should engage other civil society organizations, the media, and citizens, as well as groups with interests and beliefs contrary to yours. Building awareness and support among wide-ranging stakeholders will help generate mass demand for the changes that you are advocating. Mass demand is what ultimately pushes decision-makers to initiate change.

Using existing CSO coalitions or forming new coalitions around principles that are aligned with your goals and your monitoring work is an effective way to build support and mobilization among civil society. Coalitions can help you amplify your voice and can extend your reach through their own relations with public officials, the media and other communities. Engaging the media is another important communications and advocacy strategy. The media has a broad audience that includes diverse citizens and decision-makers. It is important to engage the media by presenting messages that can be easily turned into headlines.

After garnering the attention of civil society, citizens and the media, you can turn your focus to influencing public officials. More information on engaging public officials and institutions can be found below.

Communications Tools

There are a number of communication tools that can be used to communicate monitoring findings and recommendations. Monitoring reports are the most common type of tool, but this section also covers other types of reporting tools.

Monitoring Reports

Monitoring reports are usually period reports that organizations make public on an annual, biannual, or even monthly basis. These reports should include certain key sections: introduction, methodology, findings, conclusions and recommendations.

In the introduction, the report outlines the reason for this type of monitoring, the importance of the target political process, who is responsible for the process, and the importance of improving the process. The methodology section is where you explain the monitoring approach, what tools you used, and what activities you carried out.

The findings section should report the information you produce through your monitoring and your data. This is the evidence that shows what is happening to a political process. Infographics that tell a story and portray your information in an easy-to-understand format are included in the findings sections.

The conclusion section includes your analysis of why you think something is happening or not happening. This is where you evaluate the causes for the gaps or deficits in the target political process and the root causes of government inaction to change the issue, such as lack of will, know-how or capacity.

The recommendation section details the recommendations that were developed based on your analysis of the evidence. This section provides specific proposals for improving the target political process and serves as a call to action for the target government institution. Every recommendation in the monitoring report should abide by five rules, also known as the LEADS rules.

Each recommendation should be:

  • Linked to the findings. It is not the time to bring up new issues that were not foreshadowed in the findings section.
  • Evidenced well in the report.
  • Addressed to a specific institution to be clear about who the call to action is for.
  • Doable — they should be within the power and capacities of the target institutions.
  • Short — not too general but not too long as to be confusing.

For example, if you’re recommending that an institution be more transparent, simply stating that “the institution should be more transparent” is too short and too general. Transparency is a broad term, and as such, a recommendation is more effective if it is concise but specific to the target institution and describes the desired standard or how to improve the process. However, you should avoid going into too much detail so as not to overwhelm the audience. Additional details can be included in the findings section.

A Monitoring Report checklist can be found in the Annex to assist you in writing an effective monitoring report.

Parliamentary Monitoring Reports

The most common products coming out of parliamentary monitoring initiatives are monitoring reports, although as the internet expands, more of the reporting is moving online through websites and scorecards. Groups have published parliamentary monitoring reports either annually or after each legislative session. The reports are generally written for MPs or local governments, donors and the media, but can also be distributed to the public. Depending on the purpose of the monitoring (openness, functionality or MP performance), the reports can contain recommendations on any or all of the following topics:

  • Fulfilling legislative standards requirements;
  • Enhancing capacities to draft and approve legislation;
  • Organizing public hearings as part of the legislative development process;
  • Capturing and acting on citizen priorities;
  • Implementing legislation; and
  • Making budget procedures transparent.

Performance Scorecards

Another reporting tool used to raise awareness and communicate findings is the performance scorecard. Groups typically publish legislative scorecards primarily for the benefit of citizens and other CSOs. Scorecards can be written in a narrative form or in a more heavily statistical scorecard format. The types of information revealed in legislative scorecards are much more focused on the “nuts and bolts” of legislative procedures than the recommendations laid out in monitoring reports, and have included information on:

  • Legislator attendance in plenary and committee meetings;
  • Participation in CSO roundtables and debates;
  • The number of laws debated; and
  • The number of meetings with stakeholders initiated by legislators.

Websites and Social Media Pages

Websites can be powerful tools for raising awareness of the monitoring initiative’s findings and recommendations. Moreover, software tools that can process, analyze and visualize parliamentary data, known as “parliamentary informatics,” can be integrated into websites. Websites can also be used to hold legislators accountable and increase interaction between legislators and their constituents. The findings and analyses on monitoring websites can provide citizens with the information they need to begin advocacy or organizing campaigns, or simply allow them to make more informed decisions about how they will vote in coming elections. Monitoring websites can also contain contact information for legislators, which can improve access for citizens. More recently, websites and social media pages have been used to facilitate questions and answers between legislators and citizens on matters discussed in parliamentary proceedings.

Examples of the kind of information included on monitoring websites include:

  • Consistently updated parliamentary developments;
  • Monitoring reports;
  • Background documents on parliament, parliamentary blocs and committees;
  • Parliamentary news and studies;
  • Articles and brief analyses of monitoring data written by partners’ observers and researchers;
  • Comments posted by citizens on social media;
  • Citizen questions and MP responses;
  • MP bios; and
  • MP contact information.

Contact Relationship Management Systems

In addition to websites, awareness-raising and communications campaigns can also benefit from Contact Relationship Management (CRM) systems that allow you to more efficiently and effectively share information with the public. These tools enable you to keep a contact database of stakeholders and manage systematic communications with select contacts over time. For example, a CRM tool can be used to share monitoring reports and information with a group of interested stakeholders, identify key audiences based on their interests or actions, or even organize events or campaigns. NDI is a contributor to CiviCRM,46 the premier open-source CRM platform, which it provides for free to NDI staff and partners through its DemCloud hosting service.47

Infographics

Infographics, as the word implies, report info (information) through graphics. They tell a story in a visual and easy-to-understand manner and can help simplify complex data by “connecting the dots.” They can be used to concisely explain the target political process, who is responsible, and what the issues or gaps are.

Some creativity and technical skills are required to create infographics. However, there are plenty of resources to help you visualize your data that don’t require you to be an experienced visual designer, such as Infogram or Canva.48 These sites have various templates to help you get started and basic free versions that have some limitations but are sufficient for simple illustrations.

There are six steps to making an effective infographic: 1) identify the message you want to send; 2) select a template; 3) import your data; 4) add your message; 5) refine and simplify your message; and 6) test your messaging. If the message and graphic makes sense and are easily understandable to someone unaffiliated with your monitoring initiative, then it is probably ready to be disseminated.

Consistent branding can help improve the credibility of your infographics and ensure that they are accurately attributed to your organization if shared out of context online. This can be done by including your group’s logo, using colors consistently, or including a hashtag or hyperlink that leads back to more information about your monitoring initiative.

Town Hall Meetings

The data gathered through the monitoring initiative can also be used to inform town hall meetings, roundtables, or debates attended by a combination of legislators, CSO representatives, citizens, and political party members. Monitoring groups and grassroots CSOs have organized these types of events within their networks to provide an opportunity for citizens to directly and constructively engage with their representatives. Discussions and debates typically revolve around legislative roles and performance, citizen perceptions of legislators, and topics identified as problem issues in the monitoring findings. These public forums also provide an opportunity for you to explain the importance of the monitoring initiative, disseminate monitoring reports and other reporting products, and gather ideas for how to enhance future monitoring initiatives. These public forums have the potential to inspire citizens to use the monitoring findings to inform advocacy campaigns and other organizing and awareness-raising initiatives.

Hybrid or virtual town hall meetings can leverage technology to increase the reach of these activities. Through video conferencing tools like Zoom or Signal,49 citizens can be empowered to engage with representatives even if they cannot be physically in the room. Broadcasting and recording in-person town hall meetings can have a positive effect on transparency for both journalists and members of the public who are interested in the proceedings but might have barriers that limit their ability to engage in person. Telephone town halls, which use teleconferencing software to actively invite citizens to audio-only forums, can further increase access for people who might not have internet connections or computers capable of participating in Zoom conferences. This format of town hall can be particularly effective at reaching aging populations.

Strategic Engagement with Public Institutions

Monitoring is done to improve governance by improving a specific political process. Decisions for improvements are made by public officials, typically those who manage the target political process. Evidence from the monitoring report most likely will not be enough to move decision-makers into action. Your organization should be prepared to meet with the stakeholders of the targeted institution to persuade them to consider your findings and recommendations.

The subjects of any political process monitoring are political actors with various political interests. The interests and views of civil society and political actors may very likely collide at points during the course of the monitoring initiative, and public officials can oppose monitoring efforts if the interventions and recommendations challenge their interests and negatively expose governing failures. As such, building trust and proper relations with public officials is an essential part of any monitoring initiative. Be prepared to strategically engage public officials and institutions during the course of the monitoring initiative so that relations do not become adversarial to the point of shutting down communication and closing groups off from a process.

Officials and institutions have a public responsibility to be accountable and open to the public. But the way you approach them and the way you communicate findings and seek changes play a big role in how effective you will be in changing behaviors and influencing the decisions of public officials.

Engagement at Entry Points

In any political process monitoring, you will need access to institutions, officials, public information and documents, official meetings, and meetings with public officials. This is usually the first step to engaging public officials.

Civic groups that conduct parliamentary monitoring need access to parliamentary sessions and, possibly, committee meetings as well as access to draft legislation and supporting documents, legislative agendas, and MP’s questions to the government. Some parliamentary monitoring organizations also employ surveys with MPs as their monitoring method, which requires access to MPs. Groups monitoring follow-through of a government program need access to plans and decisions of such a program, as do groups that monitor for shadow reporting. Budget monitoring groups depend on accessing decisions on the budget process, budget documents and expenditure reports. Even campaign monitoring requires access to information and candidates for pre- and post-election periods.

In some contexts that lack right-to-information acts or have closed institutions and processes, access to information and meetings is left to the discretion and willingness of public officials. However, even if this access is regulated and considerably open, its implementation can depend on the public officials and public administration of the institution in charge of the process you are interested in monitoring.

Considering this, you must be strategic about engaging public institutions and public officials to gain access to your target process, alongside studying the legislation that regulates access to information and public institutions.

  Access to public information Access to official meetings Access to public officials
Campaign-related Monitoring
  • Candidate platforms (pre-election)
  • Executive decisions (post-election)
  • Government plans (post-election)
  • Candidate platforms (pre-election)
  • Candidates
  • Political party leaders
Parliamentary Monitoring
  • Legislative agenda
  • MP votes
  • Draft laws
  • Plenary sessions
  • Committee meetings
  • Oversight hearings
  • Speaker of Parliament
  • Secretary General of Parliament
  • Committee Chairs
Government Follow-Through
  • Government plans
  • Executive decisions
  • Working group meetings
  • Heads of Departments; Ministries
Budget Monitoring and Expenditure Tracking
  • Budget statements
  • Draft budgets
  • Budget requests
  • Expenditure reports
  • Citizen budgets
  • Working group meetings
  • Parliamentary committee meetings
  • Minister of Finance (national)
  • Department of Finance (local)
  • Committee Finance Chair
Shadow Reporting
  • Government plans
  • Executive decisions
  • Working group meetings
  • Heads of Departments; Ministries

Engagement at Intervention Points

Gaining access to public institutions goes a long way to ease monitoring efforts. But political process monitoring is a way to a means. Over the course of your monitoring activity, you will run into government actions, or lack thereof, that require interventions on your part to advance your goals. This involves engaging public officials politically to change actions, behaviors and practices.

Monitoring activities can expose fallacies in an institutional process that may be quite sensitive to public officials. Poor performance, lack of progress, mismanagement of public resources and corruption activity need intervention to correct. Naturally, this intervention has to come after monitoring activities mount sufficient evidence. But these are sensitive matters, and civic intervention in such issues requires careful engagement with public officials. Simply pointing out such practices identified during monitoring often does not yield meaningful results. Thus, you will have to engage in a more public debate with public officials and even follow their monitoring with advocacy campaigns.

Issue-based groups monitoring a government process may have goals that call for influencing a policy. Thus, after some monitoring, you will come to a point in the process when you will have to engage institutions in influencing policy.

Monitoring activities can be beneficial to groups seeking interventions in governance and policy, as it presents an opportunity to build relationships and trust with public officials. Nevertheless, protecting the objectivity of monitoring and the political independence of the civic group in monitoring is crucial to earn the respect of the target institution and public credibility.

Building Relations and Safeguarding Impartiality

Power-mapping exercises are also important for you to identify entry points for your monitoring. When it is difficult to access information, documents and institutions, you must find entry points among “supporters” or champions of open government within institutions to begin your monitoring. Using those entry points, you can begin to use your monitoring techniques to promote open governance.

Monitoring in order to open government involves relationship-building with target institutions and officials leading those institutions. Once some trust is built among the civic groups and the public institutions, those institutions eventually will move toward openness. Very rarely do institutions move toward closing down space, and only when public officials become hostile to civic involvement or media coverage. You must be careful to avoid such situations but also be prepared to use public pressure if it occurs. The best way to prevent closed governance and move toward greater openness is by gaining the trust of the institutions in the notion that monitoring efforts contribute to better governance and the institution’s improved performance.

One of the most common ways that civic groups build relationships with public institutions is by agreeing on a memorandum of understanding (MoU) when a monitoring initiative is undertaken. This is very common for a parliamentary monitoring initiative or monitoring a government program that is about to go into implementation. Even monitoring through “social audits” should seek formal support of public institutions, as they can help improve public contract management. Such MoUs between an institution and a civic group usually involve the institution guaranteeing that the civic group gets access to documents, meetings and other public gatherings, whereas the civic group guarantees political impartiality, agreeing not to damage the institution or the process with baseless criticism, and committing to offer constructive recommendations. Finding entry points and supporters within the system can be crucial to push through such MoUs.

While building relations with public institutions is important in civic monitoring, it is challenging to protect impartiality and objectivity in the process. Public officials are very concerned with their reputation and their perceived performance, and monitoring and detailed coverage of their work could be seen as threatening. That is why so often civic groups that do monitoring are labeled as working for some other force (opposition parties, someone’s agendas or even foreign agencies). Thus, public officials may want to influence or condition a civic group to allow space for monitoring only if they are portrayed favorably in performance.

It is important to note that the single most important trait a civic monitoring group can have is credibility. If a monitoring group loses credibility with the public, media or public officials because they may seem subjective to political influence, it is difficult for them to be taken seriously or have any positive effect in the long term. Maintaining impartiality and credibility to the monitoring goals can be difficult, especially where laws do not guarantee access to public information and groups rely on the discretion of public officials. In such cases, power mapping, strategic engagement of public officials, and time and patience become even more important.

Evaluating Risks in Authoritative and Semi-Authoritative Societies

Political process monitoring initiatives provide citizens with a mechanism to promote accountability and increase transparency of government actions. Ideally, monitoring government action should be a regular part of civil society’s role in the political process. However, in countries where political space is limited or closing, civil society oversight activities may appear threatening to those in power. Consequently, access to information may be limited and society’s ability to monitor government action is hampered, which may result in physical threats or detentions. In these contexts, risks need to be closely considered and managed. You need to become adept at protecting and maximizing the small amounts of political space available. In some instances, monitoring can be a way to expand political space and raise the voice of civil society in a meaningful way. When space is limited to monitor and hold government accountable, working in a network or coalition may be beneficial, as there is safety in numbers, and leveraging existing relationships is key to overcoming challenges of access to information and to disseminating findings.

Featured Case Study: Democracy Plus

About D+ and Its Initiatives

Democracy Plus (D+) was founded in 2016 by a few former observers of the Parliament of Kosovo who had become active in challenging national politicians to become better representatives and to strengthen the parliamentary democracy in Kosovo. Noticing that the power of the Parliament was constrained by certain processes that had an immediate and direct impact on people’s lives, the new organization began to look for ways it could use political process monitoring to address other accountability gaps in the country.

Over the next three years, D+ used political process monitoring techniques to monitor election campaigns, track government follow-through of promises and commitments, and track budget spending through procurement practices, in coalition with a few other organizations. At the local level, they worked with communities across Kosovo to identify and report their community complaints and priorities, and then tracked government responsiveness to such community issues.

Monitoring government follow-through became a priority focus for D+. They set out to monitor election promises and candidate pledges in election campaigns. They planned to do this by using the power of information technology and the ever-expanding internet bandwidth. Their initial aim was to use this monitoring to challenge the parties’ proposals for the next government. In TV debates, they challenged the parties in terms of the feasibility and viability of those promises. After elections, they wanted to monitor how many of those proposals were implemented by the parties that came into the government and to hold them accountable for their campaigns. D+ did this by creating a website, GovernanceNow,50 where D+ registered the 100 most-detailed campaign promises and then tracked their implementation. 

At the local level, D+ activists sought pledges from mayoral candidates to promptly address citizens’ public service complaints and integrate community priorities. In order to hold elected mayors to those commitments, they created a “fix-my-community” type of platform that allowed citizens anywhere in Kosovo to easily report a problem in their community to their local government using a mobile phone. The Fix My Community platform51 was funded by crowdsourcing, and was used to track local government responsiveness to seemingly small but urgent and vital problems for citizens. In addition, D+ worked with six different communities to prioritize a set of issues to be addressed by their local government and set up an online tracking platform52 to monitor the implementation of those community priorities.

Accountability Gap

Over the years, election campaigns in Kosovo have started to become meaningless in terms of party proposals. There were many outrageous promises — not based on any ideology, and seemingly not implementable, considering the capacities of the state (at least in some cases).

Moreover, there seemed to be no accountability of those parties for their election promises once they came into the government. Even citizens began seeing it as pointless to ask them about those problems. D+ wanted to make parties more thoughtful and realistic about their campaigns and more accountable for their election promises. D+ also wanted them to govern according to the program they campaigned on. In addition, D+ wanted the government to be more focused in campaigns on programs and projects that people wanted and cared for.

At the local level, mayoral candidates talked about how, when in office, they will work to address citizens’ complaints and priorities better than the next candidate. D+ saw it as necessary to create for the public easily accessible tools to measure this, in addition to giving citizens a space to report their complaints and priorities. These civic engagement information and communication technology (ICT) tools were designed to generate and measure local government response and increase accountability.

Monitoring the Political Process

During the campaign, D+ organized TV debates with candidates of different parties to find out what they promised they would do if elected and to challenge them on how they would implement such programs. D+ developed a monitoring tool through a spreadsheet to record, categorize and summarize those issues. Once the government was formed, D+ created a website listing 100 of the most concrete and measurable campaign projects proposed by the three parties who were now in government. D+ convened a conference in the early days of the government to showcase this website with the campaign promises that would be tracked across 10 different sectors. Subsequently, D+ used the spreadsheet from the campaign to track government planning, budget allocations, and decisions and actions on each of those promises.

Screen capture of the website (GovernanceNow, www.Qeverisjatani.org) where D+ registered the 100 most detailed campaign promises and then tracked their implementation.

Findings were communicated through press conferences and direct emails to key officials every two months. A conference with officials and civil society was held in the first month the government was in office to launch the tracking website and communicate its aim and the monitoring process, as well as to discuss the best ways for the government to make progress on the issues being monitored. At the local level, campaign pledges were drafted for a few targeted municipalities, which listed actions that mayors could take to ensure responsiveness and accountability to the voters. Parallel to that, D+ created online tools to enable citizens to address their concerns publicly and directly to local officials and track government responsiveness. Because the online tools were available to citizens and the concerns involved public institutions, citizens themselves were encouraged to become monitors and hold officials directly accountable for meeting their needs. They themselves became tools for social accountability.

Strategic Engagement of Policymakers

At the national level, the parties and governments were more alert once this website was made public. They were not directly opposed to it. However, when D+ started holding press conferences periodically to report progress on those promises, government officials began challenging some of the methodologies to discredit the monitoring work. Disagreements arose when government officials stated that a promise had been implemented fully, but D+ did not believe that to be the case, or when it was obvious that policies were not implemented to the extent that they were promised.

At times, D+ received complaints from civil society groups on why it was asking the government to implement some program that the civil society groups did not think was good for the country. D+ had to discuss with them about keeping parties accountable for what they promised in elections and for the platform on which they had won elections. In one TV debate, the Prime Minister himself tried to discredit the monitoring by arguing that the government had a list of other promises that they had already seen through. D+ had to make clear, publicly, that they only monitored promises that were concrete and measurable. As in any campaign, there are very broad election promises that cannot be measured objectively.

At the local level, mayors would occasionally try to evade responsibility by blaming contractors for delays when addressing certain public service complaints or the inability to structure administration to respond more promptly to citizen complaints. D+ activists had to discuss this constantly with mayors to address those dysfunctions. Moreover, D+ made it its duty to meet regularly with mayors and municipal directors to review progress for their municipal responsiveness to citizen complaints and pinpoint priority areas. Monthly progress reports were mailed directly to mayors and contact points in the administration. D+ also held information sessions for public officials on how to navigate the website.

Applying Technology

Building a website that was public and easily accessible to everybody was key to having an impact with this type of monitoring. The online tracking website, which was available on mobile phones, made it possible for this monitoring initiative to reach everybody: citizens all over Kosovo, journalists, civil society organizations who cared about the campaign issues, and government officials. It would not have been as effective if D+ had only published reports on this type of government follow-through. The updates were instant, whereas reports would have been more periodic — every two-to-three months or six months.

The website also became a monitoring tool, where D+ recorded new data and new information. D+ realized that, with today’s technology, integrating an ICT tool into monitoring is much more effective than making periodic monitoring reports. D+ also realized that they can be not only tools for communicating monitoring, but also monitoring tools themselves. D+ directly recorded what they had observed and tracked. However, to do this monitoring, it was key that citizens and government officials had instant and easy access to what D+ was monitoring and what the results were.

Screen capture of a map of citizen-reported issues from the Fix My Community platform (ndreqe.com).

 

Obviously, integrating an information technology tool was more work in the beginning, and it did have a cost. D+ had to decide how to design the online tracking website, the branding, the name of the website, and everything that goes into making a user-friendly website. It was challenging to design something that listed everything D+ was monitoring and tracking, and yet still have a site that was easy to navigate. However, this made the job of monitoring much easier down the road: D+ did not have to worry about producing monthly or periodic monitoring reports or about coordinating monitoring tools with NDI.

The Fix My Community platform (ndreqe.com) was adopted from open-source software offered to organizations by NDI. However, D+ wanted to customize it for Kosovo users by putting in an interface and cutting out a few steps for reporting. It turned out to be a larger, and longer, task than initially thought for adopting software that had already been developed. However, the app’s usability was drastically improved, and it was worth the extra hours when it came time to promote the website to communities and municipalities.

Evaluating Impact

The first impact of this campaign monitoring was that political parties began articulating their election promises better. The second, and more important, development was that parties in the government began paying more attention to their campaign programs. Third, citizens and the media had a tool to keep the government more accountable for their election campaigns. Social accountability in government was strengthened as citizens began taking on the role of watchdog directly.

Additionally, in the next election campaign, D+ noted that political parties were more careful and realistic about their promises and their platforms. This monitoring initiative generated more accountability and responsiveness by the government. It also generated more citizen participation in government affairs, as the people had the information about parties’ promises and their follow-through at the tips of their fingers.

The impact of the Fix My Community53 website ndreqe.com increasingly turned the attention of municipal officials to the needs of citizens and communities and provided a powerful tool to address their concerns and hold officials accountable. The website also became a useful tool for planning and identifying larger issues in neighborhoods, as citizen reports pinned on the municipality map pointed clearly to larger problems with waste management, degrading roads and damages to infrastructure.

Lessons Learned

Democracy Plus activists pointed out several lessons they learned during and after the monitoring process.

Invest time and effort at the beginning of the monitoring initiative to repeatedly stress the scope of the monitoring, what exactly the organization is monitoring and the objectives.

Manage media, citizen and politician expectations about the monitoring efforts. At times journalists asked why D+ was pushing politicians to move forward certain projects that some sections of society opposed. D+ had to explain the methodology and the monitoring objective of keeping politicians accountable to their campaign promises, rather than advocating certain projects.

Similarly, politicians questioned monitoring as being subjective because D+ monitored only a few of the campaign promises. D+ had to regularly explain that only “concrete and measurable” projects were being monitored precisely to be objective in their monitoring.

Create separate social media pages for the organization and the monitoring initiative to maintain focus on the initiative. D+, as a watchdog organization, had separate teams monitoring different political processes. Social media was an important communications tool. However, using the organization’s own social media pages often made it difficult to keep the audience’s focus on specific monitoring initiatives, and forced the organization to prioritize the different initiatives’ messages on social media or risk overwhelming the audience. Using separate social media pages for the different monitoring initiatives overcame these issues. Once D+ created separate pages for ndreqe.com, it began attracting citizens more interested in local governance issues, which was the focus of that monitoring initiative.

Invest in reaching citizens in their own communities to make them a part of the monitoring efforts, as well as inform and educate them better about the political processes you are monitoring. Ultimately, change and improvements in a political process occur if there is pressure from constituencies. While D+ made efforts to go into various locations across Kosovo to educate people about the importance of keeping politicians accountable and transparent and inform them about using their monitoring platforms, it was assessed as crucial that this should have been done more often and more broadly.

Democracy Plus began to put most of these lessons in practice over the course of their monitoring. The resulting impact demonstrates that monitoring is a continuous course of action for lasting change.

Campaign Monitoring Case Study

About Centre UA and Its Initiatives

In 2019, the Centre for United Actions (Centre UA), a Ukrainian civil society organization, launched an initiative to monitor campaign pledges and government decisions in order to strengthen democratic development and promote citizen participation in decision-making.

Centre UA tracks, analyzes and compares campaign pledges made by presidential candidates and political party platforms at the national level. In 2020, Centre UA expanded its monitoring efforts to track mayoral candidate campaign pledges and political party platforms in local council elections. In addition to monitoring the fulfillment of campaign pledges post-election, Centre UA monitors government decisions and analyzes compliance with good governance and democratic development principles. In particular, Centre UA is focused on monitoring pledges, policy platforms and decisions related to constitutional reform, anti-corruption and decentralization issues, as well as human rights issues, such as LGBTQI+ rights, freedom of speech, gender equality, and national language and memory policies. In addition to tracking and compiling information on candidate and party pledges and platforms, Centre UA analyzes candidate pledges and political party platforms to assess whether they fall within the scope and responsibilities of the respective office; whether they constitute a continuation or break from current state policy; and whether they align with democratic governance principles and Ukraine’s international obligations.

Monitoring the Political Process

Before launching the initiative, Centre UA conducted a political context and SWOT analysis to better understand the operating environment as well as the resources and capabilities needed to effectively implement the monitoring activities. Based on the analysis, Centre UA determined that maintaining a reputation as an independent, impartial monitor was key to their success. With NDI’s support, Centre UA developed its monitoring capacities by meticulously planning its monitoring methodology and hiring analysts with expertise in the thematic monitoring areas.

Centre UA gathers data from official sources, including the Central Election Commission (CEC) website and candidate and political party websites, and by submitting information requests to campaign headquarters. To verify campaign pledges, Centre UA sends each candidate a questionnaire54 to state their position on the key issues that fall within the president’s mandate, such as geopolitics, national security, democratic development, human rights and service delivery. The candidates are also asked to state their positions on top citizen priorities as identified in recent national surveys. To verify political party policy platforms, Centre UA sends each party a letter notifying them of the framework of policies expected to be covered in their platforms. Following the CEC’s deadline for submitting campaign documents, Centre UA follows up with a letter55 requesting information and a pre-filled questionnaire56 based on the information officially published on the CEC website. Once Centre UA receives the validated questionnaire, it conducts a comparative analysis of the party platforms and evaluates their alignment with democratic principles and Ukraine’s international obligations. To monitor and analyze the pledges and policy platforms in local elections, Centre UA monitors key policy areas that fall under the mayor and local council’s mandate in order to evaluate how they correspond with national policy and the city’s strategic development plan.

Communicating the Findings

Following a communication landscape and target audience analysis, Centre UA developed and piloted the initiative’s communication tools, which include an analytical “digest,” an online comparison tool and a podcast. To help citizens make informed voting decisions, Centre UA developed online platforms for users to easily compare campaign pledges and platforms in presidential and parliamentary elections57 and local elections.58 These websites compile information on each candidate and political party and serve to track and monitor fulfillment by elected officials. For each resolution, decree or bill, Centre UA produces a digest that includes the key details as well as a summary of who is affected by the decision, how it relates to citizen priorities and needs, and its alignment with good governance and democratic principles. Centre UA’s communication strategy also includes video explainers; infographics; social media campaigns; and analytical presentations for radio, print media and in-person events. In all of its communications outputs, Centre UA aims to make it easier for citizens to understand how different policies and decisions might impact them and democratic development in Ukraine. For example, Centre UA launched a campaign to raise citizen awareness around the president’s proposed constitutional amendments and the potential impacts through a series of explanatory articles, videos, and lectures in universities and regional centers, as well as presentations on TV and radio.

Evaluating Impact

In its evaluation, Centre UA found that over 17.3% of voters received information on candidate campaign pledges during the 2019-2020 presidential, parliamentary and local elections, and over 6 million people received Centre UA’s analysis on the decisions of elected officials. Through the ability to compare and analyze candidate pledges and platforms and government decisions, the initiative increased public demand for accountability and enabled citizens to make more informed voting decisions. Since launching the initiative, Centre UA has noticed a shift in the political conversation away from personality-based issues to policy issues as public interest in coherent campaign platforms increased and political actors demonstrated greater restraint in making populistic pledges outside their scope of authority.

Lessons Learned

  • Demonstrate your credibility by strictly adhering to the monitoring methodology and consistently producing impartial and sound analysis. Centre UA experienced pushback from some political actors and organizations with their own policy agendas, but were able to overcome any resistance by using this technique.
  • Maintain local level media partnerships. This can be challenging during elections as the media is overwhelmed by fast-changing, politics-related content and often do not have the bandwidth to publish analytical articles. To mitigate these challenges, Centre UA simplified their analytical articles and supplemented them with comprehensive infographics to meet the expectations of the media and their audience.

Parliamentary Monitoring Case Study

About DeFacto and its Initiative Objectives

DeFacto is a local watchdog organization in Moldova that began monitoring presidential campaign promises in 2017, and by 2019 had expanded its efforts to include monitoring MP campaign promises and performance.

DeFacto was motivated to begin monitoring campaign promises and MP actions after observing a lack of MP accountability toward fulfilling their campaign promises. DeFacto also identified limited citizen knowledge of MP promises and achievements and a lack of awareness of MP roles and responsibilities.59 Its objectives were thus to:

  • Improve Member of Parliament (MP) accountability to citizens, through fulfilling campaign promises, solving major voter issues, and making the widest possible use of the MP powers provided by law; and
  • Increase citizen awareness of the responsibilities of MPs and demonstrate how responsive and open the MPs are to citizens in order to help voters make more informed decisions.

DeFacto began monitoring MPs’ campaign promises and performance as a means to promote greater accountability and transparency, improve MP performance, and generate greater public trust in parliament. DeFacto monitors MP performance related to the three primary functions of MP legal authorities, as provided for in the constitution: 1) actions during parliamentary procedures, 2) intervening between the administration and citizens, and 3) listening to and gathering information from citizens on the issues that matter to them.

Monitoring the Political Process with Technology Tools

At the beginning of the initiative, DeFacto’s monitoring target was MPs from newly formed single-mandate districts. Single-mandate MPs, at the time representing 51 out of 101 MPs, are charged with representing a geographically distinct group of voters rather than their parties or county as a whole. DeFacto monitored 46 single-mandate MPs, including three from the diaspora and two from separatist regions.

To mobilize the community and other organizations around this initiative, DeFacto works with 18 accountability partners. These CSO partners are responsible for tracking MP activities in their respective regions. They receive training from DeFacto and NDI on how to gather and verify data and how to represent their public constituencies. The accountability partners regularly monitor the respective MPs’ activity and collect the data to produce monthly reports. Data is collected from parliamentary plenary sessions, which are livestreamed; standing committee sessions; the official Parliament website, which lists information such as attendance and bill sponsorship; and DeFacto’s Facebook social media account. After reviewing the data collection process, DeFacto developed online reporting forms through Google Forms for MPs to self-report and provide evidence of their activities. DeFacto cross-references these responses for accuracy. As part of the monitoring initiative, DeFacto developed two tools: the MP Accountability Tracker, an online platform that tracks and evaluates MP’s official activities, and an online survey tool, Părerea Ta (“Your Opinion”).

For the MP Accountability Tracker, DeFacto developed a weighted point system to calculate an MPs accountability to their constituency, which is then ranked against the other MPs. Points are awarded based on a weighted criteria, such as proposing amendments or attending committee meetings. MPs receive double points when a certain legal authority is used to fulfill a campaign promise or to fulfill a direct response from citizens. These ratings serve to generate public trust in their representatives and motivate MPs to be more responsive to their constituents’ needs and transparent in reporting their activities. The website extension for the MP Accountability Tracker was created by DeFacto’s developer and is hosted on DeFacto’s website, where the information, including the MP ratings, the data collected and the rating criteria, is made publicly available. DeFacto reviews the tracker methodology and updates the online platform on a monthly basis.

The Părerea Ta survey tool helps DeFacto’s accountability partners, elected officials and CSOs collect and aggregate citizens’ views on different issues. Users can design surveys that target specific groups or the general public, and receive answers online in real time. With this tool as well as with the accountability tracker, which provides the contact information for each MP reviewed, DeFacto aims to provide constituents with channels of communication with both civic and elected leaders.

In order to collect user feedback on the information DeFacto publishes and improve the user experience (UX) design, DeFacto has integrated the website data analytics widget Hotjar on its accountability tracker website. Similarly, in order to collect feedback and communicate with respondents, owners of the Părerea Ta surveys can collect respondents’ contact information and compile the contacts using a downloadable MS Excel file. Both tools were developed from scratch and tested along the development process to make improvements by incorporating feedback from DeFacto team members, NDI staff and partners. DeFacto continues to refine Părerea Ta based on user feedback and consultations with NDI and other partners.

Strategic Engagement of Policymakers

Throughout its initiative, DeFacto has aimed to establish relationships with MPs and build trust in its work as a non-partisan organization. Initially, DeFacto used a variety of channels to communicate the purpose of the monitoring initiative with MPs and to request information, such as Facebook, phone calls, SMS messages and Viber. However, DeFacto received limited responses from MPs and, moving forward, decided to only use official communication channels to avoid any appearance of bias. DeFacto presented the Accountability Tracker to a number of MPs, and managed to secure commitments from some to submit monthly reports on their own activities to DeFacto. However, some MPs have been more willing to collaborate with DeFacto than others. DeFacto has faced some claims from lower-ranked MPs who argue that DeFacto does not have the authority to judge them. While the leading political party has historically been less supportive of the initiative, some of its members received high rankings and, as such, the initiative has not received significant pushback from the administrative bodies of parliament. In fact, the leading party began publishing the results of their highest rated members on the party’s Facebook pages.

Communicating the Findings

DeFacto publishes the information and findings from its monitoring work on its website as well as in regular monitoring reports. To disseminate its monitoring results more broadly, DeFacto sends its reports as press releases to media outlets that then publish the findings. Some outlets have gone on to publish stories related to the monitoring findings, and one organization covers the ratings of the MPs from the region in which it is located.

As part of its communications strategy, DeFacto created communities of 70+ people on the secure messaging application Viber that are coordinated by an accountability partner or a local official. Organized by geographical location, these communities are a space for members to discuss the issue of accountability among their political representatives, the MP activities that are conducted in their locality, and the local priorities. DeFacto assists the communities with training on how to conduct strategic communications with MPs, and some communities have coordinated meetings with their MPs. Some MPs have also held meetings with the accountability partners, and the meeting outcomes are then communicated to the Viber communities.

Evaluating the Impact

Since beginning the monitoring initiative, DeFacto has seen an increase in MP responses to their requests for information on their activities. DeFacto’s more recent publication of its findings saw a public response from citizens taking to social media to vocalize support to MPs who received high ratings and to call out those with low ratings. DeFacto plans to continue raising awareness of the initiative’s tools and increasing public awareness and transparency around the issue of MP accountability.

Lessons Learned

  • Recognize the iterative nature of the monitoring process, including the development of tools, which requires continuously searching for ways to improve the methodology and the accessibility of tools to help citizens engage. For example, after feedback, DeFacto developed QR codes to allow citizens to more easily access Părerea Ta.
  • Publish regular monitoring reports to increase awareness of an organization’s work and build its organizational reputation. DeFacto found that publishing regular reports increased its public recognition and its status as a trustworthy monitoring organization. DeFacto is seeking to publish additional content, such as blog posts and different types of reports, to further engage the public on information around key issues.
  • Start by working with a small group of elected officials who believe in the issues and can “champion” accountability initiatives to begin to shift the political culture toward one of accountability and responsiveness.
  • Make data accessible to the public so that interested parties can use the data for their own initiatives. DeFacto is shifting to an open data concept for their website so that the public, civic activists and the media can use the raw data for their own activism or initiatives. This helps citizens perform their own monitoring work.

Government Follow-Through Case Study

About Pace on Peaceful Pluralism and its Initiative Objectives

In April 2019, Pace on Peaceful Pluralism (PoPP), a civil society organization focused on budget monitoring, expenditure tracking and subnational parliamentary monitoring, began an initiative to track state education expenditures and monitor the implementation of a school construction program in Myanmar’s Bago region.

The regional government planned to restore or construct over 600 primary and middle schools in the region. The government faced many questions related to their ability to deliver quality schools that meet established standards, effectively and efficiently implement the program on time and on budget, and respond to the needs and interests of the stakeholder communities. Through the monitoring initiative, PoPP aimed to improve service delivery, promote greater transparency and accountability of the regional government and government contractors, and increase space and opportunities for civil society and local communities to engage in decision-making processes for the public-funded education projects. PoPP focused on monitoring the legal obligations of the Bago government and the contractors in the implementation of the program and individual construction projects and the responsiveness to communities.

Monitoring the Political Process and Strategic Engagement

To gather data, PoPP conducted desk research, interviews and direct observation visits. Before launching the primary data collection process, PoPP carried out extensive desk research of the plans and legal obligations of the government’s school building program. This research helped identify information gaps and enabled PoPP to establish standards against which they could track and monitor. Through publicizing information on the government’s region-wide school infrastructure improvement plans, PoPP was able to begin raising citizen awareness and interest in the initiative.

After identifying the information gaps, PoPP submitted requests for information to the government. However, at the time, the right to information was not legally established in Myanmar. Therefore, PoPP prioritized relationship-building and entered into a dialogue with the regional government in order to ensure access to information and to lay the groundwork for constructive engagement with their findings and recommendations. In consultations, PoPP explained how civil society monitoring would help hold the contractors accountable, promote more informed and involved communities, and gather detailed data on the status of the construction projects that the government may not otherwise be able to access. Through sustained strategic engagement, PoPP was able to receive some of the requested information on the program’s budget, procurement processes and construction plans.

To support data collection, PoPP developed monitoring tools to track the implementation of the school building program in its entirety as well as the status of individual school construction projects. These tools included targeted questionnaires and a facilitator guide for interviews and focus group discussions with the school building committees and other relevant officials, as well as a checklist to support direct observation visits at the construction sites. PoPP also developed a spreadsheet to track the implementation of the entire program, including the budget and the procurement process.

PoPP trained over 100 volunteers to administer the monitoring tools in their communities. PoPP processed and analyzed the data, finding significant delays and incomplete projects throughout the townships in the Bago region. PoPP found delays in implementation, failure to adhere to legal requirements to adequately inform communities about the construction project, and insufficient gender and ethnic representation on the school building committees, among other issues. Based on their findings, PoPP developed recommendations to improve the effectiveness and efficiency in the program’s implementation and to ensure greater transparency, accountability and citizen participation at the community level.

Communicating the Findings

Despite the outbreak of the COVID-19 pandemic, which interrupted their communication plans, PoPP presented their findings to key regional government ministries and to members of parliament who were eager to learn from Bago’s program. PoPP communicated their findings and recommendations to the regional government but planned to continue monitoring and tracking the program until its completion.

Evaluating the Impact

By the end of the initiative, PoPP found that sustained engagement with government officials fostered greater awareness about the importance of transparency in large-scale development projects. One notable outcome was the government’s decision to post signboards at the project sites that provide information about the project’s timeframe, the responsible company and other facts.

Lessons Learned

  • Take time to adequately plan and prepare for the initiative, which proved crucial for PoPP’s success, particularly efforts to lay the groundwork for cooperation with the government in information-sharing and to raise awareness and interest among stakeholder communities.

Budget Monitoring and Expenditure Tracking Case Study

About the Institute of Economic Affairs and its Initiative Objectives

To foster greater public financial accountability and transparency, NDI partnered with the Institute of Economic Affairs (IEA), a Kenyan public policy and analysis think tank, to implement a budget monitoring initiative in 2019. In recent years, Kenyan citizens have increasingly voiced concerns about corruption and a pervasive lack of accountability and transparency in the management of public resources. In addition, recent legislation aimed at curbing corruption by introducing new requirements for reporting and evaluating public expenditures has fallen short due to insufficient enforcement and oversight mechanisms. To promote a culture of transparency and accountability in public expenditures, the monitoring initiative provided citizens, civic groups and accountability champions with credible and easy-to-understand information on government appropriations and spending, areas of financial malfeasance, and gaps in public oversight and action.

Monitoring the Political Process and Communicating the Findings

IEA conducted a comprehensive analysis of key national budget documents as well as the Report of the Auditor General on the National Government’s Ministries, Departments, and Agencies. Based on this analysis, IEA identified critical gaps in public financial oversight and reporting and areas of misappropriation or malfeasance. IEA issued high-level recommendations aimed at increasing transparency and accountability in public financial management. In collaboration with NDI, IEA translated the highly technical information into digestible and actionable reports, fact-sheets and briefings that are accessible to a wide array of actors.

To disseminate the findings and foster more informed oversight and advocacy initiatives, IEA led engagement and outreach campaigns with diverse actors in government, civil society, media and academia. IEA convened government briefings on public appropriation and spending issues, as well as responsiveness to citizens. In collaboration with NDI, IEA conducted a power-mapping exercise to identify potential changemakers within government who demonstrated a strong commitment to fighting corruption. These “accountability champions” were equipped with the knowledge and tools to maintain public dialogue and pressure within government to promote culture change. As the program progressed, IEA saw an increase in engagement on issues by government officials and parliamentary committees responsible for performing key oversight roles.

IEA also convened representatives from civil society, academia and the media for expert briefings on gaps in fiscal and budgetary performance and financial responsibilities and to identify opportunities for greater public participation and engagement in the budget cycle and in public oversight of government fiscal responsibilities. In some instances, IEA combined the briefings with targeted trainings. For example, to strengthen the media’s role in raising public awareness through evidence-based reporting, IEA and NDI facilitated training for media practitioners and journalists on budget analysis and fiscal responsibilities and procedures. Following the training, various media outlets increased coverage of national budget-related issues and explicitly covered the gaps and recommendations outlined in IEA’s reports.

In an effort to leverage religious groups’ significant public following to raise awareness and enhance public demand for accountability, IEA facilitated a briefing with religious leaders on the findings and identified strategies for religious structures and platforms to amplify public voices on gaps in budget implementation and reporting. Following the briefing, the religious leaders expressed interest in engaging in additional advocacy initiatives related to public financial management accountability and transparency.

IEA also held a series of town halls and conducted social media campaigns to raise public awareness and promote online conversations. IEA created citizen-friendly fact-sheets that distilled the analytical reports into a concise, digestible language to support greater accessibility by a lay audience. The fact-sheets, which were disseminated at the town halls and through IEA’s website, Twitter and Facebook pages, generated higher levels of engagement and circulation than the analytical reports, while also driving increased traffic to the reports. Using “Tweet Chats,” IEA sparked significant online engagement and conversation around their conclusions on the Auditor General’s report. Engagements with the event’s hashtag #TrackingOurShillingKE numbered over 8,000, making this type of social media campaign IEA’s most effective for broad public engagement. NDI also created WhatsApp groups to support sustained communication among civil society actors who participated in IEA’s briefings. Members have used the groups to share real-time information on public finance issues, and to coordinate advocacy initiatives.

Evaluating Impact

Through this initiative, IEA succeeded in simplifying and publicly disseminating analysis and credible information on current patterns of risk and mismanagement in national government spending of public resources. Through IEA’s accessible reports, fact-sheets, and in-person and online outreach campaigns, the public gained a better understanding of the national government’s management of public funds. Citizens and civic groups now benefit from access to IEA’s fact-sheets and analytical reports, which provide an overview of fiscal and budgetary performance in simplified language that lends itself to public dialogue and advocacy initiatives. Public finance management has increasingly become a topic of conversation through enhanced media coverage and burgeoning online engagement. Briefings, town hall meetings, strategy meetings, and capacity-building sessions convened by NDI and IEA with participants across CSO, media, parliament and academic sectors have also forged a network of diverse accountability advocates. In addition, IEA benefits from more robust working relationships with key government financial offices and is in a stronger position to push for the timely and regular release of key budget reports. NDI and IEA intend to build on the analysis and public awareness-raising fostered at the national level to conduct activities aimed at increasing financial accountability and transparency at the county level.

Lessons Learned

  • Consider online engagements through social media platforms. These played an important role in expanding outreach to a wider audience beyond government actors, academia and CSOs with a technical focus on finance issues.
  • Understand the difficulty in identifying accountability champions both willing and well-positioned to continue independent advocacy. This underscores the challenge inherent in transformative culture change.
  • Be prepared for delays and lack of access. Delays in the release of government financial reports or a lack of access to information can inhibit the regular release of analytical reports and the interrelated capacity of accountability actors to address gaps and issues in a timely fashion.
  • Craft an inclusive communication strategy. The highly technical nature of financial analysis can present challenges to the broad dissemination of findings, and in fostering awareness and conversation among the general public and diverse accountability actors without financial expertise. It is crucial to craft an inclusive communication strategy that translates technical conclusions into everyday, nontechnical language. Presenting findings in digestible and accessible formats and making the findings and recommendations applicable to everyday life can better generate public conversations on public expenditure and fiscal responsibility.

Shadow Reporting Case Study

About IraQueer and its Initiative Objectives

Founded in March 2015, IraQueer is the first LGBTQI+ organization in Iraq. Their mission is to empower the LGBTQI+ community to organize and lead a movement in Iraq to create a country where LGBTQI+ people are recognized and protected. To achieve this mission, IraQueer provides direct services and undertakes education and advocacy initiatives.

Despite commitments under international human rights conventions and treaties, violence and discrimination against LGBTQI+ people in Iraq persist, and victims are guaranteed virtually no protection or redress under Iraqi law and the legal system. In response, IraQueer engages in advocacy around human rights protections for LGBTQI+ people at the domestic and international levels. Its objectives for this initiative are:

  • Promote international and domestic action to advance the rights and ensure greater protections for LGBTQI+ people in Iraq; and
  • Ensure that human rights violations against LGBTQI+ people are clearly documented and publicly available.

Monitoring the Political Process and Creating Shadow Reports

As part of this strategy, IraQueer has published multiple shadow reports highlighting the state of LGBTQI+ human rights and protections in Iraq. In partnership with other civil society and human rights organizations, IraQueer has submitted shadow reports to the U.N. Universal Periodic Review (UPR) Committee and the Committee on the Elimination of All Forms of Discrimination Against Women (CEDAW). IraQueer has also published its own baseline study, Fighting for the Right to Live, documenting the state of LGBTQI+ rights in Iraq.

IraQueer aims to use these reports to establish a public record of LGBTQI+ rights in Iraq and to raise awareness and mobilize international action to hold the Iraqi government accountable. By working through international mechanisms, building sustained relationships with international actors, and establishing a network of local LGBTQI+ allies, IraQueer seeks to advance the rights and ensure greater protections of LGBTQI+ people in Iraq.

To develop the shadow reports, IraQueer works with a coalition of other domestic organizations, such as an Iraqi feminist human rights organization, as well as international organizations, such as Outright Action International.60 For example, to draft a CEDAW shadow report, IraQueer partnered with Outright Action International, MADRE61 and CUNY Law School to analyze the data, develop strategic recommendations and draft the reports.

Because of the sensitive subject matter, the data collection process is slow and requires careful planning. Staff visited the target communities 5-10 times over the course of a few months. Visiting a community for a shorter period of time, over multiple visits, was a security measure for the data collector and a strategy to avoid raising suspicions among the community. The data collectors were from either the specific community they were gathering data in or the close surrounding area. Because they knew the local context, they already had a nuanced understanding of the dynamics around the interview topics and had relationships with many of the participants. This facilitated greater trust and helped elicit more detailed responses from participants. However, it was still a challenge to speak to the families of LGBTQI+ victims for a number of reasons, such as the family’s fear that the community would find out about the relationship, because they had previously disowned the victim, or because family members had victimized the LGBTQI+ person themselves.

To avoid raising the community’s suspicion and generating fear among the respondents, the data collectors conducted interviews without the use of any interview tools and did not take notes. As such, the data collectors frequently broke up the interviews into multiple parts over the course of a few days, asking a few survey questions each time, before returning to a safe space to record the survey responses.

After submitting the reports to the relevant international body, IraQueer engages in an inclusive advocacy strategy focused on expanding the local LGBTQI+ movement in Iraq and developing sustained support internationally to advance their advocacy priorities. IraQueer engages directly with the United Nations and other international platforms and organizations to raise awareness of the LGBTQI+ situation and human rights status in Iraq and to promote concerted international advocacy and engagement with the Iraqi government. By raising awareness and fostering commitment from international stakeholders, IraQueer hopes to encourage international stakeholders to convene behind-the-scenes conversations with Iraqi officials to advance strategic recommendations.

Domestically, IraQueer aims to increase the intersectionality in the Iraqi human rights movement, equipping LGBTQI+ people to join the movement and training other human rights activists to be more inclusive in their own advocacy work. Using the shadow reports, IraQueer also engages with members of parliament and government officials to raise awareness about the state of rights for LGBTQI+ people in Iraq and to advocate for government action to protect these individuals.

While high-profile support is important to advance strategic priorities, IraQueer recognizes that it is public officials at the local level, who are responsible for implementing laws and government services, who have the most direct impact on LGBTQI+ rights and protections. IraQueer uses the shadow reports to empower local officials with information about the status of LGBTQI+ people in Iraq and provides training on approaches and tools to better support LGBTQI+ people who request services or report violations.

Through outreach and trainings led by its 600+ volunteer network, IraQueer is also building a coalition of nongovernment organizations to support LGBTQI+ rights as part of their own human rights advocacy initiatives, including women’s rights movement, children’s rights and others. Through sustained engagement, IraQueer aims to have these other organizations contribute to future shadow reports.

Evaluating Impact

Although there have been some shifts at the national political level toward greater support and recognition for the right to life of all, regardless of sexual orientation, IraQueer still struggles to get direct meetings with officials and to get them to publicly commit to action. By working through international mechanisms and platforms, such as CEDAW and UPR, engaging with international stakeholders, and building a network of domestic LGBTQI+ allies, IraQueer aims to mobilize support for LGBTQI+ rights and increase public pressure on the Iraqi government to uphold its international commitments and obligations.

Lessons Learned

  • Undertake significant research and outreach. The international human rights system is complicated, and understanding the different bodies, platforms and procedures and how best to engage them requires significant research and outreach.
  • Build relationships with international, national and local partners. While the shadow reports feed into international mechanisms and help apply pressure on the government to take action, relationships are key to sustained commitment on LGTBQI+ issues and substantive political, legal and societal change. It is useful to build a multipronged advocacy strategy that builds an intersectional and inclusive local advocacy network that equips both LGBTQI+ people and other human rights activists to advance LGBTQI+ rights; educates and trains government officials at the local and national level; and engages international stakeholders in advocating for LGBTQI+ rights and protections. Building relationships helps to sustain commitment, and face-to-face meetings are important to generate greater investment in the movement.
  • Prioritize data confidentiality and security. When collecting data on a sensitive topic, the safety and security of data collectors and interview participants is a high priority. Before traveling to the field, the data collection team needs to consider how to adapt collection strategies to the local context and how to ensure data confidentiality and security. Therefore, it is important to plan accordingly, as the data collection process of the monitoring initiative will likely take an extended amount of time.
  • Manage and store data securely. Because of the sensitive and broad nature of the IraQueer’s data field on human rights violations against LGBTQI+ people, managing and storing the data in a safe, secure and accessible manner is a top priority. IraQueer has used Google Sheets to store and manage the data, but it is becoming increasingly challenging to manage and time-consuming to use. For example, IraQueer staff must manually search for relevant statistics (e.g., type of violation, geographic area, etc.) for the shadow reports, which is a slow and tedious process. As such, there is a need to identify or develop a more efficient and secure database.

Featured Case Study: Guxo

About Guxo and its Initiative Objectives

In 2021, Guxo, an Albanian youth-led civil society organization, launched an initiative to monitor public procurement in the education sector to enhance transparency, accountability, and citizen participation. Following a devastating 2019 earthquake, the Albanian government allocated 33.4 million for the reconstruction of public schools in Albania, aiming to build 37 new schools in Albania in 2021. Guxo’s initiative aimed to monitor the public procurements managed by the Municipality of Tirana for the school reconstruction program and analyze the spending data to assess whether key institutions were fulfilling their responsibilities, uncover potential cases of corruption or abuse, and identify gaps in transparency and citizen participation throughout the process. By presenting their findings and recommendations to key audiences, Guxo aimed to: 

  • Foster more responsive, responsible, and transparent public procurement processes; and  
  • Increase citizen awareness and access to information about public procurement.

Monitoring the Political Process

Prior to launching their initiative, Guxo conducted a political economy analysis to assess the power dynamics among stakeholders, the public procurement regulatory environment, the accessibility of procurement information, and other monitoring efforts focused on the government’s fiscal responsibilities. This analysis helped inform the planning and design of the monitoring campaign. 

Because of the technical nature of public procurement monitoring, Guxo also sought to build their knowledge and skills on Albania’s public procurement system and public procurement monitoring more broadly. With NDI’s technical support, the Guxo team learned about the legislation, institutions, procedures, and actors that manage Albania’s public finance system and the strategies and techniques to independently monitor procurement.  This allowed Guxo to produce an initial monitoring plan, including data collection targets and an analytical framework.  

Once they had the monitoring campaign parameters in place, Guxo held a series of workshops with their team to understand their key target audiences and how to take a human-centered design approach to implementation and communications. After conducting a power-interest mapping session and revising their target audience analysis, the team concluded the key target audiences of the campaign were teachers and the municipal government. They developed user personas that identified the needs and motivations of the target audiences relative to the campaign objectives, in order to ground the data analysis and communications plan in an approach that directly engages these audiences and better meets their needs and priorities. Following the workshops, Guxo finalized its monitoring methodology, data collection tools, and an action plan that outlined the initiative’s phases, timeline, and activities. Guxo’s monitoring tools included an observation form to track indicators for each school construction project, a survey for school community stakeholders, and a database in Excel to catalog and track the data collected.

Data Collection and Analysis

Guxo created monitoring tools to gather data from primary and secondary sources. Secondary sources included scraping publicly available data from open procurement sites and submitting information requests to the Municipality of Tirana and primary sources included conducting site observation visits and interviews with stakeholders from the school communities. Guxo conducted over two dozen on-site observation visits to project sites and surrounding communities to assess progress and gather stakeholder feedback. Utilizing the survey questions, Guxo met with different school community stakeholders, including teachers, students, and parents, to better understand their expectations for reconstruction and their experience to date engaging with the reconstruction process. 

The inclusion of site observation visits, as a direct form of monitoring, allowed Guxo to develop a relationship with the school community and bring them into the campaign. As a result, in addition to collecting data and gaining a better understanding of the local perspective, this built a relationship with the communities that enabled better follow-on communications and information sharing. 

NDI helped Guxo engage a public procurement expert to facilitate a planning workshop on data processing and analysis that included sessions on refining and organizing data, analyzing and comparing procurement contracts, and identifying complex patterns in the data. Guxo developed a framework for identifying integrity and collusion risks and developed corresponding risk indicators. For example, some of the risk indicators were to identify the tendency of single-bidder contracts and the buyer share of awarded contracts. Based on this framework, Guxo broadened its data collection efforts to include historical data over the past three years to incorporate a comparative analysis, which provided new insights into the risk indicators.

After completing the data collection and reviewing the risk indicators, Guxo analyzed the data to identify critical transparency, accessibility, and citizen engagement gaps as well as areas of risk for malfeasance or corruption. Following the analysis, Guxo developed a series of recommendations for its key audiences designed to:

  • Promote more responsive, responsible, and transparent sharing of public procurement data online by the Municipality of Tirana; and 
  • Create more opportunities for public consultation on reconstruction efforts and raise awareness of existing channels for citizens to engage with the Municipality of Tirana on public procurement projects. 

In order to achieve these campaign objectives, Guxo developed a strategic communications plan that would ensure the messaging around the recommendations resonated with their target audiences in ways that would call them to action.

Communicating your Monitoring

To disseminate their findings and recommendations, Guxo developed a monitoring report and targeted communications strategies for its target audiences - teachers and the Tirana Municipality. With an understanding of the core monitoring findings and recommendations, Guxo continued the human-centered design process to understand how their target audiences would engage with this information and how best to craft the messaging to achieve the campaign’s objectives. Through a series of exercises, Guxo began by confirming two key audiences to prioritize their outreach to the Municipality of Tirana and public school teachers. For each target audience, Guxo brainstormed the messages and type of engagement that would most directly resonate with them. Guxo built user stories for how different subsets of actors within the target audiences would engage with the campaign’s findings and recommendations based on their needs and motivations. This allowed Guxo to explore and challenge assumptions around what format of information and messaging would reach the audiences most effectively. To validate the user stories and messages, Guxo conducted a group interview with teachers from local public schools to gather their perspectives and feedback on the findings and communications strategies. These exercises helped uncover any assumptions about the target audience’s needs and motivations, examine local power dynamics, and identify potentially sensitive topics. Based on these insights, Guxo was able to refine the communications strategy to elicit more powerful responses from the target audiences in ways that would advance their monitoring objectives. 

Following the feedback sessions, Guxo held a prototyping session to design two products for its target audiences. One product was a draft of an online public procurement database designed to demonstrate to the Municipality of Tirana how and in what format public procurement data could be presented in a more accessible format. To ensure political sensitivity, Guxo brought the government into the process by meeting with representatives to brief them on their findings and recommendations that were forthcoming. By briefing the representatives ahead of the official release, Guxo aimed to establish trust and a relationship with the municipality that could facilitate greater openness to listening to and adopting the recommendations. In their messaging, Guxo also leveraged the municipality’s current political objectives, for example, Tirana’s reputation as the 2022 European Youth Capital, to promote greater openness and collaboration. 

The other product involved a webpage on Guxo’s website with concise, easy-to-understand information for teachers and the broader school community to learn about the status of the school reconstruction projects, how to engage in the process to gather information, and provide feedback to the municipality, and the public consultations held to-date. This webpage presents a simplified version of Guxo’s monitoring data that tailors the messaging to share information that is most relevant to the school communities and motivates engagement, rather than emphasizing the technical details or negative findings. The goal is to raise awareness among school communities around not only updates in the school procurement processes and the monitoring findings, but also about the benefits of engaging in the public procurement process for school reconstruction and the type of communication and accountability they should be able to expect from the municipality. Adopting an iterative approach, Guxo intends to regularly observe engagement with their webpage to adapt the messaging as needed and is considering expanding on it to create social media and WhatsApp campaigns to present the school communities with accessible information on how to get engaged.

Evaluating the Impact 

In the fall of 2022, Guxo was still in the early days of communicating the findings and recommendations as well as evaluating the impact of its campaign.  Through gathering critical information about the level of citizen awareness and engagement in the school reconstruction process and the accessibility of public procurement data more generally, Guxo was able to channel this insight into carefully crafted recommendations and targeted communications strategies. Through regular engagement with school communities during the data collection process, Guxo found that students and parents began showing an increased interest in the reconstruction process and expressed a desire for more information about how to engage and advocate for their interests. Guxo intends to build on its connections with the municipality and the school communities, particularly students, to serve as a bridge to enable greater communication and to advance mechanisms for students to channel their interests and feedback to the municipality.

Lessons Learned 

  • Understand the skills and knowledge needed for public procurement monitoring. Evaluate internally the skills and knowledge that the group brings to the table and where there are gaps that need to be developed, and bring in the needed expertise as early on in the monitoring campaign as possible. In particular, this is true for the skills needed to understand the public procurement system and to develop and implement the monitoring initiative accordingly, particularly the data collection and analysis.
  • Plan to dedicate enough time to the preparation phase and the data collection phase. One of Guxo’s major takeaways is the need to dedicate sufficient time to data collection. Depending on the monitoring purpose and the transparency and accessibility of the data, the data collection process could take multiple months. Relatedly, it is important to build in additional time for data processing and analysis, which in Guxo’s case, led to an additional round of data collection before the analysis was able to be completed. To support effective planning and implementation of the campaign, it is critical to determine data analysis indicators when finalizing the monitoring methodology and to develop the communications plan as early as possible to prevent delays during implementation. The timeline will be different for each project, but Guxo’s general recommendation is monitoring preparation (months 1-4), data collection (months 3-8), data analysis (1 week), communications (ongoing). 
  • Consider how to leverage the unique advantages of youth in monitoring. Public officials might be more willing to share information. 

 

Data Collection Challenges

During the data collection process, Guxo faced challenges collecting information because the data in Albania is not fully machine-readable. This increased the challenges related to the data collection process because Guxo needed to use different strategies to access information, including manually extracting the data and cross-checking it across several sources. As such, the data collection process proved more intensive regarding the time and manual labor required than Guxo had initially anticipated. Guxo also ran into difficulties with certain public procurement data not having been made publicly available at all. As a solution, Guxo conducted outreach to the relevant authorities to request access to the missing information. This further slowed the data collection process as Guxo frequently needed to submit multiple requests to different institutions before receiving the information. Based on these experiences,  Guxo subsequently issued a recommendation to the Municipality of Tirana to strengthen practices for both open and accessible data.