United Nations, New York
On April 8, 2013, IDLO’s Director-General Irene Khan delivered a statement on “Measuring the Effectiveness of Rule of Law” at an Open Roundtable on “Measuring the effectiveness of support to the rule of law” held at United Nations Headquarters in New York. The event was organized by the UN Rule of Law Unit to foster discussion on the best available mechanisms to measure the effectiveness of rule of law.
I would like to commend the UN Rule of Law Unit for taking this initiative. It is timely, given the interest in rule of law across the UN system and the international community, and especially in the context of the post-2015 development discussions.
I speak from the perspective of a field-based rule of law organization. We are the only inter-governmental organization exclusively devoted to the promotion of rule of law. We operate both in countries emerging from conflict, such as Afghanistan, South Sudan, Somalia and Kyrgyzstan as well as middle income countries such as Kenya. We began our rule of law assistance work in Afghanistan in 2003, and we are now the largest provider of training in the justice sector nation-wide in Afghanistan.
IDLO introduced Results Based Management of its own projects and programs some five years ago, and is constantly reviewing and seeking to improve the evaluation of its own work. We have also sought to assess and disseminate best practice in the rule of law sector, the most recent example of that being our report earlier this year on women's access to justice, which looked at eight projects from around the world.
Measuring is key to advancing the rule of law. There are over one hundred systems of measurement, including some highly sophisticated ones, sponsored by multilateral actors such as the World Bank or non-governmental initiatives such as The World Justice Project.
But if we look more closely at them, we will find that there are enormous challenges.
There are technical difficulties. Data is patchy, difficult to collect and even more difficult to compare. Legal cultures vary enormously across countries. All these difficulties are multiplied a thousand-fold in countries torn by conflict and insecurity, or struggling to establish democratic and legal institutions.
There are also political difficulties. Measuring justice means opening the governance system to external scrutiny, it means involving civil society, it means taking a close look at issues of accountability and power: not easy for any society but particularly difficult in post-conflict states with fragile institutions.
There is also the issue of time. Justice is about institution-building - and that takes time.
Last year, as part of IDLO's Strategic Planning process we engaged in a consultative process with around 600 people, including many legal and judicial experts and practitioners and civil society actors in over 40 countries across five continents. What we heard, I think, is relevant to this discussion on measuring justice. Five issues kept coming up:
"End user" focus
- Reform initiatives need to focus on building people’s trust in the rule of law. It means measuring the success of reforms according to the extent to which they respond to people's needs and demands for justice. That is particularly important in post-conflict societies where there is often a trust deficit between state and citizen.
Local ownership, international standards
- The best results come when the reforms are tailored to meet local needs based on locally defined parameters but are at the same time anchored within the framework of international standards and values.
Engagement with informal as well as formal legal systems
- Reform initiatives need to focus not only on the formal system but also on informal systems, and on the relationship between the two to ensure that the state does not abdicate its responsibilities to women, the poor and the marginalized who are the most common users of informal systems of justice.
Empowering solutions
- Legal reforms alone do not produce justice. Combating discrimination and fighting injustice requires tackling the underlying power dynamics that deprive people from equal protection of the law. Legal and non-legal empowerment strategies are critical in helping people access justice and claim their rights.
That's what we heard, but what do we do? (and in using "we" I refer to the major providers of rule of law assistance, including IDLO and the UN as well as others)?
We usually measure:
- Performance: do institutions provide efficient services?
- Transparency and accountability: are institutions operating transparently and with integrity, and are they accountable to rules and standards of conduct?
- Capacity: do institutions have the human and material resources necessary to perform their functions, and the administrative and management capacity, to deploy these resources effectively?
- We usually fail to measure impact: whether justice institutions are accessible and responsive to the needs of the people, especially those who are disenfranchised and marginalized such as women, indigenous peoples, minorities, victims, and children.
Take judicial reform. Most judicial systems track the volume of cases passing through the system, the speed of decision making and the kinds of decisions courts make.
Most justice systems do not collect information about who uses the justice system and the impact of the court’s decisions on them. If courts are to be accessible and trusted, as well as efficient, if courts are to become the venue for justice for all people - the poor, disenfranchised, discriminated against, underprivileged, and marginalized, and not just for the elite - then the indicators of justice must measure issues other than speed, volume and quality of judicial decision-making.
WHAT SHOULD WE MEASURE?
Justice
We need to measure justice. In peace-keeping and post-conflict situations, "rule of law" often means security first - i.e. police and prisons - and courts and the law later. The justice sector is the Cinderella of rule of law funding in post- conflict countries.
But defining "justice" is not easy. While most governments agree on how to measure maternal mortality or hunger, we know from the UNGA debate last year how difficult it can be to get everyone to agree on what justice means. How can you measure something that you are not prepared to define?
The concept of justice is neither un-measurable nor intangible, but it is ideological, and politically sensitive.
In 2004, the United Nations Secretary General provided a detailed definition in an effort to promote uniformity in usage and understanding. It emphasizes equity, accountability, and legal certainty and supremacy of the law, but also the fundamental principles of human rights. The same elements permeate the 2012 UN Declaration, and could be a useful framework to measure and evaluate the rule of law programs.
Furthermore, it is a challenge to define and measure the rule of law in a way that reflects the universal standards of human rights but also is meaningful locally. This is where the post-2015 development debate on integrating a rights-based approach to development could be helpful, especially in ensuring that justice is not translated only into criminal justice but also as access to economic, social and cultural rights.
National and international law are inter-connected. Local ownership is not about disregarding international human rights standards but about making them "real" and meaningful to people in concrete ways. That can be challenging but there are many interesting examples of how that is being done in many developing countries.
The interconnectedness of the international and national rule of law is perhaps nowhere more clear than in the context of the International Criminal Court (ICC) and the Statute of Rome and the project of international criminal justice will only succeed if national justice institutions have the capacity and will to prosecute and adjudicate core crimes. Notwithstanding the challenges of transitional justice, measurement of justice systems in post-conflict societies cannot exclude this element.
Access to justice
We all know that “the poor need the law - the rich can take care of themselves." The experiences of women, people living in poverty and other marginalized groups is often a test of the effectiveness of the justice system. In Afghanistan, we run a program to develop prosecution units in the Attorney General's offices for gender-based violence, and we find that where the program works for women, it works for everyone else too. Indicators of justice should be disaggregated with a clear pro-gender, pro-poor focus.
Traditional indicators of access to justice focus often on legal and physical access to the courts, for instance on the independence of the judiciary and legal aid. But in practice our work on the ground shows that access can depend on a range of other issues, such as information about laws and the process, arrangements for witness protection, or, in the case of gender-based violence, shelter and employment opportunities for women victims of violence.
Research has shown that the absence of bias and corruption are powerful signs that access to justice is working. Corruption can be a matter of ethics but also a matter of economics - such as low salaries of judicial officials - or poorly structured case-management systems.
Bias in the justice system can be reduced through greater diversity in the justice sector, in other words who participates in dispensing justice. We know from our work on women's access to justice that women are more likely to feel confident about seeking justice when there are more women on the other side - as judges, investigators, prosecutors, defense attorneys or judicial officials. That is why at the UNGA last year IDLO pledged to work to increase the participation of women in the justice sector, and we are looking for partners who are willing to join us in this endeavor.
Informal justice
In many parts of the world, the poorest members of society, minority groups, women, and those who live in geographically remote areas often rely on religious, traditional, or other informal justice mechanisms to resolve legal problems. The use of informal systems may be a sign that the formal system is not working, or it may be the preferred method for resolving disputes because it is familiar, cheap and accessible.
There is growing acknowledgment by governments and donors of the important role that these systems play in people's lives. However, measurement initiatives have yet to catch up. If measuring formal justice systems is difficult, doing so with informal systems is even more so. Non-state institutions take many forms; record keeping is often poor or non-existent; and those who turn to non-state systems for justice may be unwilling to share their experiences.
Making informal systems more effective may mean making them more transparent, ensuring respect for the human rights of those who use them, and enhancing cooperation between non-state and state systems, so that the state does not abdicate its responsibility to ensure justice for all.
What other issues matter?
- Coordination. Rule of law assistance is increasingly a crowded field. The United Nations provides rule of law assistance in 150 countries, and in many of these countries, through multiple UN agencies. As the number of rule of law initiatives and actors continue to expand, coordination should be strengthened both within the UN system and with other international actors.
- Civil society. Involving the government is essential, but it is not possible to get a true picture of how effective the justice system is, without involving civil society actors, NGOs, community groups and the legal professional organizations.
- Culture of justice. It is crucial to look holistically at justice. It is important, not simply to check if one aspect is working but whether the overall system is producing equitable outcomes - otherwise the results may well be counter-productive. For instance, the pressure to ensure security and prosecute more crimes could end up violating protection of fundamental rights.
The challenge at the strategic level is to determine whether the rule of law is leading to a culture of justice.