Ausführliche Informationen über die Verwendung von Cookies auf dieser Website finden Sie in unserer Datenschutzerklärung. Sie können Ihre Cookie-Einstellungen unten anpassen.
Kontakt
Flawed systems can't create accurate data
Biases in open-source human rights investigations
03. Februar 2023
Reflections | Alex-motoc-qxHiV5nIOjc | UnsplashOpen-source investigation are often said to revolutionise human rights work. Evidence found online, however, often isn’t as objective as it seems to be. Rather, it reflects, and in some cases even exacerbates, existing inequalities and power asymmetries. Ultimately, data is only as good as the system that created it.
Since March 2022, open-source investigators of the Ukrainian Archive have collected more than 593,000 videos documenting war crimes and human rights violations of all parties, just like others have done before in Syria, Sudan, Yemen and many other countries worldwide.
Open-source investigations are often said to revolutionise human rights investigations. And, indeed, they help to address some of the great challenges in traditional human rights work: As everyone with access to the internet and a smartphone can document abuses and share their experience, open-source investigations have the potential to increase the share of documented human rights violations. Moreover, the dependence on witness testimony, that is biased by perception, is reduced. Graphical footage, in contrast, seems to increase objectivity as it shows what happened in a neutral way.
The seemingly objective
The data that seem to be objective, however, are susceptible to bias and can create a distorted picture of reality. Activists, analysts, and scholars alike are aware of several sources of bias, technical and cognitive ones. Access bias, a technical bias, addresses the issue of who has access to the tools needed for documenting events of abuse. The Global Connectivity Report 2023 draws attention to several digital divides: Between men and women, the young and old, cities and rural areas; showing that only specific, privileged parts of society can contribute to fact-finding. Algorithms used to find and analyse information, intensify this tendency, as they often built on biased training data. Face recognition algorithms, for instance, are mainly trained with images of white individuals, leading to a weaker performance when it comes to identifying the faces of people of colour. Thus, not even technical biases are, at their core, technical. Rather, they reflect the inequalities of the social context technology was developed in.
Cognitive biases directly refer to the evaluation of information by humans. This evaluation starts with the selection process. While access to technologies influences which information can be found online, the researcher’s training influences which piece of information the researcher can find and select for investigation. Language barriers are a tangible example: When searching for user-generated content on social media, the search term used must match the term used by the person that tagged the content. In the case information is tagged in English, this might not pose a problem, in the case of many local dialects, however, researchers might be ill-equipped. Additionally, information on stigmatised crimes or crimes happening in the private sphere, is much harder to find. Evidence on sexual or gender-based violence, for example, is less openly shared on social media or only tagged with coded language and are therefore at risk of not being found at all.
Reflections of power
Open-source information used for human rights investigations reflect structural inequalities and power asymmetries. First, as depicted above, they display inequalities between societal groups. Second, open-source investigations are better suited for the documentation for crimes committed in public than in private. These two factors intersect: The most marginalised are often most vulnerable to crimes that are least visible. Relying (primarily) on open-source information is therefore likely to exacerbate already existing inequalities.
Power is reflected also on a global scale. Open-source investigations are time and resource-consuming. Only few organisations have the financial means for the needed technologies and the coaching necessary for investigators to use them. Access to technologies and methods mostly stays where they were designed: in the Global North.
Representation, thus, only reaches limited spheres: While information is increasingly generated in a bottom-up process, decisions are continuously made top-down. The decision-making power about the development of technologies, the information selected, the way it is finally interpreted and used still lies in the hands of organisations and institutions primarily located and funded by the Global North instead of the people most severely affected. Challenges and limitations like these have been pointed out in the human rights sector as well as by those calling for decolonising the development sector.
Do not let the perfect be the enemy of improvement
Downplaying the limitations of open-source investigations is dangerous as the biases reflect and potentially accelerate inequalities and power asymmetries. Ignoring the benefits, however, would be equally foolish. While open-source investigations are not a panacea that brings about a full representation and objectivity, it allows more voices to be heard compared to traditional human rights investigations and is an important addition to the toolbox of human rights investigators.
Furthermore, practitioners and scholars develop mechanisms to mitigate biases: The Berkeley Protocol sets international standards for open-source investigations in the context of legal prosecution and raises awareness about biases, the American Association for Advancement of Science developed a decision-making tool for the collection and sharing of data, Amnesty International, among others, provides training to (young) investigators and the Ukrainian Archive helps to develop investigation capacities, for instance, in Ukraine. When investigators are trained appropriately, are aware of biases and the power structures that create them, they can mitigate their own and institutional blind spots. Researchers need to be trained to know what to look for, where to look for it online and when to fall back on other means for investigations, on- or offline.
Ultimately, data is only as good as the system that created it. And our systems are flawed by power asymmetries. Experts are aware of these flaws and mitigate them the best possible. Beyond mitigation, the biases and limitations must be communicated in a transparent way. Not only by investigators, but by every actor using and publishing open-source data, including the media. While transparency is an important first step, it is not enough. The discussion needs to start at a reflection of the root cause for these biases: Our privilege and power – are we willing to give it up?