Finding Data

October 18, 2013 in Data for CSOs

Evidence is power, as the School of Data’s motto says: the power to transform your material and social circumstances by understanding them. But so long as you don’t know how to gain access to data, you are dispossessed of this power. This guide aims to get you started finding data and taking power back.

Data explorers with a regional Balkan focus must mostly resort to “wobbing”: taking advantage of Freedom of Information (FOI) legislation to request data from public authorities. Placing FOI requests requires knowledge of your rights as well as patience. Investigations on a more international scale can make use of freely available data provided by governments and non-governmental organizations. Getting this data is mostly a matter of knowing how to find it.

Freedom of Information

The United Nations General Assembly has asserted that the freedom of information is a fundamental human right. Over the past fifteen years, all of the Balkan states have enshrined this right in law. Citizens of the Balkans have the right to petition state bodies for information of a public character. According to Freedom House’s studies of press freedom as well as the RTI Rating, this right is widely respected. Indeed, Serbia and Slovenia are considered exemplary for their respect of the right to information.

Filing FOI requests can be slow and may encounter resistance. This is no reason to get discouraged. Data that is only available through an FOI request is typically the most valuable data of all, and journalism in the Balkans furnishes many examples of the worth of such data. The Data Journalism Handbook’s case studies of FOI-based journalism include the story of an investigation of the Yugoslav arms trade by a team of Slovenian, Croatian, and Bosnian journalists. Bulgarian journalists like Genka Shikerova and Hristo Hristov have used FOI requests to expose corruption and assassination. Patience and determination will bring rewards.

Your legal right to information is the key to a successful FOI request. Before placing your request, learn your rights. Investigate your country’s FOI legislation and learn the names of the regulations, constitutional clauses, and so on that guarantee your access to data. Make sure to find out any fees associated with information requests and to learn what time limits the FOI legislation sets for replying to your request. When placing a request, it is helpful to mention the legal basis for the request and for your expectations of a reply. Focused and unambiguous requests are the most likely to bear fruit.

Investigations in Kosovo can make use of Informata Zyrtare, a web service built on the Alaveteli platform making it easier to submit FOI requests to Kosovar authorities and to republish the results. Even if you are not using Informata Zyrtare, consider republishing responses to FOI requests as open data as a service to your community. For quantitative data, your FOI request should ask for the data in a machine-readable digital form, making this republication easier.

Portals and searches

Random search

Much useful data has already been made available on the web in centralized repositories hosted by governments and major NGOs. Open Data Albania is the only such repository in the Balkans. If your investigation’s scope is more international, however, and if you have a clear idea of what you want, data portals make a good starting point.

If you are looking for data from a particular state, see if the state has its own data portal. The Open Data Census and are two indexes which can help you locate a particular portal. The Guardian also provides a world government data search engine which allows you to search numerous state data portals simultaneously.

Comparative data on a large number of states is available from the data portals of the United Nations, World Bank, and World Health Organization. Regional portals like Africa Open Data, OpenData Latinoamérica, and the European Union open data portal aggregate data from groups of states.

A great deal of government data is indexed by ordinary web search engines. The trick to finding this data is anticipating its file format: if you limit your searches to machine-readable file formats specific to the type of data you want (e.g. CSV or XLS for tabular data, SQL or DB for databases), your search results are likely to be relevant data. Append “+filetype:extension” to your Google query to look for files with a specific extension, e.g. “+filetype:csv” to look for CSV files.

You may not be the first person to think of collecting the data you’re interested in. Check the Open Knowledge Foundation’s Data Hub, “a community-run catalogue of useful sets of data on the Internet”, to see if anyone else has put up the data you’re looking for.

Flattr this!