A Weekend of Data, Hacks and Maps in Nigeria

olubabayemi - September 16, 2014 in charity data, Data Cleaning, Data Expeditions, event, Mapping, maps, School_Of_Data, Spreadsheets, Visualisation

It was another weekend of hacking for good all around the world, and Abuja, Nigeria was not left out of the weekend of good, as 30 participants gathered at the Indigo Trust funded space of Connected Development [CODE] on 12 – 14 September, scraping datasets, brainstorming creating technology for good, and not leaving one thing out – talking soccer (because it was a weekend, and Nigeria “techies” love soccer especially the English premiership).

Participants at the Hack4Good 2014 in Nigeria

Participants at the Hack4Good 2014 in Nigeria

Leading the team, was Dimgba Kalu (Software Architect with Integrated Business Network and founder TechNigeria), who kick started the 3 day event that was built around 12 coders with other 18 participants that worked on the Climate Change adaptation stream of this year #Hack4Good. So what data did we explore and what was hacked over the weekend in Nigeria? Three streams were worked :

  1. Creating a satellite imagery tagging/tasking system that can help the National Space Research Development Agency deploy micromappers to tag satellite imageries from the NigeriaSat1 and NigeriaSat2
  2. Creating an i-reporting system that allows citizen reporting during disasters to Nigeria Emergency Management Agency
  3. Creating an application that allows citizens know the next water point and its quality within their community and using the newly released dataset from the Nigeria Millennium Development Goal Information System on water points in the country.

Looking at the three systems that was proposed to be developed by the 12 coders, one thing stands out, that in Nigeria application developers still find it difficult to produce apps that can engage citizens – a particular reason being that Nigerians communicate easily through the radio, followed by SMS as it was confirmed while I did a survey during the data exploration session.

Coders Hackspace

Coders Hackspace

Going forward, all participants agreed that incorporating the above medium (Radio and SMS) and making games out of these application could arouse the interest of users in Nigeria.  “It doesn’t mean that Nigerian users are not interested in mobile apps, what we as developers need is to make our apps more interesting” confirmed Jeremiah Ageni, a participant.

The three days event started with the cleaning of the water points data, while going through the data pipelines, allowing the participants to understand how these pipelines relates to mapping and hacking. While the 12 hackers were drawn into groups, the second day saw thorough hacking – into datasets and maps! Some hours into the second day, it became clear that the first task wouldn’t be achievable; so much energy should be channelled towards the second and third task.

SchoolofData Fellow - Oludotun Babayemi taking on the Data Exploration session

SchoolofData Fellow – Oludotun Babayemi taking on the Data Exploration session

Hacking could be fun at times, when some other side attractions and talks come up – Manchester United winning big (there was a coder, that was checking every minutes and announcing scores)  , old laptops breaking (seems coders in Abuja have old  ones), coffee and tea running out (seems we ran out of coffee, like it was a sprint), failing operating systems (interestingly, no coders in the house had a Mac operating system), fear of power outage (all thanks to the power authority – we had 70 hours of uninterrupted power supply) , and no encouragement from the opposite sex (there was only two ladies that strolled into the hack space).

Bring on the energy to the hackspace

Bring on the energy to the hackspace

As the weekend drew to a close, coders were finalizing and preparing to show their great works.  A demo and prototype of streams 2 and 3 were produced. The first team (working on stream 2), that won the hackathon developed EMERGY, an application that allows citizens to send geo-referenced reports disasters such as floods, oil spills, deforestation to the National Emergency Management Agency of Nigeria, and also create a situation awareness on disaster tagged/prone communities, while the second team, working on stream 3, developed KNOW YOUR WATER POINT an application that gives a geo-referenced position of water points in the country. It allows communities; emergency managers and international aid organizations know the next community where there is a water source, the type, and the condition of the water source.

(The winning team of the Hack4Good Nigeria) From Left -Ben; Manga; SchoolofData Fellow -Oludotun Babayemi; Habib; Chief Executive, CODE - Hamzat

(The winning team of the Hack4Good Nigeria) From Left -Ben; Manga; SchoolofData Fellow -Oludotun Babayemi; Habib; Chief Executive, CODE – Hamzat

Living with coders all through the weekend, was mind blowing, and these results and outputs would not be scaled without its challenges. “Bringing our EMERGY application live as an application that cuts across several platforms such as java that allows it to work on feature phones can be time consuming and needs financial and ideology support” said Manga, leader of the first team. Perhaps, if you want to code, do endeavour to code for good!


flattr this!

Video: School of Data Summer Camp

Heather Leson - September 15, 2014 in Community, Events, Impact Case Study, School_Of_Data

We’re proud to share a video all about School of Data. In 5 minutes, you will learn about School of Data with introductions to School of Data Network members, Fellows and staff. You will get a window into the spirit of School of Data Summer Camp 2014.

About School of Data Summer Camp

The School of Data Summer Camp brought together local initiatives and School of Data Fellows to build the foundation for a buzzing and productive coming year. Last year we kicked off a Spanish, French, Portuguese and Greek site as well as local initiatives around the world. On a more individual level we’ve been working with fellows worldwide to bring data skills to their communities. With a new round of fellows coming we want to get together and unify our vision, goals and methods. Our goal was to make everyone go home with better skills and a clear vision where we’ll take the School of Data together!

Special thanks goes to our funders who made all of this possible: Partnership for Open Data (World Bank, Open Data Institute, and Open Knowledge), Hewlett Foundation, Open Society Foundations, IndigoTrust, the British Embassy Skopje, Hivos, and SEATTI.

(Video Created by Sam Muirhead of Camera Libre)

flattr this!

Easy Access to World Bank and UN Development Data from IPython Notebooks

Tony Hirst - September 12, 2014 in Open Development Toolkit, Scraping

Although more and more data is being published in an open format, getting hold of it in a form that you can quickly start to work with can often be problematic. In this post, I’ll describe one way in which we can start to make it easier to work with data sets from remote data sources such as the World Bank, the UN datastore and the UN Population division from an IPython Notebook data analysis environment.

For an example of how to run an IPython Notebook in a Chrome browser as a browser extension, see Working With Data in the Browser Using python – coLaboratory. Unfortunately, of the wrappers described in this post, only the tools for accessing World Bank Indicators will work – the others currently require libraries to be installed that are not available within the coLaboratory extension.

The pandas Python library is a programming library that provides powerful support for working with tabular datasets. Data is loaded into a dataframe, the rows and columns of which can be manipulated in much the same way as the rows or columns of a spreadsheet in a spreadsheet application. For example, we can easily find the sum of values in a column of numbers, or the mean value; or we can add values from two or more columns together. We can also run grouping operations, a bit like pivot tables, summing values from all rows associated with a particular category as described by a particular value in a category column.

Dataframes can also be “reshaped” so we can get the data into a form that looks like the form we want to be.

But how do we get the data into this environment? One way is to load in the data from a CSV file or Excel spreadsheet file, either one that has been downloaded to our desktop, or one that lives on the web and can be identified by a URL. Another approach is to access the data directly from a remote API – that is, a machine readable interface that allows the data to be grabbed directly from a data source as a data feed – such as the World Bank indicator data API.

On most occasions, some work is needed to transform the data received from the remote API into a form that we can actually work with it, such as a pandas dataframe. However, programming libraries may also be provided that handle this step for you – so all you need to do is load in the programming library and then simply call the data in to a dataframe.

The pandas library offers native support for pulling data from several APIs, including the World Bank Development Indicators API. You can see an example of it in action in this example IPython notebook: World Bank Indicators API – IPython Notebook/pandas demo.


Whilst the World Bank publishes a wide range of datasets, there are plenty of other datasets around that deal with other sorts of development related data. So it would be handy if we could access data from those sources just as easily as we can the World Bank Development Indicators data.

In some cases, the data publishers may offer an API, in which case we can write a library a bit like the pandas remote data access library for the World Bank API. Such a library would “wrap” the API and allow us to make calls directly to it from an IPython notebook, getting the data back in the form of a pandas dataframe we can work with directly.

Many websites, however, do not publish an API – or occasionally, if they do, the API may be difficult to work with. On the other hand, the sites may publish a web interface that allows us to find a dataset, select particular items, and then download the corresponding data file so that we can work with it.

This can be quite a laborious process though – rather than just pulling a dataset in to a notebook, we have to go to the data publisher’s site, find the data, download it to our desktop and then upload it into a notebook.

One solution to this is to write a wrapper that acts as a screenscraper, which does the work of going to the data publisher’s website, find the data we want, downloading it automatically and then transforming it into a pandas dataframe we can work with.

In other words, we can effectively create our own ad hoc data APIs for data publishers who have published the data via a set of human useable webpages, rather than a machine readable API.

A couple of examples of how to construct such wrappers are linked to below – they show how the ad hoc API can be constructed, as well as demonstrating their use – a use as simple as using the pandas remote data access functions show above.

  • The UN Department of Social and Economic Affairs Population Division on-line database makes available a wide range of data relating to population statistics. Particular indicators and the countries you require the data for are selected from two separate listboxes, and the data is then downloaded as a CSV file. By scraping the contents of the list boxes, we can provide a simple command based interface for selecting a dataset containing data fro the desired indicators and countries, automatically download the data and parse it into a pandas dataframe: UN Population Division Data API. So for example, we can get a list of indicators: nbviewer2_ipython_org_github_psychemedia_dataviz4development_blob_master_dataloaders_UN_Population_Division_Data_API_ipynb We can also get a list of countries (that we can search on) and then pull back the required data for the specified countries and indicators. nbviewer_ipython_org_github_psychemedia_dataviz4development_blob_master_dataloaders_UN_Population_Division_Data_API_ipynb. Note that the web interface limits how many countries and indicators can be specified in any single data download request. We could cope with this in our ad hoc API by making repeated calls to the UN website if we want to get a much wider selection of data, aggregating the results into a a single dataframe before presenting them back to the user.
  • The UNdata website publishes an official XML API, but I couldn’t make much (quick) sense of it when I looked at it, so I made a simple scraper for the actual website that allows me to request data by searching for an indicator, pulling back the list of results, and then downloading the data I want as a CSV file from a URL contained within the search results and parsing it into a pandas dataframe: UNdata Informal API. nbviewer_ipython_org_github_psychemedia_dataviz4development_blob_master_dataloaders_UNdata_Informal_API_ipynb

By using such libraries, we can make it much easier to pull data into the environments within which we actually want to work with the data. We can also imagine creating “linked” data access libraries that can pull datasets from multiple sources and then merge them together – for example, we might pull back data from both the World Bank and the UN datastore into a single dataframe.

If there are any data sources that you think are good candidates for opening up in this sort of way, so that the data can be pulled more easily from them, please let us know via the comments below.

And if you create any of your own notebooks to analyse the data from any of the sources described above, please let us know about those too:-)

flattr this!

Dispatch from India: Intro to Data Journalism Workshop in Bangalore

nisha - September 12, 2014 in Data Journalism, School_Of_Data, Uncategorized

This post is crossposted from DataMeet.org, an organization that promotes open data in India.

Sunday, August 31st, DataMeet worked with an Economic Times Journalist Jayadevan PK with support from School of Data, to design an intro to data journalism workshop. For a while now there has been quite a bit of interest and discussion of data journalism in India. Currently there are a few courses and events around promoting data journalism, we thought there was definitely room to start to build a few modules on working with data for storytelling. Given that we have not done too many of these we decided to do an introduction and leave it limited to a few people.


You can see the agenda with notes here and the resources we shared on the data journalism resource wiki page, as well as refer to the data catalog that DataMeet has been putting together.

Thanks to Knolby Media for hosting us and for School of Data for the support. Thank you to Vikras Mishra for volunteering and taking notes, pictures, and video.

We had four story tellers with us, from various backgrounds. We spent the morning doing introduction and what was their experience with data, what their definition of data journalism is and why they wanted to take this workshop. Then we had them put up some expectations so we can gauge what the afternoon should focus on.


We then had Jaya go through the context of data journalism in terms of the world scale and the new digital journalism era.

Then we spent some time going over examples of good data journalism and bad.

After we went through resources people can use to get data. We touched upon the legal issues around using data and copyright issues. Then we discussed accuracy and how to properly attribute sources.

Then we demonstrated a few tools

Datameet 5

Tableau CartoDB Scraping tools - Scraper wikiIMACROS MapBox QGIS

Visualization Roadmap The participants thought understanding how to visualize would be helpful.  So we went through a sort of visualization roadmap.  Then went through stories they were working on to see how we would create a visualization and also how to examine the data and come up with a data strategy for each story.

Datameet 6 20140831_155126

Then showed some more tools to address the suggestions from the exercise. - BHUVANTimelinesOdysseyFusion TablesBUMP

Feedback session


People wanted another day to let the lessons be absorbed and some more time to actually have hands on time with the tools.  Also even at the intro level it is important to make people come prepared with stories, so they have something to apply the ideas to.

To say we learned a lot is an understatement. We will definitely be planning more intro workshops and hopefully more advanced workshops in the future, we hope to continue to learn what people think is important and will keep track and see what kinds of stories come out of these learning session.


flattr this!

Lessons learned while exploring Copenhagen’s bicycle paths using data – Part 1

Michelle Kovacevic - September 11, 2014 in Data Expeditions

Pink and Copenhagen Blue

When I signed up to the School of Data mailing list, I didn’t quite know what I was getting myself into.

Within two days of joining, I was invited to lead a data expedition at the Kopenlab citizen science festival alongside the EuroScience Open Forum in Copenhagen, Denmark.

My first reaction was trepidation (I didn’t know what a data expedition was and I haven’t worked extensively with datasets for a few years) but Michael Bauer at the School of Data assured me that it would be a fun learning experience. So I enthusiastically agreed and my first quest with data began.

I quickly learned that a data expedition aims to discover stories hidden in the ‘Land of Data’. As the expedition guide, I would set the topic and encourage my expedition team to work together to solve real-life problems, answer questions and tell stories with data.

An important side note (and one I reiterated several times during the expedition) is that there are no right answers and no expected final output. The point of a data expedition is to think freely and creatively, learn from each other and hopefully develop some new skills and a lifelong love of data.

Given Copenhagen’s reputation as the most bike friendly city in the world, we choose to focus on the comprehensive cycling statistics that Denmark collects every day.

For example, did you know that more people in greater Copenhagen commute by bicycle than everyone who rides bikes to work in the entire United States? This information can be found in easily accessible datasets such as the EU public dataset and Denmark’s national statistics database.

We came up with a few guiding to stimulate the imaginations of our expedition team:

How far do I have to walk to get a bikerack in Copenhagen?
Are there areas where bikeracks are more dense and how does this correlate with where people are riding bikes?
How many bike accidents are caused in Copenhagen because cyclists are drunk?
Do more young or old people ride bikes in Copenhagen?
At which age do people spend most money on bicycles?

So armed with some sample datasets, a laptop and flipchart, I set off to Copenhagen to meet Ioana, Deepak, Akash, Mirela and Tobias – my expedition team.

After finding 10 things in common with each other, our first task was to work out everyone’s strengths and weaknesses so we could set loose roles. Ioana became our analyst & engineer (data cruncher), Deepak and Akash were our storytellers (found interesting angles to explore and shaped the final story), Mirela was our scout (data hunter) and Tobias was our designer (beautify the outputs to make sure the story really comes through).

Our next task was to come up with our expedition questions and we took to this task very enthusiastically, coming up with more questions than we had time to explore! To make the questions easier to tackle, we decided to group them by theme (travel usage, life cycle/accidents/rules/compliance, geographical stats, economics, policy, culture). The group split in half to tackle two different sets of questions.

flipchart Deepak, Akash and Tobias looked at what policies influenced cycling adoption in Denmark and compared these to a number of different cities across the world.

Mirela and Ioana mapped the number of cyclists in different areas of Copenhagen in order to develop a density map, which could be overlayed with other data such as where cyclists are most active at certain times of day, accident rates/locales and bikerack density.

We spent the next two hours of the expedition searching and scraping various datasets (a full list can be found in this googledoc) in order to come up with our stories to tell the Kopenlab citizen science festival attendees.

We came across a few hurdles, namely the “cleanness” and consistency of the data. Often the datasets were only available as PDFs (CSV and excel spreadsheets are much easier to work with) and data headers often didn’t have references.

“It would be nice to have it all in a bigger table,” Ioana said.

In the face of these challenges we gave each other a helping hand to find alternative exploration routes (much like a real quest, really).

Another one of the great aspects of a data expedition the focus on skill sharing. Ioana had a great understanding of Google fusion tables so she was able to show some of the other participants how to sort and analyse data using this tool. Unfortunately we didn’t get much time to explore the plethora of open source data analysis and visualization tools (some are listed on page 5 of this doc).

So after three hours traversing the wilds of Copenhagen’s bike data we had two stories to tell.

Ioana presented her team’s heat map showing that the number of cyclists was most dense in the northwest part of Copenhagen.

Deepak presented his team’s infographic showing that many factors influence cycling usage in urban centers:


We had a great time exploring these datasets, but with the short time we had available, we only really scraped the surface of Copenhagen’s bike data stories. Luckily Matias and his bikestorming crew ran another expedition in Copenhagen two months later and were able to build on what we learnt…

Stay tuned for part two of our biking blog series written by Matias Kalwill, founder and designer of Bikestorming.

More pics .

flattr this!

How to: Network Mapping Builds Community

Heather Leson - September 10, 2014 in Community, HowTo, School_Of_Data

Who is in your network? Who are your stakeholders? Network Mapping can help you plan, grow and sustain your organization. Nisha Thompson of Datameet.org and a School of Data Fellow shares her Network Mapping Skills in this 40 minute video. See the accompanying slides and resources below.

Network Mapping Resources:

Here are some resources provided by Nisha and the team to get you started on your Network Mapping journey:

Network Mapping – Nisha Thompson

We’ll be hosting more School of Data Fellow Skillshares in the coming weeks. See our wiki for more details.

Title image: Social network in a course CC-by Hans Põldoja, on Flickr

flattr this!

Tips for teaching/training on data skills

marielgm - August 29, 2014 in Community, HowTo

(photo of Ignasi, Olu and Ketty by Heather Leson, July 2014 (CC-by))

(photo of Ignasi, Olu and Ketty by Heather Leson, July 2014 (CC-by))

You probably have a skill or knowledge that others would love to acquire… but teaching can be intimidating. Fear not! In this post, we will share a few tips from the School of Data network, which is filled with individuals who hold continuous trainings on all things data worldwide.

Prepare! It’s not a great idea to improvise when you are frozen by stage fright, nor to realize in the middle of a workshop that you can’t continue as planned because you are missing materials. That’s why formal planning of each workshop can help. Here’s an example you could use.

Michael from School of Data in Berlin has a special piece of advice for your planning: “Be yourself! Find the teaching method you feel comfortable with (I like to do things ad-hoc, Anders prefers slides, e.g.)”

Also, maybe it’s a good idea to partner up. Cédric from School of Data in France makes a great point: “There are two essential things in a workshop: knowledge of methodology and knowledge of the subject. More often than not, it’s better to separate them between two people. One will make sure that the workshop goes smoothly, and the other will help individuals get past roadblocks”.

Be mindful of how you speak Beyond what you say, the way you speak can have an impact on the success of your workshop. Michael (again) and Heather from School of Data in Toronto recommend that you try to speak a bit slower than you’re used to, with simple sentences, and avoiding jargon or descriptive metaphors.

Make it a friendly environment Helping people feel comfortable and welcome is necessary in every educational setting. Happy from School of Data in the Philippines explains it: “The point is to keep it as trivial as possible so that people don’t feel intimidated by the skill level of others”.

Codrina from School of Data in Romania has a lot of experience here: she recommends not keeping it too serious, and rather make small jokes; also, “give a little pat on the back for those who ask questions”… And don’t forget to take breaks! Yuandra from School of Data in Indonesia reminds us of something crucial: refreshments and water. People won’t learn if they’re distracted by hunger.

Also, icebreakers. We all love icebreakers, and Olu from School of Data in Nigeria has these in mind.

Try to connect with your audience We use this phrase a lot, but what does it mean? Ketty from School of Data in Uganda puts it in very practical terms: try to read the learner’s facial expressions for e.g. confusion/tiredness/intent. This will help you find the best ways to continue.

Also, Ketty adds, “sometimes you have to be flexible and allow the learners to change your program… A bit of a give & take approach”.

On a slightly different topic, but still related to your connection with the audience, Olu thinks your audience will be inspired to work harder in your workshop if you tell them stories of what data/open data can be used for. You can find some at the World Bank Open Data Blog, and here on School of Data.

Some other didactic considerations Heather recommends that you repeat key things 3 times (but not right after each other – spread them throughout the workshop). Also, Codrina recommends repeating questions when they are asked so everyone can hear before the answer is given.

Another recommendation: If you have a really successful workshop, try to replicate it through other media. For example, run it on a hangout, write it out on a tutorial. Multiple content won’t be redundant – it will mean more and more people will have a chance to learn from it.

Happy has a great tip: “When you want to get the group to mingle and pair up (data analysts paired with visualizers, for example) one way to do it is to divide the group, 1 line for data analysts, another for visualizers. Then we ask them to line up according to a range of categories – from technical categories or something as simple as personal information, like the number of house they lived in during their childhood, for example”.

Make an effort to keep track of time and exactly how long you spend on each part, Cédric recommends, as this will help you plan for future trainings.

Communicate Your audience may well be outside the room where you are doing the training. Cédric adds: “Sometimes good suggestions can come from social media platforms like Twitter, so if you have an audience there, you might want to share some updates during the event. People might answer with ideas, technical advice or more”.

Evaluate The workshop was fun and people attended. But did they really learn?

Try to evaluate this learning through different methods. Was everyone able to complete the exercises? What did they respond that they learned in your ‘exit survey’? Did you get good responses to your last round of oral questions?

Olu kindly shared a couple of forms that can be used for this purpose both before and after the training. Feel free to use them!

A few resources shared by the School of Data community Notes from the OKFest How to Teach Data Session (July 2014) Aspiration Tech has great tips in their guides (via Heather) PSFK on how people make/learn (via Heather) Escuela de Datos on our Local LATAM training lessons learned

flattr this!

Hacking the world’s most complex election system: part 2

Codrina Ilie - August 28, 2014 in Data for CSOs, Election data, On the Road

[Ed. Note: The School of Data was recently in Bosnia. This blog post is the 2nd in the series. See the first post by Codrina (School of Data Fellow).]

After all the brainstorming, all the coding, all the mapping, all the laughing and chatting the results of the Bosnian election data hacking are slowly, but surely finding their way to the surface.

School of Data Bosnia

After the crash course of only 3 hours for grasping how the Bosnian election system actually works and after a powerful and general brainstorming of what we have and what we want to get, the happy hackers divided into 3 teams: the data team, the development team and the storytellers each with clear tasks to achieve. After a week’s work, the [git] repository is fully packed. Raw data, cleaned data, code and a wiki explaining how we aimed to make sense of the quite special Bosnian elections system. Still curious why we keep saying this election system is complicated? Check out this great video that the storytellers created:

The end of this wonderful event was marked down by a 2 hours power brainstorming regarding the flourishment of the Data Academy. Great ideas flew around the place and landed on our Balkan Data Academy sheet.

World, stay tuned! Something is definitely cooking in the wonderful Balkans!!

flattr this!

Using Data Journalism to probe economics in the West Bank

Eva Constantaras - August 27, 2014 in Data Expeditions, Data Journalism

Ramallah Cityscape

Weeks before the current conflict erupted between Israel and Hamas, twenty Palestinian journalists came together in Ramallah for three days to use data to untangle the economic reality for Palestinians.

The fourth in a series of workshops aimed at establishing economic beat reporting in the West Bank, the Data Journalism for Economic Reporting workshop immersed journalists in the raw economic data that could provide objective, analytical content on a highly politicized local and global topic and explore viable solutions.

Watch a video from one of the workshops:

For the first time, journalists took a deeper look at the data behind buzzwords such as “economic peace” and “economic packages” that form part of the negotiation process between Israel, the Palestinians and donors. Almost immediately journalists identified cases in which a better understanding of data would have served the needs of their audience.

When the World Bank issued the report Area C and the Future of the Palestinian Economy, most journalists just reported on it using a version of the official press release Palestinians Access to Area C Key to Economic Recovery and Sustainable Growth. None requested the raw data to determine what areas of the economy have the most growth potential, what policy changes would be key in negotiating for market growth and what vocational and other educational programs could be put into place to prepare the workforce for a lifting of current restrictions.

“The language of statistics and figures are stronger and more credible,” explained Abubaker Qaret from PADECO Co, an investment firm.

Participants planned to both request the data from the World Bank study and investigate audit data from the donors who keep the Palestinian Authority afloat.

Over the course of three days, journalists practiced the skills to produce the first data-driven economic reporting in the West Bank. Trainees learned to scrape data (extract data from human-readable output) using scraper extensions, identify story angles in monthly economic data releases, answer basic questions about economic growth and government spending using Excel and visualize their findings using Google Charts.

Akram Natcha, a journalist from Al-Quds TV has a financial background but had not thought to apply some of the technical skills to his journalistic work. “This is the first time I used Excel data analysis with the aim of publishing.”

During a Google Charts visualization exercise, trainees used data scraped from PDFs downloaded from the Palestinian Ministry of Finance website to calculate and visualize which sectors of the economy experienced the largest growth during 2013.

Abubaker Qurt visualized his findings:

Trainees also compared unemployment rates and demographics to other countries in the region, calculated growth and absorption rates of the Palestinian Territories’ current workforce and calculated the per capita international aid received compared with its neighbors. They then practiced translating this information into narrative storytelling that would put a human face on pressing economic issues.

The Data Expedition that concluded the workshop focused on evaluating the Palestinian Authority’s fiscal management by examining the last three years of government expenditure data. In groups, trainees proposed and honed in on three specific questions:

  • Which government departments spend the largest portion of their budget on wages and least on implementing projects and which department is responsible for spending the most overall on staff costs?
  • How did spending on neglected areas such as cultural heritage and scientific research compare to how much was allocated by regional neighbors for those activities?
  • How do trends in investment in education correlate to results on standardized tests and growth in related economic areas?

Following the workshop, several participants pursued and published investigations into the economic impact of the heightened presence of the Israeli army in the West Bank.

“I benefited from the workshop to identify story angles through the tables,” said Rabee Dweikat a press officer at the Bank of Palestine. “I discovered new information from the data.”

The training series is funded by the US Consulate in Jerusalem.

This post is cross posted from the Internews Blog

flattr this!

Seven Ways to Create a Storymap

Tony Hirst - August 25, 2014 in Data Journalism, Data Stories, HowTo, Storytelling

If you have a story that unfolds across several places over a period of time, storymaps can provide an engaging interactive medium with which to tell the story. This post reviews some examples of how interactive map legends can be used to annotate a story, and then rounds up seven tools that provide a great way to get started creating your own storymaps.

Interactive Map Legends

The New York Times interactives team regularly come up with beautiful ways to support digital storytelling. The following three examples all mahe use of floating interactive map legends to show you the current location a story relates to as they relate a journey based story.

Riding the Silk Road, from July 2013, is a pictorial review featuring photographs captured from a railway journey that follows the route of the Silk Road. The route is traced out in the map on the left hand side as you scroll down through the photos. Each image is linked to a placemark on the route to show where it was taken.


The Russia Left Behind tells the story of a 12 hour drive from St. Petersburg to Moscow. Primarily a textual narrative, with rich photography and video clips to illustrate the story, an animated map legend traces out the route as you read through the story of the journey. Once again, the animated journey line gives you a sense of moving through the landscape as you scroll through the story.


A Rogue State Along Two Rivers, from July 2014, describes the progress made by Isis forces along the Tigris and Euphrates Rivers is shown using two maps. Each plots the course of one of the rivers and uses place linked words and photos to tell the story of the Isis manoeuvres along each of the river ways. An interactive map legend shows where exactly along the river the current map view relates to, providing a wider geographical context to the local view shown by the more detailed map.


All three of these approaches help give the reader a sense of motion though the journey traversed that leads the narrator being in the places described at different geographical storypoints described or alluded to in the written text. The unfolding of the map helps give the reader the sense that a journey must be taken to get from one location to another and the map view – and the map scale – help the reader get a sense of this journey both in terms of the physical, geographical distance it relates to and also, by implication, the time that must have been expended on making the journey.

A Cartographic Narrative

Slave Revolt in Jamaica, 1760-1761, a cartographic narrative, a collaboration between Axis Maps and Harvard University’s Vincent Brown, describes itself as an animated thematic map that narrates the spatial history of the greatest slave insurrection in the eighteenth century British Empire. When played automatically, a sequence of timeline associated maps are played through, each one separately animated to illustrate the supporting text for that particular map view. The source code is available here.


This form of narrative is in many ways akin to a free running, or user-stepped, animated presentation. As a visual form, it also resembles the pre-produced linear cut scenes that are used to set the scene or drive the narrative in an interactive computer game.

Creating you own storymaps

The New York Times storymaps use animated map legends to give the reader the sense of going on a journey by tracing out the route being taken as the story unfolds. The third example, A Rogue State Along Two Rivers also makes use of a satellite map as the background to the story, which at it’s heart is nothing more than a set of image markers placed on to an an interactive map that has been oriented and constrained so that you can only scroll down. Even though the maps scrolls down the page, the inset legend shows the route being taken may not be a North-South one at all.

The linear, downward scroll mechanic helps the reader feel as if they are reading down through a story – control is very definitely in the hands of the author. This is perhaps one of the defining features of the story map idea – the author is in control of unraveling the story in a linear way, although the location of the story may change. The use of the map helps orientate the reader as to where the scenes being described in the current part of the story relate to, particularly any imagery.

Recently, several tools and Javascript code libraries have been made available from a variety of sources that make it easy to create your own story maps within which you can tell a geographically evolving story using linked images, or text, or both.

Knight Lab StoryMap JS

The Knight Lab StoryMap JS tool provides a simple editor synched to a Google Drive editor that allows you to create a storymap as a sequence of presentation slides that each describe a map location, some header text, some explanatory text and an optional media asset such as an image or embedded video. Clicking between slides animates the map from one location to the next, showing a line between consecutive points to make explicit the linkstep between them. The story is described using a custom JSON data format saved to the linked Google Drive account.


[StoryMapJS code on Github]

CartoDB Odyssey.js

Odyssey.js provides a templated editing environment that supports the creation of three types of storymap: a slide based view, where each slide displays a location, explanatory text (written using markdown) and optional media assets; a scroll based view, where the user scrolls down through a stroy and different sections of the story trigger the display of a particular location in a map view fixed at the top of the screen; and a torque view which supports the display and playback of animated data views over a fixed map view.


A simple editor – the Odyssey sandbox – allows you to script the storymap using a combination of markdown and map commands. Storymaps can be published by saving them to a central githib repository, or downloaded as an HTML file that defines the storymap, bundled within a zip file that contains any other necessary CSS and Javascript files.

[Odyssey.js code on Github]

Open Knowledge TimeMapper

TimeMapper is an Open Knowledge Labs project that allows you to describe location points, dates, and descriptive text in a Google spreadsheet and then render the data using linked map and timeline widgets.


[Timemapper code on Github]

JourneyMap (featuring waypoints.js

JourneyMap is a simple demonstration by Keir Clarke that shows how to use the waypoints.js Javascript library to produce a simple web page containing a scrollable text area that can be used to trigger the display of markers (that is, waypoints) on a map.


[waypoints.js on Githhub; JourneyMap src]

Google Earth TourBuilder

Google Earth TourBuilder is a tool for building interactive 3D Google Earth Tours using a Google Earth browser plugin. Tours are saved (as KML files?) to a Google Drive account.


[Note: Google Earth browser plugin required.]

ESRI/ArcGIS Story Maps

ESRI/ArcGIS Story Maps are created using an online ArcGIS account and come in three type with a range of flavours for each type: “sequential, place-based narratives” (map tours), that provide either an image carousel (map tour) that allows you to step through a sequence of images that are displayed separately alongside a map showing a corresponding location or a scrollable text (map journal) with linked location markers (the display of half page images rather than maps can also be triggered from the text); curated points-of-interest lists that provide a palette of images, each member of which can be associated with a map marker and detailed information viewed via a pop-up (shortlist), a numerically sequence list that displays map location and large associated images (countdown list), and a playlist that lets you select items from a list and display pop up infoboxes associated with map markers; or map comparisons provided either as simple tabbed views that allow you to describe separate maps, each with its own sidebar description, across a series of tabs, with separate map views and descriptions contained within an accordion view, and swipe maps that allow you to put one map on top of another and then move a sliding window bar across them to show either the top layer or the lower layer. A variant of the swipe map – the spyglass view alternatively displays one layer but lets you use a movable “spyglass” to look at corresponding areas of the other layer.


[Code on github: map-tour (carousel) and map journal; shortlist (image palette), countdown (numbered list), playlist; tabbed views, accordion map and swipe maps]

Leaflet.js Playback

Leaflet.js Playback is a leaflet.js plugin that allows you to play back a time stamped geojson file, such as a GPS log file.


[Code on Github]


The above examples describe a wide range of geographical and geotemporal storytelling models, often based around quite simple data files containing information about individual events. Many of the tools make a strong use of image files as pat of the display.

it may be interesting to complete a more detailed review that describes the exact data models used by each of the techniques, with a view to identifying a generic data model that could be used by each of the different models, or transformed into the distinct data representations supported by each of the separate tools.

UPDATE 29/8/14: via the Google Maps Mania blog some examples of storymaps made with MapboxGL, embedded within longer form texts: detailed Satellite views, and from the Guardian: The impact of migrants on Falfurrias [scroll down]. Keir Clarke also put together this demo: London Olympic Park.

UPDATE 31/8/14: via @ThomasG77, Open Streetmap’s uMap tool (about uMap) for creating map layers, which includes a slideshow mode that can be used to create simple storymaps. uMap also provides a way of adding a layer to map from a KML or geojson file hosted elsewhere on the web (example).

flattr this!