A Gentle Introduction into Extracting Data

Curriculum

2 Sections
8 Lessons
Lifetime

Expand all sectionsCollapse all sections

Extracting Data from PDFs
Far too much data is trapped in PDFs. In order to be able to work with, analyse and visualise data, we need it in machine-readable formats. It's often not easy to the data out again, but sometimes possible - find out how here.
3
Making data on the web useful: scraping
Learn how to scrape without code in 5 minutes and when you might need to invest time in more sophisticated techniques.
5

Code-free Scraping in 5 minutes using Google Spreadsheets & Google Chrome

Knowing the structure of a website is the first step towards extracting and using the data. Let’s get our data into a spreadsheet – so we can use it further. An easy way to do this is provided by a special formula in Google Spreadsheets.

Save yourselves hours of time in copy-paste agony with the ImportHTML command in Google Spreadsheets. It really is magic!

Recipes

In order to complete the next challenge, take a look in the Handbook at one of the following recipes:

Extracting data from HTML tables.
Scraping using the Scraper Extension for Chrome

Both methods are useful for:

Extracting individual lists or tables from single webpages

The latter can do slightly more complex tasks, such as extracting nested information. Take a look at the recipe for more details.

Neither will work for:

Extracting data spread across multiple webpages

Challenge

Task: Find a website with a table and scrape the information from it. Share your result on datahub.io (make sure to tag your dataset with schoolofdata.org)

Tip

Once you’ve got your table into the spreadsheet, you may want to move it around, or put it in another sheet. Right click the top left cell and select “paste special” – “paste values only”.