Thursday, May 4, 2017

Doing more with your Google Location History

With the discontinuation of the Google Latitude API several years ago some nice third party tools to visualize your location history disappeared, which at that time triggered me to look into what can be done with Google Takeout data. The result was the location_history_json_converter (or latitude_json_converter as it was called back then) which thanks to several contributions from the open source community has turned out to be a rather useful and powerful tool to prepare your takeout data for further manipulation and visualization.
Since I've recently used the tool myself again for a travel report (more about this below) and I never actively promoted or explained the tool, I've decided to put together this blog post to tell you about the tool and some samples of what can be done with it.

Getting your data

Your first stop is at Google Takeout to download your Location History via this URL:


Once the archive is done, you can download and unzip it to get a LocationHistory.json with all your data.

Converting your data

For converting the date you will need my Python script from here:


For the most basic usage of the script you would provide an input file (i.e the LocationHistory.json file from Google Takeout) the name of the output file to create and the format you want the file to be converted to:
location_history_json_converter.py LocationHistory.json -o output.ext -f format
There are several formats to choose from:
kml
KML file with placemarks for each location in your Location History. Each placemark will have a location, a timestamp, and accuracy/speed/altitude as available. Data produced is valid KML 2.2.

csv
Comma-separated text file with a timestamp field and a location field.

json
Smaller JSON file with only the timestamp and the location.

js
JavaScript file which sets a variable in global namespace (default: window.locationJsonData) to the full data object for easy access in local scripts. Just include the js file before your actual script. Only timestamp and location are included.

gpx
GPS Exchange Format including location, timestamp, and accuracy/speed/altitude as available. Data produced is valid GPX 1.1. Points are stored as individual, unrelated waypoints (like the other formats, except for gpxtracks).

gpxtracks
GPS Exchange Format including location, timestamp, and accuracy/speed/altitude as available. Data produced is valid GPX 1.1. Points are grouped together into tracks by time and location (specifically, two chronological points split a track if they differ by over 10 minutes or approximately 40 kilometers).

Below I'll give you some examples of how you could use these formats and show some of the other options the converter has to offer.


Using your data


Google Earth (KML)

The first obvious thing you might try is to just convert to KML and import the data to Google Earth.

location_history_json_converter.py LocationHistory.json -o earth.kml -f kml

With the amount of points this would produce it will most likely crash Google Earth, but you can use the -s and -e options to define start and end dates, limiting the amount of locations to be displayed.

location_history_json_converter.py LocationHistory.json \
                                   -o earth.kml -f kml \
                                   -s 2017-01-01 -e 2017-01-31

Of course having tons of pins on Google Earth isn't all that good looking, but it's a start.


Google My Maps (gpxtracks)

For the travel report of our recent vacations in Laos I've created a map with our routes and places we visited. As a starting point I've used my location history of the time converted to gpxtracks to get the routes.

location_history_json_converter.py LocationHistory.json \
                                   -o laos.gpx -f gpxtracks \
                                   -s 2017-01-12 -e 2017-01-29 -c

With location history being reverse-chronological by default it's important to add the -c (chronological) parameter to make sure the data is created correctly.

After creating a new map in Google My Maps you can then import the gpx file.

There's still a lot of work involved in editing and cleaning up the map, but it's a great point to start from. And you get some statistics for free.

The most annoying part of the import is the "stars" created by GPS-signals shifting around when it should actually be stable. I guess this could be fixed (maybe by adding a filter for location accuracy to the script) but I'll have to do some testing.


Location Trail (js)

One of my first experiments with Google Takeout data resulted in this simple visualization of a "snake" wiggling its way through your location history. For this I used the js output because it makes including the data in a website trivial.

You can read more about this project here and here. And it's also open sourc eif you want to play with your own data: https://github.com/Scarygami/location-history-trail

Journey to the Moon (json)

Journey to the Moon was a fun project I got up and running with the help of some friends, that takes your location history data and does some statistics, mainly telling you how far you have travelled compared to the distance to the moon. Theoretically you could use the downloaded LocationHistory.json on this page directly, but those files tend to be big and JavaScript has problems handling too big files. Using the json format option of the script you can shrink the size of the file down to almost 25% because it removes all "unnecessary" data, keeping only the timestamps and locations.

The project is also open source: https://github.com/Scarygami/journey-to-the-moon

Your Turn

I've seen some examples of my script used out in the wild already, but I'm curious to hear your ideas or examples of putting your location history to a better use (with or without my script). So don't be shy and go explore your history.