Map overlays (or how to break Google Maps)
28 Nov 2010
TechnologyOriginally posted at https://tech.labs.oliverwyman.com/blog/2010/11/28/map-overlays-or-how-to-break-google-maps/
I’ve been trying to use the TfL bike hire system recently, in a vague aim to get a bit more exercise. One problem with this was that although I had a vague idea where the stops where, I wasn’t exactly sure where the boundaries were offhand. My major use case being the idea of travelling from work to wherever I’m going out in the evening via bike and wondering if that was possible. Given in most cases, I’m checking the location via Google Maps, it made sense to investigate the overlay support therein
You can manually enter points, but Google Maps also supports importing data in a variety of formats. I decided to go with KML on the basis of it being pretty simple to work with. There’s a lot you can do with it, but here’s everything you actually need:
There’s a kml outer tag containing a Document tag and a series of Placemark tags. All you need to do is set the “name” and “coordinates” (latitude, longitude) appropriately and you’re done!
You’re also going to need some data to feed in. For the TfL bikes, my source was the Boris Bikes API which has a useful set of station data in XML that we can easily munge into KML (generator for this).
You’ll also need to insert this into Google Maps as follows:
- Goto Maps
- If you’ve made your own map
- Goto "My maps"
- "Create new map"
- "Import" - hand it the KML you just made
- Or, if you want one someone created earlier
- Do a search on "User-created maps"
- Find one
- Click "Save to My Maps"
Either way, you should have at least one map in “My Maps” now.
Net result of all of this is the following:
This is the combination of an overlay and a location marker (actually from my phone, which I’ll talk about in a second). To get this sort of display on your desktop:
- Search for some location you want on the map as well as the overlay (e.g. “N1 6PE”)
- Click on “My Maps”
- Select the map you just added
- At the bottom of the sidebar there will be a little section marked “Hide”. Expand this and tick the markers for both your search and the overlay you just picked
Also, as I mentioned, you can get this on your phone 
Works certainly on Android phones, and might be ok on other smartphones with Google Maps.
- Bring up “Google Maps”
- Bring up the overlays menu (icon at the top, looks like
) - Click “More layers”
- Select “My Maps”
- Pick the map you just made
Ok, so we’ve got TfL bikes enabled. I was then hunting for more data sources, and one of my colleagues suggested Streetcar. They’ve got a reasonably nifty searcher interface, but it’s not particularly mobile friendly. On the other hand, some digging around in it found me http://www.streetcar.co.uk/LocationsXml.xml
(just a fragment of the available data). Writing a generator for this was once again pretty simple.
I was almost done, but I wanted something a bit more interesting. Like say, the location of every ATM in the UK. Couldn’t quite do that, but the Link site, and it’s non-mobile-friendly search interface got me the list of Link ATMs. This was a bit harder, and needed some work with Firebug, but I was able to get the data I wanted.
This dataset had a lot more points than the earlier ones (~65K points v.s. ~1K), and I started noticing that I wasn’t seeing all the data on Maps that i was expecting. The imported data was still well below the import limits listed (topped out at about 1.1mb, with the limit being 10mb), so I was a bit confused, and therefore I wrote a stress tester. The tester generates a square of points 200m apart, but the results from Maps look like the following instead:
In both cases, what you should be seeing is a 19×19 grid. The left is a screenshot from my desktop, the right from my phone. As you can see, both fail, and in different ways. Also, the results sometimes seem to vary from time to time, and you’ll get a different subset of points at different times. I was a bit confused about this, but I spoke to a few other people and it turns out this is a known problem. I’d been relying on the Google Maps control to be sane and non-broken, but it turns out this was a silly idea. Others have managed to get up to 5,000,000 placemarks in, but only via loading in with the Javascript interfaces as opposed to bulk upload. It appears that it’s got some sort of internal timing issues and if you load more than a certain number of points at one go (I estimate ~100 based on the stress tester, which means the bikes and Streetcar maps will be missing some data as well) it’ll start randomly dropping some of them.
So, where do I go from here? The obvious option would be to write a new app/website that does the right thing with Maps i.e. use “My Maps” as a backend data source and then feed data into the Map control only as fast as it can handle things. However, the Maps Data API is now officially deprecated and although Fusion Tables have been announced as the official replacement, their interaction with “My Maps” still isn’t particularly clear, so I think I’m leaving this one alone for a bit.
FYI, as always, all the code detailed here is available in a Github repository.