Wednesday, October 9, 2013

World Births/Deaths Simulation - Adding World Cities

I've been tweaking on the World Birth/Deaths simulation a little bit more.  The (still beta) version is on googledrive at any of these sites:

I have also wrapped this in a self-contained "Chrome Packaged App" that can run in Google's Chrome browser.  You can get the app on the chrome app store here.

I was able to round up world city names and locations, so that this could be included as well.  Specifically, I came across  Josh Smith's worldwide-city-database on github, which itself is based on the Maxmind WorldCities Database.  This is a tab-delimited ~130MB file of a more than two million world cities/places, their population, and latitude/longitude.  Since I already had US cities and their location, I parsed out just the non-us cities with a population of at least 10,000 via a sequence of awk commands (included for my future reference):
awk -F $'\t' '{if ($3!="us") {print $0}}' cities.sql >
awk -F $'\t' '{if ($2>10000) {print $0}}' >
This resulted in a file of about 1MB containing information on about 21,000 cities.  I then needed to convert it from ISO-8859-1 to UTF-8 via this command
iconv -f ISO-8859-1 -t UTF-8 > 
which resulted in the data file used in the visualization (I only realized I needed to do this conversion after some of the characters were not rendering correctly).

Here's an example record from the data file (with the header row included)
Andorra la Vella, Andorra20430ad0742.51.5166667
For a given non-US country, a random city is chosen by picking randomly based on the city's population as fraction of the sum of all of the cities for the country in this data file - this leaves out the smaller cities that might need to be included in some cases, and so I might return to this.

A recent screenshot is below.  Clicking on one of the cities in the lists will perform a google search on that city in a new window.  I still need to deal with the challenge of displaying the events on the map as they occur in a way that makes it easier to discern the specific places.

There are so many cities I just had no idea existed.  And seeing the simulated events in such specific geographic context as the simulation progresses adds a sobering poignancy.

Note: The simulation described here includes data created by MaxMind, available from

Popular Posts