NOTE (Mar 16, 2014): This visualization has been modified to work with the new 1.1 twitter api, as twitter removed the version that this visualization was originally built against. You can see the updated version at the link below:
https://ustweetmap-bflnni.rhcloud.com/
Per the new api, you must sign in via twitter in order to use the search functionality. There are some notes on this implementation using node.js in this blog post.
All of the information below pertains to the old, no-longer-working version.
Using this svg map of the United States, in conjunction with the public twitter search api and the awesome D3.js javascript library, I've created a searchable real-time map of the public tweets from the US. The map can currently be found here (link is updated to new site). The design is based on that of the US Births/Deaths simulation I was a collaborator on. The list is updated about every minute.
For tweets that include lat/long information, the county is determined via a dynamic call to the FCC census block conversions api (I haven't gotten around to geocoding this map which would make this step unnecessary).
There are already tweet maps out there - enough to make me think I should have named this little project "Yet Another Tweet Map". Most seem to use the google maps api. Some of these are also hooked straight into the twitter public stream api rather than simply making use of the twitter search API as is done here.
- TwitterMap, a project by Matthew Isabel, is also based on D3, and allows a search of tweets in the US - he plots the tweets at their actual latitude/longitude on the US map
- Tweereal integrates google maps with the twitter public stream to show the pattern of tweets across the world; the tweets (without the tweet contents) are marked with colorful circles that beautifully appear and fade like raindrops hitting the globe
- the One Million Tweet map shows a world map of the last
onetwo million tweets, has options for filtering by keyword, and can show activity either via big circles with tweet counts, or via a heat map - Trendsmap is another map shows Twitter trends around the world
- CoffeeMorning shows tweets in the UK about the Macmillan Cancer Support fundraising event (via certain hashtags)
- Tweet to Map is an inexpensive jQuery plugin that simplifies integration with Google maps (it seems to use Twitter's api rather than the twitter firehouse)
- #uksnow integrates google maps with the twitter search api to report on where it is snowing in the UK
- Globaltweets is another mashup with google maps that shows tweets around the world
- Twittervision is a perhaps one of the first mashups with google maps (from 2007!), but this is apparently no longer working
Searching for Tweets
The page starts with a sample search that demonstrates what I considered to cover the most likely scenarios: requiring something to be in the search, and something to not be in the search.
Searching can be done in a somewhat "google-like" manner, although you've got to include a "+" to force the term or quoted phrase to be there. You can also make sure that a term not be included by prepending a "-". If neither prefix is included, then the term is optional. The entered search string is mapped to the proper format needed by the twitter api itself. Well, the version 1 api, which is deprecated and will be unavailable at some point. The v1.1 search api requires a "signed user context", which involves using oAuth with twitter, and that's a little project for another day.
Lots-o-Stuff Happening
One interesting challenge with this is that there can of course be a burst of tweets, and decisions must be made as to how much of the burst to display. Some of the other tweet maps out there have to deal with this to an even great degree. It's not clear how to show all of the tweets on the map in any reasonable time frame while preserving readability. So, I don't even try. If there are more than a certain number of tweets returned for a given search (five in this case), I let the tweet popup on the map, but quickly shove all but the last handful of tweets into the list on left.
County and List Element Color based on Number of Tweets in County
As tweets occur in a county, its color is gradually changed, ultimately resulting in a solid red county once a certain threshold is reached (ten tweets in this case). In parallel with this color change, the border of the all corresponding tweets in that county that are already in the list will change as well. Moving your mouse over a county will bring up a list of all of the tweets recorded for the county itself.
Legend o' the Lakes
I have previously written a short note on making an svg legend with D3. The legend here is placed above the Great Lakes. There is also space out in the Atlantic, but it seemed to feel a little lost out there. I toyed with the idea of using some kind of shape that followed the contour of the lakes or follow the northern border of the US, rather than the rounded rectangle, but decided that it might be a distraction.
Smooth Scrolldown when Inserting into List
xkcd is On To Me
This is an interesting project to work on, but the geographic aspect hasn't seemed to be particularly insightful yet. Searching for "+rain +school" the other night, it did show little Knox County,TN as a single bright red county across the country from angry school kids when all the counties around it closed schools for rain the next day and Knoxville didn't. So, you can see some geographic uniqueness by choosing specific search terms, but I have a feeling that in most cases you're just watching a population density heat map get filled in (note that this is definitely the case for the birth/death simulation I was part of putting together). xkcd has already spoken on this:
 |
| from http://xkcd.com/1138/) |
