Founder of OpenHeatMap Pete Warden provides a simple introduction to MapReduce, the Google-created framework for examining large datasets. This video tutorial includes four videos: an intro to the topic, writing your first MapReduce job, and then two videos on running MapReduce jobs on the Amazon Elastic MapReduce cloud service. By the time you’re done watching the videos, you’ll understand what MapReduce is and how to write simple MapReduce jobs, and you’ll be ready to move on to more advanced MapReduce topics.
The videos use Python as the language for writing your mapper and reducer, but the code stays simple and even programmers unfamiliar with Python should follow along just fine. The tutorial has you writing real code, working on real data, and producing real results. You learn the principles necessary to apply to more advanced mapping and reducing scenarios. You can reasonably launch from the course material to write complex MapReduce jobs, depending on your programming skills and imagination.
This is definitely an introduction; the material moves a little slow, and some of the questions from the audience are pretty basic. Part of the Amazon foray is troubleshooting some of the audience members’ setups, for example, though Warden’s lead is straightforward throughout. For novice programmers, I’m sure the troubleshooting scenes are valuable; for professional programmers, they can be a bit tedious. It’s a small price to pay, however, for the information you’ll extract.
If you’re looking to take the mystery out of MapReduce and understand how to use the Amazon Elastic MapReduce service to run your own MapReduce jobs, you’ll find what you need here.
A new web service, If This Then That (ifttt), allows you to wire web triggers to web actions. Go check it out. I just wired this blog to Twitter: anytime I blog something (the trigger or “this”), then I’ll automatically tweet about it (the action or “that”). In addition to informing you about this service, this blog post exists to test whether my ifttt task works!
Greatest Player Ever
Pros: Celtic pride, Crisply detailed
Best Uses: Game room
Describe Yourself: Die Hard Sports Fan
Larry Bird was the greatest ever on the best sports franchise ever. This will bring back memories!
I’m posting this so others can avoid the frustration I felt yesterday trying to use OmniAuth in a Rails 3.1 application. After adding this to my Gemfile:
gem "omniauth", :git => "git://github.com/intridea/omniauth.git"
I got this gem installed:
* omniauth (1.0.0.alpha ba4bd3e)
When I tried running anything, however, such as
rails generate . . . or
rake spec, I’d get a stack trace complaining about:
uninitialized constant OmniAuth::Builder
I googled and tried various things, coming to understand that I really wanted omniauth 0.3.0.rc1 but realizing I had no clue how to get it until I found this: http://collectiveidea.com/blog/archives/2010/09/28/gemfiles-and-branches/. So I added this to my Gemfile:
gem "omniauth", ">= 0.3.0.rc1", :git => "git://github.com/intridea/omniauth.git", :branch => "0-3-stable"
and everything worked.
- Once people get your point, shut up.
- RT @rbazinet: This is awesome -You call yourself a programmer? Well then, use a REAL programmer's keyboard: t.co/YO5OsgYP8B (via @an…
- RT @qrush: My guess is 2 years for Tumblr to get shuttered. One year to get renamed into Yahoo! Tumbles
Follow @hoop33 on twitter.
What I’m Writing
- May 2013 (3)
- April 2013 (6)
- February 2013 (1)
- January 2013 (1)
- November 2012 (6)
- September 2012 (1)
- August 2012 (6)
- July 2012 (10)
- June 2012 (4)
- May 2012 (7)
- April 2012 (4)
- March 2012 (8)
- February 2012 (7)
- January 2012 (7)
- December 2011 (1)
- November 2011 (1)
- September 2011 (4)
- August 2011 (2)
- July 2011 (2)
- June 2011 (2)
- May 2011 (4)
- April 2011 (2)
- February 2011 (1)
- January 2011 (1)
- December 2010 (3)
- November 2010 (3)
- October 2010 (4)
- September 2010 (6)
- August 2010 (96)