Spark is the hot next thing for Hadoop / MapReduce, and yesterday Matei Zaharia, a PhD student in UC Berkeley’s AMP Lab, gave us a terrific lecture about how it works and what’s coming next. The key idea is to make analysis of big data interactive and able to respond in real time. Matei also gave a live demo. Watch here:
http://youtu.be/rpXxsp1vSEs
Next up in the research agenda is streaming data and blending real time and batch processing. This research team is doing truly amazing work work and we’re very grateful to @matei_zaharia for the clear and impressive lecture! (slides here)
(Sorry, no camera on this one so we can’t see Matei.)