Category Archives: Uncategorized

Alpa Jain has great experience teaching from her time as a graduate student at Columbia University, and it shows in the clarity of her descriptions of SVD and other recommendation algorithms in today’s lecture: http://www.youtube.com/watch?v=NSscbT7JwxY When I asked at the … Continue reading →

On Thursday, Delip Rao electrified the class with a lecture on how Twitter combats the pervasive threat of tweet spam: (Lecture notes are not available.) UC Berkeley students are lucky to have gotten to hear @deliprao in person.  Thank you … Continue reading →

For those of you who follow the latest developments in the Big Data technology stack, you’ll know that GraphLab is the hottest technology for processing huge graphs in fast time.  We got to hear the algorithms behind GraphLab 2 even … Continue reading →

Learn about weak ties, triadic closures, and personal pagerank, and how they all relate to the Twitter social graph from Aneesh Sharma: http://youtu.be/_eWDR3P8Lhw I especially enjoyed his fascinating and clear explanation of the Watts-Strogatz model, its link to Kleinberg’s model, … Continue reading →

Today we learned about an alternative software architecture for processing large data, getting the technical details from Splunk’s VP of Engineering, Stephen Sorkin. Splunk also has a really amazing GUI for analyzing Twitter and other data sources in real time; … Continue reading →

Brian Larson gives a brilliant technical talk about how real-time search works at Twitter; He really knows what he’s talking about given that he’s the tech lead for search and relevance at Twitter! http://youtu.be/PAYOjnLICo4 Follow him @larsonite to learn more. … Continue reading →

Have you ever wondered how Twitter computes its Trending Topics?  Kostas T. is one of the wizards behind that, and today he shared some of the secrets with our class: http://www.youtube.com/watch?v=duHxpSTmwW0 See his lecture notes.  Kostas, Rion and I are … Continue reading →

Today Rion Snow saved us huge amounts of time by giving us a primo introduction to the Twitter API.  We learned about both the RESTful API and the streaming API for both Java and Python. After this intro, it looks … Continue reading →

Jon Coveney gives an in-depth tutorial on Apache Pig, including how it interacts with Hadoop: http://youtu.be/UiIjEzW3br8 His slides are a fantastic resource for learning much much more about Pig: (pdf) Thanks for a really illuminating lecture, Jon!

Bill Graham, who is active in the Hadoop community and a Pig contributor, gave a very clear and detailed intro to Hadoop and outlined how it is used at Twitter: http://www.youtube.com/watch?v=t3fEGhE-HYA&feature=youtu.be His slides can be found here. Bill is a terrific … Continue reading →