Tools and services for data intensive research

Add to your list(s) Download to your calendar using vCal

• Roger Barga (Microsoft Research)
• Wednesday 01 July 2009, 15:00-16:00
• Large public lecture room, Microsoft Research, Roger Needham Building, 7 J J Thomson Avenue, Cambridge CB3 0FB.

If you have a question about this talk, please contact Dr Fabien Petitcolas.

Abstract: For many important research investigations, especially in science, efficiently analyzing large data sets is a major challenge. Microsoft’s Dryad is a high-performance, general-purpose distributed computing engine that handles some of the most difficult aspects of cluster-based distributed computing. It’s powerful: Microsoft routinely uses Dryad applications to analyze petabytes of data on clusters of thousands of computers. Microsoft Research has also developed DryadLINQ, which allows developers to use an extended version of the LINQ programming model and API to implement Dryad applications in managed code. DryadLINQ code is similar to what you’ll see in a conventional LINQ -to-objects application, and the application core is often only a few lines of code. Behind the scenes though, a DryadLINQ provider automatically converts the LINQ query into a Dryad job and executes the query as a distributed application on a cluster. Using Dryad through DryadLINQ, even a novice at parallel processing or cluster-based computing can implement a high-performance distributed application to efficiently analyze terabytes of data.

In this talk we present an introduction to Dryad and DryadLINQ for data intensive research and we compare and contrast it to other related technologies. We describe our ongoing efforts to collaborate with external researchers to explore the application Dryad and DryadLINQ to big data research problems in science. We also highlight our efforts to offer software and services to researchers across the world, through the academic release of Dryad and DryadLINQ with associated programming user documentation.

Biography: Roger Barga is Principal Architect at Microsoft Research, where he leads the Advanced Research Services and Tools (ARTS) team. The ARTS team is responsible for developing innovative tools and services using Microsoft products and technology to revolutionize and accelerate research, and it provides strategic and tactical hands-on technological leadership to many research projects. Roger joined Microsoft in 1997 as a Researcher in the Database Group of Microsoft Research, where he directed both systems research and product development efforts in database, workflow and stream processing systems. He has developed ideas from basic research, through proof of concept prototypes to incubation efforts in product groups. Roger has published over 50 peer reviewed papers, filed over 30 patent applications, and served more than 70 times on program committees for more than 30 different international conferences and workshops.

This talk is part of the Microsoft Research Summer School series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge talks
• Chris Davis' list
• Guy Emerson's list
• Interested Talks
• Large public lecture room, Microsoft Research, Roger Needham Building, 7 J J Thomson Avenue, Cambridge CB3 0FB
• Microsoft Research Cambridge, public talks
• Microsoft Research Summer School
• Optics for the Cloud
• PMRFPS's
• School of Technology
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• personal list
• rp587

Note that ex-directory lists are not shown.