University of Cambridge > Talks.cam > Microsoft Research Cambridge, public talks > CIEL: a universal execution engine for distributed data-flow computing

  Browse A-Z

Log in

Cambridge users (raven)details Other usersdetails No account?details

Information on

Subscribing to talksdetails Finding a talkdetails Adding a talkdetails Disseminating talksdetails Help and Documentationdetails

CIEL: a universal execution engine for distributed data-flow computing

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Microsoft Research Cambridge Talks Admins.

Distributed data-flow frameworks, such as MapReduce and Dryad, have made it simpler to program a large number of computers at once. However, as many researchers have observed, existing frameworks provide a restricted programming model that inhibits the implementation of many naturally data-parallel algorithms. To address this problem, we have developed CIEL , which is a distributed execution engine that extends previous systems by allowing data-dependent control flow. This enables CIEL to natively execute iterative and recursive algorithms, including k-means clustering, PageRank and many sparse linear algebra methods.

In this talk, I will introduce CIEL , and describe the techniques that we use to make execution reliable. I will also describe Skywriting, which is a Turing-complete scripting language that runs directly on a CIEL cluster, and simplifies the construction of iterative jobs. We have deployed CIEL on a cloud computing platform, and I will present performance results that compare CIEL with Hadoop, the open-source implementation of MapReduce.

This talk is part of the Microsoft Research Cambridge, public talks series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

Other lists

BAS Aurora Innovation Centre Dio-Gandhi equations Reproduction on Film: Outlaws

Other talks

The Gopakumar-Vafa conjecture for symplectic manifolds Neurodevelopment disorders of genetic origin – what can we learn? Current-Induced Stresses in Ceramic Lithium-Ion Conductors Refugees and Migration Participatory approaches to encourage responsible use of antibiotics in livestock Group covariance functions for Gaussian process metamodels with categorical inputs "Mechanosensitive regulation of cancer epigenetics and pluripotency" BP KEYNOTE LECTURE: Importance of C-O Bond Activation for CO2/COUtilization - An Approach to Energy Conversion and Storage Refugees and Migration Fields of definition of Fukaya categories of Calabi-Yau hypersurfaces Microtubule Modulation of Myocyte Mechanics Bears, Bulls and Boers: Market Making and Southern African Mining Finance, 1894-1899
 

© 2006-2024 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity