Yelp: Datastore Architecture - Orchestrating Cassandra with Kubernetes: Challenges and Opportunities

Add to your list(s) Download to your calendar using vCal

• Raghavendra D Prabhu
• Wednesday 13 November 2019, 13:00-14:30
• FW26, Computer Laboratory.

If you have a question about this talk, please contact Christine Georgiou.

Food Provided

Yelp is a website and app that connects people with great local businesses and revolves around the connections between them. To facilitate these applications, Yelp has hundreds of microservices deployed and updated in the backend every day. To support these microservices, Yelp has a polyglot architecture consisting of datastores such as MySQL, Cassandra, Elasticsearch, and so on.

Cassandra, the NoSQL database of choice at Yelp, has been deployed on AWS compute (EC2) and AutoScaling Groups (ASG), backed by AWS Block Storage (EBS) for storage. This deployment model has proven to be quite robust against ASG lifecycle state transitions while presenting its own set of challenges. In order to make our Cassandra deployment more resilient and reduce the toil associated with the constantly growing infrastructure, we are abstracting Cassandra deployments further away from EC2 by deploying them in Docker and orchestrating with Kubernetes (K8s) operators. We are also leveraging Yelp PaaSTA for K8s for features such as load-based autoscaling with Clusterman and Spot fleets, features that would be quite useful for an efficient datastore deployment.

This talk is about the opportunities in reliability and availability of such an orchestrated Cassandra deployment. We also discuss the challenges that we have faced along the way and tradeoffs done.

Speaker bio: Raghavendra works as a Software Engineer in the Database Reliability Engineering team at Yelp’s London office, dealing with datastores such as Cassandra and MySQL. Prior to that, he worked at Percona as the Product Lead of Percona XtraDB Cluster (PXC) and at Yahoo as a Systems Engineer. He has also spoken at various conferences such as Percona Live, FOSDEM , linux.conf.au (LCA), Fossetcon, RICON , Highload++, and SCaLE. His main interests include distributed systems, databases, operating systems, and virtualization.

This talk is part of the Technical Talks - Department of Computer Science and Technology series.

This talk is included in these lists:

• All Talks (aka the CURE list)
• Cambridge Centre for Data-Driven Discovery (C2D3)
• Cambridge talks
• Chris Davis' list
• Department of Computer Science and Technology talks and seminars
• FW26, Computer Laboratory
• Guy Emerson's list
• Interested Talks
• School of Technology
• Technical Talks - Department of Computer Science and Technology
• Trust & Technology Initiative - interesting events
• bld31
• ndk22's list
• ob366-ai4er
• rp587

Note that ex-directory lists are not shown.