At Yelp, usage of Prometheus, the open-source monitoring system and time series database, is blossoming. Yelp is initially focusing on onboarding infrastructure services to be monitored via Prometheus, one such service being Apache Kafka. This blogpost discusses some of the performance issues we initially encountered while monitoring Kafka with Prometheus, and how we solved them by contributing back to the Prometheus community. Kafka at Yelp primer Kafka is an integral part of Yelp’s infrastructure, clusters are varied in size and often contain several thousand topics. By default, Kafka exposes a lot of metrics that can be collected, most of which...

Continue reading

Understanding data is a vital part of Yelp’s success. To connect our consumers with great local businesses, we make millions of recommendations every day for a variety of tasks like: Finding you immediate quotes for a plumber to fix your leaky sink Helping you discover which restaurants are open for delivery right now Identifying the most popular dishes for you to try at those restaurants Inferring possible service offerings so business owners can confidently and accurately represent their business on Yelp In the early days of Yelp circa 2004, engineers painstakingly designed heuristic rules to power recommendations like these, but...

Continue reading

Yelp periodically releases an open, all-purpose dataset for learning. The dataset is a subset of our businesses, reviews, and user data to inform government policy, academic research, and business strategy, among other uses. It has provided opportunities including teaching students about databases, helping others study natural language processing, sampling production data while learning to create mobile apps, and discovering compelling research findings. Our most recent dataset was published in March 2020. Businesses everywhere are adapting to the effects of the Coronavirus and have been using Yelp features to stay connected with their customers. To this end, we’re releasing an addendum...

Continue reading

We’re open sourcing dataloader-codegen, an opinionated JavaScript library for automatically generating DataLoaders over a set of resources (e.g. HTTP endpoints). Go check it out on GitHub! This blog post discusses the motivation and some the lessons we learned along the way. Managing GraphQL DataLoaders at Scale At Yelp, we use GraphQL to provide data for our React webapps. The GraphQL Server is deployed as a public gateway that wraps hundreds of internal HTTP endpoints that are distributed across hundreds of services. GraphQL Request Diagram DataLoaders DataLoaders provide an important caching/optimization layer in many GraphQL servers. If you aren’t already familiar...

Continue reading

It’s 2014 and your teams are divided by platform, something like: Web, Mobile Web, Android, and iOS. In order to launch features, product managers jump from platform to platform and teams move fast. Really fast. Lines of code in each repository increase to the point where you now name them “monoliths.” A few engineers maintain these monoliths when they need to, but no one is solely dedicated to the task. Engineers are distributed by platform; so communication on when to maintain the monoliths is easy, but presents another problem. Can you continue to ship code efficiently if you depend entirely...

Continue reading

Yelp extensively utilizes distributed batch processing for a diverse set of problems and workflows. Some examples include: Computation over Yelp’s review corpus to identify restaurants that have great views Training ML models to predict personalized business collections for individual users Analytics to extract the most in-demand service offerings for Request a Quote projects On-demand workloads to investigate surges in bot traffic so we can quickly react to keep Yelp safe Over the past two years, Yelp engineering has undertaken a series of projects to consolidate our batch processing technologies and standardize on Apache Spark. These projects aimed to simultaneously accelerate...

Continue reading

Summary Here at Yelp, we generate business wins and a better platform by running A/B tests to measure the revenue impact of different user and business experience interventions. Accurately estimating key revenue indicators, such as the probability a customer retains at least \(n\)-days (\(n\)-day retention) or the expected dollar amount a customer spends over their first \(n\) days (\(n\)-day spend) is core to this experimentation process. Historically at Yelp, \(n\)-day customer or user retention was typically estimated as the proportion of customers/users we observed for more than \(n\) days who retained more than \(n\) days. Similarly, \(n\)-day spend was estimated...

Continue reading

Last October we sent a group of Yelpers to the 2019 Grace Hopper Celebration! Here are a few takeaways and reflections from some of our attendees. Who attended? Surashree K., software engineer on Semantic Business Information Clara M., product design lead on Content Anna F., machine learning engineer on Semantic Business Information Nikunja G., software engineer on Infrastructure Security Catlyn K., software engineer on Stream Processing What was your favorite session? Surashree: Honestly, it’s hard to choose, but the one that stuck with me was the talk by Jackie Tsay and Matthew Dierker on Google’s Smart Compose, the Gmail feature...

Continue reading