Engineering Blog

Reflections from Grace Hopper (Part 2)

Welcome back! Today we have Wei, Rachel, Jen, Virginia and Anusha sharing their experiences. Wei is an engineer on the consumer team and she brings amazing user experiences to our customers. Rachel and Jen are both engineers on our international team, bringing the power of Yelp to all of our international communities. Virginia works as an engineer on our partnerships team and Anusha is an engineer on our infrastructure team. Overall, we all had a blast getting to know each other, meeting other amazing women in the industry, hearing some great stories from inspiring women role models, and sourcing some...

Continue reading

Reflections from Grace Hopper (Part 1)

As you probably heard, Yelp attended Grace Hopper this year. Nine software engineers from different teams attended and, for many of us, it was our first time. It was a unique experience to see so many talented women in one place. In addition to the talks and panel discussions, we also had the opportunity and pleasure to represent Yelp at the career fair. It was amazing to see a consistent flow of students and industry talent, all happy customers, stop by our booth to speak with us and tell us their stories of using Yelp. We had such a great...

Continue reading

Scaling Traffic from 0 to 139 Million Unique Visitors

At LAUNCH Scale last week, I gave a talk to over 75 co-founders (CEOs and CTOs) on how we’ve scaled traffic here at Yelp. It brought back memories of Darwin biting through our ethernet cable and reminded me of the run up to our IPO, making sure we had enough capacity to handle the expected surge in traffic from the world’s press (and more recently, the launch of Yelp in Hong Kong!). For close to 8 years, I’ve had the privilege to work alongside some of the best engineers in the world and have seen the meticulous work and thought...

Continue reading

Introducing Pyleus: An Open-source Framework for Building Storm Topologies in Pure Python

Yelp loves Python, and we use it at scale to power our websites and process the huge amount of data we produce. Pyleus is a new open-source framework that aims to do for Storm what mrjob, another open-source Yelp project, does for Hadoop: let developers process large amounts of data in pure Python and iterate quickly, spending more time solving business-related problems and less time concerned with the underlying platform. First, a brief introduction to Storm. From the project’s website, “Apache Storm is a free and open source distributed realtime computation system. Storm makes it easy to reliably process unbounded...

Continue reading

Using MOE, the Metric Optimization Engine, to Optimize an A/B Testing Experiment Framework

A/B Testing Experiment Frameworks and MOE We recently open sourced MOE, the Metric Optimization Engine, a machine learning tool for solving global, black box optimization problems. An example application for such a system is optimally running online A/B experiments. A/B testing segments the users that come to a site into buckets, or cohorts, and show different versions of the site to different cohorts of users. One can show 50% of users one version of a site (version A) and 50% of users another version of a site (version B). After some amount of time we can see which version of...

Continue reading

October Events at Yelp

October will be a busy month for all of us, with a lot of great conferences and events happening back-to-back. We start the month off with an SF Python meetup in collaboration with PyLadies. Zach Musgrave will be presenting on performance profiling in production. Later this month, Scott Clark will be presenting on MOE for SF Machine Learning here at Yelp HQ too! If you’re not familiar with MOE, it’s our black box optimization engine to help you with real world metric optimization. We’ll also be at Grace Hopper on October 8-10, StrataConf on October 15-17, and at HTML5DevConf on...

Continue reading

Yelp At Grace Hopper

Yelp is happy to announce that we’re headed to the Grace Hopper Celebration of Women in Computing conference in October! Admiral Grace Hopper was a pioneer in computer science. She was a driving force behind the development of English-based programming languages and an inspiring example of how intelligence and perseverance can overcome technical and cultural adversity. The Grace Hopper conference, presented by the Anita Borg Institute and ACM, is the largest gathering of women in computing in the world. Its mission to “connect, inspire, and guide women in computing and organizations that view technology innovation as a strategic imperative” is...

Continue reading

Intern Project: Real-Time Log Tailing with Franz, our Kafka Wrapper

At Yelp, we often need to analyze data in real time. Right now most of our data is aggregated and logged using a system called Scribe. This means that real-time processes currently depend on tailing Scribe, reading messages as they are added to the end of the logs. Simply tailing the logs means that any time a tailer is not running, due to maintenance or an unexpected error, it misses all the information logged during that time. We want a more flexible system that allows processes to pick up where they left off, so our analyses account for all of...

Continue reading