Yelp Dataset Challenge Round 11 Is On! The eleventh round of the Yelp Dataset Challenge has opened. It will run until June 30, 2018. As in the past, the Yelp Dataset Challenge gives college students access to reviews and businesses from 11 metropolitan areas scattered over 4 different countries. This time around, there are a staggering 5.26 million reviews written by 1.3 million users about 175,000 businesses, as well as 146,350 check-ins and 1.1 million tips left by these users. Moreover, we have added photos about these businesses in a separate file, for convenience. With such a trove of data,...

Continue reading

Growing Cache Friendly Trees In case you missed it, Part I of this blog series outlines how we built a distributed machine learning platform to train gradient boosted tree (GBT) models on large datasets. While we were able to observe significant improvements in offline metrics, the resulting models are too large for standard XGBoost prediction libraries to meet our latency requirements. As result, we were unable to launch the models in production as we needed to serve ads within 50ms (p50) and evaluating these large models caused time-out exceptions. This article will discuss how we compressed and reordered the trees...

Continue reading

Building a Distributed Machine Learning Pipeline As a part of Yelp’s mission to connect people with great local businesses, we help businesses reach potential customers through advertising and organic search results. The goal of Yelp’s advertising platform is to show relevant businesses to users without compromising their experience with the product. In order to do so, we’ve built a click-through-rate (CTR) prediction model that determines whether or not to serve ads from a particular advertiser. The predicted CTR determines how relevant the business is to the user’s intention and how much would need to be bid to beat a competitor...

Continue reading

Yelp Dataset Challenge Round 9 Winners The ninth round of the Yelp Dataset Challenge ran throughout the first half of 2017 and, as usual, we received a large number of highly impressive and interesting submissions. Needless to say, we were struck by the quality of the entries: keep up the good work! Today, we are proud to announce the grand prize winner of the $5,000 award: “CORALS: Who are My Potential New Customers? Tapping into the Wisdom of Customers’ Decisions” by Ruirui Li, Chelsea J-T Ju, Jyunyu Jiang, and Wei Wang (from the Department of Computer Science of the University...

Continue reading

Earlier this year, Gmail users across the globe were affected by one of the largest phishing attacks of its kind. Yelp emails were among the many corporate email systems that experienced this Google Docs phishing attack. Fortunately, our security engineers had already prepared for this level of security threat and were able to delete the suspicious emails before impacting employees. As phishing attacks have become more and more prevalent, the need for new tools and countermeasures to protect users has become more important than ever. According to the last IBM X-Force Threat Intelligence Index report, the amount of spam email...

Continue reading

As we’ve discussed in earlier blog posts, Yelp Engineering has been working hard to break down our largest monolithic code base (yelp-main) for the past few years. We’ve made great progress but some of our oldest, most critical code remains within yelp-main. A great example of an older, more established system is our monthly subscription billing cycle. The system is core to how Yelp collects revenue and has proven technically challenging and risky to transition. The Revenue engineering team knows these older systems should be moved into services, but the challenge of extracting tangled, business-critical code has proven expensive and...

Continue reading

We deeply value code review and feel that it’s crucial to being a high-functioning engineering organization. Code review results in higher quality code that is more broadly understood. It also lets engineers learn from their peers, practice mentorship, and engage in open dialog and discussion about what they build. The benefits of code review align well with Yelp’s value Play Well with Others and support our culture of continually teaching and learning. As our organization has continued to grow, there are certain patterns that have made code reviews more beneficial and keep them from becoming a bottleneck. We’ve been following...

Continue reading

Yelp returned to Grace Hopper Celebration once more, this time in Orlando! Let’s take a look at what GHC2017 is all about from the insights of our attendees. Who Neha H.: Android engineer @ Search User Experience team Grace J.: Recruiting manager @ University Recruiting team Tiffany K.: Product manager @ Contributions team Lauren C: Product designer @ Messaging team Xun T.: Software engineer @ Ad Creative team, first time speaker at GHC this year. Favorite sessions during GHC Tiffany: Women Who Build The Product Management Journey. I really appreciated hearing from women leaders who have been successful in the...

Continue reading