At Yelp, we’re very big fans of Amazon’s RedShift data warehouse. We have multiple deployments of RedShift with different data sets in use by product management, sales analytics, ads, SeatMe and many other teams. While it minimizes a lot of the work the RedShift team has done to call RedShift a simple fork of Postgres 8.4, RedShift does share a common code ancestry with PG 8.4. This means that much of the advanced query functionality of Postgres is available, which, when combined with the petabyte scale of RedShift, offers some amazingly powerful analytics tools. Because most of the PG 8.4...

Continue reading

2014 was a busy year for us at Yelp Engineering and with 2015 having just kicked off, we wanted to highlight some of our most exciting events from last year. Open Source Yelp loves open source. We love contributing back to the developer and open source communities and last year was no exception to that. We saw a large number of our engineers contributing to many different projects. In July, to help organize all of the projects we worked on, we launched a new open source site along with the revamped developer site. Over the course of the year, we...

Continue reading

Introducing OSXCollector We use Macs a lot at Yelp, which means that we see our fair share of Mac-specific security alerts. Host based detectors will tell us about known malware infestations or weird new startup items. Network based detectors see potential C2 callouts or DNS requests to resolve suspicious domains. Sometimes our awesome employees just let us know, “I think I have like Stuxnet or conficker or something on my laptop.” When alerts fire, our incident response team’s first goal is to “stop the bleeding” - to contain and then eradicate the threat. Next, we move to “root cause the...

Continue reading

This past November, Yelp Engineering hosted a Bay Area Girl Geek Dinner and I had the pleasure of presenting a brief overview of how Yelp uses MySQL. I received a lot of great questions and wanted to take a few minutes to share the information with our wider blog audience. Overview: MySQL @ Yelp Yelp has used MySQL since the beginning, and in more recent years has adopted the Percona fork. We like MySQL because it fits our data needs, is configurable, fast, and scalable. The Percona fork includes performance enhancements and additional features on top of stock MySQL, we...

Continue reading

At Yelp, we solve a lot of different information retrieval problems ranging from highlighting reviews for businesses to recommending nearby restaurants. We recently improved one system called business matching, moving from a naive ranking function to a machine learned approach using Learn to Rank. The Business Matching Problem Business matching is a system that accepts a description of a business (e.g. name, location, phone number) and returns the Yelp businesses that description best fits. We can use this in several different contexts. For example, in 2012 Yelp announced a partnership spearheaded by the City of San Francisco to allow municipalities...

Continue reading

While we’re all still recovering from our Thanksgiving-induced food comas, we wanted to share some of our upcoming events for December! This Thursday, some of our engineers will be giving lightning talks at the Women Who Code meetup so make sure to come! We’re also very excited to be sponsoring the Women Who Code Holiday Party in Waterloo Ontario on December 11th. If you’re in the area, RSVP and stop by! For the rest of the week (through December 5th), Yelp will also be matching donations to Women Who Code (up to $5,000 total). Please consider donating and helping them...

Continue reading

We recently held the 15th edition of Hackathon, a two-day event comprising of pure, unadulterated hacking, innovation, creativity and fun. Folks clustered together, hammering on their keyboards, scribbling on whiteboards, and some electrical engineers were plugging away on their breadboards. The kitchens were filled with delicious catered food, and fresh fruits and snacks lined up in the common spaces. We even had a school of remote-controlled sharks lurking around the office and taking over the meeting rooms. Those ominous-looking RC sharks do not seem to deter our intrepid hackers! Close to 60 great projects came out of the event from...

Continue reading

Yelp uses Elasticsearch to rapidly prototype and launch new search applications, and moving quickly at our scale raises challenges. In particular, we often encounter difficulty making changes to query logic without impacting users, as well as finding client library bugs, problems with multi-tenancy, and general reliability issues. As the number of engineers at Yelp writing new Elasticsearch queries grew, our Search Infrastructure team was having difficulty supporting the multitude of ways engineers were finding to send queries to our Elasticsearch clusters. The infrastructure we designed for a single team to communicate with a single cluster did not scale to tens...

Continue reading