One of the values that we cherish at Yelp is to “Be Unboring”. It’s the quality of never accepting “standard” as okay and the guiding principle for creating new and remarkable things. One of the ways we foster that creative spirit at Yelp Engineering is through our internal hackathons. Many remarkable projects have come out of them - some of them open sourced, some of them revealing interesting demographic insights and some of them pushing the boundaries of science & technology. The 19th edition of our internal hackathon wasn’t any different. Close to 80 fantastic projects across our engineering offices in San Francisco and Hamburg came out of the event.

Here are some highlights:

Food Coma

Yelp hosts tens of millions of photos uploaded by Yelpers from around the world. A lot of fun, useful and cool projects have been built on top of our rich photo data. A team comprising of Shahid C., Keegan P. and Piyush K. decided to take it one step further - what if we took some of those photos and created cool, psychedelic dream sequences out of them? They put together an algorithm called a “dream filter” and applied it on photos of dishes from local SF restaurants to create sequences like this :

To build the “dream filter”, they used the GoogleNet Caffe model from BVLC (Berkeley Vision and Learning Center) that was pre-trained on the ImageNet dataset. Their filter consisted of a forward-pass through a convolutional neural network up to a specified layer. They then set the gradients equal to the activations at that layer. That way, they could backpropagate through the network and then transform the original image accordingly. They passed in numerous photos through this filter and stitched up the individual frames together to create a single dream sequence. Because this process requires hundreds of frames, they used an AWS EC2 GPU (g2.2xlarge) instance to improve performance in Caffe.

MrKafkaJob

We run a lot of mapreduce jobs at Yelp, some on local Hadoop clusters and some on Amazon’s EMR framework. We also heavily rely on technologies like Kafka and Docker, with many of our systems using Kafka to transport data, and Docker for containerization of services (as part of PaaSTA, our open-sourced SOA platform). Would it be possible to leverage both of these technologies to replace Hadoop? Ryan I. set out to answer that question. He put together a workflow that pieces together Docker and Kafka, to create a new way of running mapreduce jobs. Docker compose creates an ephemeral Kafka cluster of desired scale, enabling mappers and reducers to communicate in parallel via partitioned Kafka topics. Turns out that it worked pretty well!

Frontend Services on Python 3

We value our ability to quickly ship code. One key factor that has helped us maintain our iteration speed has been our move to a service-oriented infrastructure. Over the past couple of years, we’ve been breaking apart our monolith to create new frontend services which serve individual pages of our website. Many of these services use Python2. As the community moves towards adoption of Python3, there may be a growing need for us to switch our services to use Python3 instead. Driven by the curiosity to find how complex that switch would be, Anthony S. spent his hackathon porting a handful of modules of varying complexity from Python2 to Python3 with backward compatibility support. Some of those modules included pgctl (our playground controller to manage developer playgrounds), Cheetah (a templating engine), yelp_clog (Python package for logging to scribe), bravado (Python client library for Swagger) and fido (an asynchronous HTTP client).

Mechanical Unix Clock

As engineers, we all love hardware hacks. Especially the ones that involve building something from scratch. And especially, the ones that involve Unix. A team comprising of Evan K., Joseph L. and Matthew S. came up with a clever hardware hack to keep track of unix time - they built a clock! Using our resident 3D printer at the office, the team printed out tiny gears that could interlock and spin each other. Each of these gears represented a digit in unix time. They then hooked up these gears to a motor calibrated to move the first gear to increment by a second. The subsequent gears would then move in lockstep to try to keep the unix time consistent.

What happens when we hit the next leap second? Hmm…only time will tell.

Back to blog