Engineering Blog

July 31st, 2015

Know Thyself: My Internship Quest to Build a Framework to Analyze Our Yelp Feed

Over the years Yelp has built cool integrations with partners such as Apple, Microsoft, Yahoo and many others that have enabled users to interact with our rich dataset in unique ways. One of the methods we’ve used to achieve these integrations is by building a robust, scalable data feeds system, which contains information for over 50 million businesses.

As we allow partners to have their own configurations, information included in these feeds may vary from one partner to another and we did not have an easy way to access the history of data feeds that were delivered to our partners. Having the ability to explore our data feeds is crucial as it allows us to perform quality assurance, pick up on trends and enable us to make future business partnership decisions.

Such insights into our feeds would allow us to answer questions like:

  1. How many of the business listings we sent to partner-A yesterday were merged into other business listings because they were duplicates?
  2. Is it true that a business listing we sent to a partner has actually been closed since last Saturday?
  3. How many business listings in the US don’t have phone numbers specified on them?

As you can imagine, the answers to these questions could be critical for partners consuming this data and could also help us catch bugs in our feed generation process.

This problem statement fascinated me, and I got a chance to implement a solution for this as my intern project this past spring. Since this was a new system, I had a lot of freedom in its implementation, and after researching the various available options and evaluating their tradeoffs, I decided to use ElasticSearch, Kibana, ElastAlert (Yelp’s open source project), Amazon S3 and Amazon EMR.

Architecture

The main objective of this project was to build a visualization system on top of our data feeds. The most important piece was to choose a visualization tool that is easily navigable and has enriched functionality, and there are many visualization tools that can output data in a graphical form. At the beginning of the project, we considered either Kibana, Redash or Grafana, all have their pros and cons. We decided to use Kibana because it provides us with 11 different types of graphs, flexible filters and simple queries. Redash and Grafana are also good visualization tools but are not suitable in our use case. For example, Redash has great support for using arbitrary and complex queries on a desired data set but, in our project, it would have introduced the overhead of maintaining queries for more than 25 partners with 13 different graphs individually (325 queries in total).

By using Kibana, it was a natural choice to use an ElasticSearch cluster as our backend data store for the feeds. Yelp has an internal ElasticSearch cluster template which allowed us to easily spin up our own cluster for this project on AWS.

Before spinning up a new ElasticSearch cluster, we had to estimate how much data we could afford to store. This estimation is important for two reasons – we want to optimize the monetary cost of provisioning AWS machines and we want to optimize Kibana’s efficiency in visualizing the data. We estimated at least 45GB of new data per day, a representative subset of our overall feeds. We decided to go with a one month retention policy along with one replica for each shard. With this setup, we would index roughly 111M documents every day. These calculations led us to choose three dedicated master nodes and 5 data nodes, master nodes would use m3.medium and data nodes would use c4.8xlarge AWS machines.

Now that the architecture and configurations were decided, I went on to implement a two-step workflow to import these data feeds into the newly provisioned ElasticSearch cluster.

image00

The first step takes data feeds generated by the Feeds system (which dumps its output to S3) from S3 and uses our MRJob package to initiate an EMR job. This cycle, called Data Feeds Transformation, aims to retain only necessary information from data feeds. The results, data metrics, will be uploaded back to S3 from EMR. The second step reads these data metrics from S3 and then dumps them to the ElasticSearch cluster.

This two-step process is applied to each individual partner that we generate feeds for.

Problems

As mentioned earlier, the feeds system is highly customizable and gives partners a lot of freedom to select what information they want to receive. Every partner has its own configuration that controls the output of its data feed. These configurations are specified through our own DSL that uses YAML as its representation. Data feeds are generated by applying these partner configurations to our underlying source data. For each configuration, it explicitly states what information to keep and what operations to perform.

For example, a snippet of partner configuration in YAML format can look like:

Given the above configuration, the data feed for the partner will be structured like:

This data feed includes the city and state information for the given partner. Notice that the field changes to ‘state’ from ‘state_code’ inside of the location structure as a ‘_rename’ operation has been applied on that field.

Besides rename operation, there are other operations that can be performed, such as flattening addresses. Some partners choose to have operations on certain fields, whereas some choose not to. Data feeds will have different structures and different fields depending on the selected operations.

As a result, our feed data is not uniform across partners. This makes it hard for us to visualize them on the same scale, or even analyze them for the same patterns. Thus, we had to build a generic workflow that is capable of processing every data feed and bring uniformity to the final data metrics. In order to bring back uniformity, I expanded on the same DSL to define operations that would essentially reverse the effect of the partner-specific transformations, bringing all partner feeds into a uniform format.

Once we generate this unified format, we can easily apply another configuration to filter what we need to get the final data metrics.

Monitoring

Once we got our data metrics into ElasticSearch, we wanted to have a way to monitor our data. We used ElastAlert to notify us if any of the metrics at any given time has spikes or any anomalous patterns that we have defined.

For example, we implemented an alert to check if the number of businesses sent to any partner drops or increases by 10% from one day to the next. This can help us to detect any possible problems with our feed generation service immediately.

A snippet of our rule configuration is below,

In the configuration, we simply specified the type of rule in the ‘type’ field. Each rule has its own required fields to be configured. In our case, ‘spike_height’, ‘spike_type’ and ‘timeframe’ are mandatory in ‘spike’ rule type.

In addition, we configured our rule to trigger an alert when the given condition is met. The ‘alert’ field allows us to specify an alert type. ElastAlert is integrated with two default types of alert – Email and JIRA. Users can also create their own alert types. The snippet above shows a use case of our customized Sensu alert.

If you are interested in using the ElastAlert for your project, a more complete walkthrough of ElastAlert can be found here.

Results

This Feed Metrics system that we built has been online since April of 2015. Below is part of the configured dashboard showing one partner with fake numbers. The upper histogram shows the total number of business counts we sent to a subset of our partners in the last seven days. The lower chart shows the number of businesses that are closed or not closed. With the dashboard, we can easily navigate through all the preconfigured graphs to visualize daily data feeds. In addition, we are able to perform numerous quality assurance operations by using preconfigured filters and queries on data we are delivering.

image00

July 17th, 2015

Glide – How Yelp’s Android App Loads Images

Dynamically loaded images are a cornerstone of many Android applications. At Yelp, images are critical to how we connect consumers with great businesses. As network connections and device hardware have become more powerful, the quantity and quality of images that users have come to expect has continued to increase. Images can easily become the largest consumer of device memory and network data, and handling the downloading and management of this data becomes a daunting task. We explored several solutions to this problem, and ultimately decided that Glide provided a great combination of performance, ease of use, and a robust feature set.

Glide, in the simplest use case, will load images either from a remote server or the local file system, put them into disk and memory based caches, and then load them into views. While it can be used for pretty much all images in an app, Glide is optimized for scrolling lists that contain images as smoothly as possible.

The Object Pool

At the center of Glide’s approach is keeping an object pool data structure for bitmaps. The main goal of an object pool is to increase performance by reducing the number of large object allocations and instead reuse them (for an overview of object pools, check out this Android performance pattern video).

Both the Dalvik virtual machine and ART (for the moment) don’t use a compacting garbage collector, a model in which the GC will run through the heap and move living objects into adjacent memory locations, leaving larger chunks of memory available for future allocations. Because Android doesn’t use this model, the heap can end up in a situation where allocated objects are spread out with only small chunks of available memory in between them. If the application tries to allocate an object that is bigger than the largest contiguous chunk of free memory it will run into an OutOfMemoryError and crash, even if the total amount of free memory was greater than the size of the object.

Using an object pool also helps scrolling performance because reusing bitmaps means fewer objects are created and garbage collected. Garbage collection passes cause “stop the world” events where all threads (including UI) are paused while the collector is executing. During this time, frames can’t be rendered and the UI may stall, which is especially noticeable during scrolling.

scrolling-slight

Using Glide

Glide is simple to get up and running, and it automatically includes bitmap pooling without any special configuration.

DrawableRequestBuilder requestBuilder = Glide.with(context).load(imageUrl);
requestBuilder.into(imageView);

That’s all that is required to begin loading an image. As always with Android, it can be unclear what type of context to pass into the with() method. It is important to note that the type of context passed in affects how much Glide can optimize loading. If an Activity context is passed in, Glide will monitor the activity lifecycle methods and automatically cancel pending requests if it sees the activity is starting to tear down. However, if an Application context is used, you lose out on this optimization.

Optimization Features

Along the same lines, Glide automatically cancels pending requests for images in a ListView if the associated list item is scrolled off the screen. Since most developers take advantage of view recycling in their adapters, it does this by attaching a tag to the ImageView when requesting an image, checking for that tag before loading another image, and canceling the first request if it exists.

Glide provides a few features that give the impression of faster image loading. The first is the ability to prefetch images in a list before they are shown on the screen. It provides a ListPreloader class, which is instantiated with the number of items ahead it should prefetch. That is then passed to the ListView through setOnScrollListener(OnScrollListener). Need to prefetch images outside of a ListView as well? Not a problem, that is supported as well. Using the builder object above, simply call builder.downloadOnly();.

We’ve found the out-of-the-box utility that Glide provides has greatly increased the performance, reliability, and aesthetics of the areas in our Android application that load images. The availability of these additional features and optimizations really allow applications to fine tune their image loading experience into something that creates a delightful experience for the user.

July 2nd, 2015

Yelp API Now Returns Action Links

Savvy developers like you already know that the Yelp API is the best place to get information on local businesses. Today, as a part of our on-going integration with our friends at both Yelp SeatMe and Yelp Eat24, we are excited to announce an additional feature of the API: “Action Links.”

Action Links will allow your users to directly make a reservation or even start a food order for delivery or pickup from wherever you’re displaying Yelp data.

To see this in action (pardon the pun), next time you use the Search endpoint or the Business endpoint, simply pass in the optional “actionlinks=true” parameter. When requested, action links will be returned in fields called reservation_url and eat24_url with the response for businesses that support them. Checkout the handy API documentation for more details.

You can use these links within your application to launch the Yelp Eat24 ordering experience or into the Yelp SeatMe reservation experience.

Ordering Food on Yelp Eat24

Ordering Food on Yelp Eat24

Making A Yelp SeatMe Reservation

Making A Yelp SeatMe Reservation

Go ahead, give it a whirl and let us know what you think. We love hearing from fellow developers, so share your cool creations with us on Twitter via @YelpEngineering or write us at api@yelp.com and stay tuned for even more API goodies coming your way.

June 30th, 2015

New Yelp API Console: Life’s Easier in v2

Yelp’s API allows any developer to build rich user experiences by integrating Yelp’s local business information, reviews, and pictures into their web and mobile applications. We announced in January that we have ambitious plans to make it easier than ever for developers to integrate a local layer into their apps.

Today, we’re excited to present one of these efforts: a brand new API console that allows developers to explore the detail and depth of responses returned by the Yelp API, without having to write a single line of code.

Sign up for a free account, create your API v2 credentials, and check it out on the Yelp Developer Site.

image00

API V1 End of Life

As we continue to invest in Version 2 of the API, we will be discontinuing the previously deprecated Version 1 of the API. All v1 API endpoints will be shut down on July 15, 2015. If you are currently using any v1 API endpoints you can find out how to migrate to v2 endpoints on the Yelp developer site.

Build Your Yelp API Powered App Today

Finally, if this has got you hungry to build something, you can head over to our Github Repo to find examples and libraries to get you underway with your Yelp API powered App.

We love hearing from fellow developers, so share your cool creations with us on Twitter via @YelpEngineering or write us at api@yelp.com and stay tuned for even more API goodies coming your way.

June 12th, 2015

Advanced UITableViews Made Simple: YLTableView

Let’s say you had a great new idea for a Todo app and set out to build an awesome iPhone application. After firing up your copy of Xcode, you’d probably start by creating a UITableView – a critical part of the iOS SDK that allows you to build scrolling views.

At Yelp, we’ve been playing around with different table view architectures for years. After a lot of trial and error, we’ve come up with a framework that easily scales from the simplest list all the way up to our business page. We’re open sourcing it today – check out YLTableView on GitHub!

The Activity Feed and the Business View both make use of table views

The Activity Feed and the Business View both make use of table views

A table view contains a number of sections, each of which can be separated by a small margin. Each section can contain an arbitrary number of cells. We use table views all over the Yelp app, in places like the business page and Activity Feed above. The feed contains a list of ‘items’ representing actions other users have taken – like posting photos or checking-in at a business. In the Activity Feed, each feed item is a single section made up of multiple cells.

A section (green) has multiple cells (blue)

A section (green) has multiple cells (blue)

To implement something like the Activity Feed, you need to tell UITableView about the sections and cells in your table view by implementing a few methods from the UITableViewDataSource protocol:

However, you’re not done yet – if you want to have section headers or make your cells do something when tapped, you’ll need to implement some of the UITableViewDelegate protocol. The two protocols have a lot of overlap and it can get confusing really quickly. When trying to figure all of this out, you might come across UITableViewController, and end up with an architecture that looks like this:

This might work for a while, but the view controller is going to get complex very quickly since it will violate the single responsibility principle. The view controller has to deal with loading content, micromanaging the table view cells, and pushing on new view controllers for cells. It might work for a small table view, but it’ll get complicated very quickly. Enter: YLTableview.

At it’s core, YLTableView works with a series of models. Each cell expects a given type of model and, once it’s given that model, is able to configure itself. Separating out the model helps encourage modularity through your application, and makes it easy to build interfaces out of common components.

To support the models, we’ve created YLTableViewDataSource which implements both UITableViewDelegate and UITableViewDataSource. The two protocols typically need the same data, and we’ve found that combining them helps simplify your code. To use YLTableViewDataSource, make a subclass and implement a few easy methods:

YLTableViewDataSource will then take care of creating the cell, configuring it with your model, and even telling the table view how tall it should be. When using YLTableView, your architecture will look a bit like this:

image03

Unlike the UITableViewController design we looked at earlier, this architecture does a great job of separating out responsibilities. Your view controller loads content – like reviews or businesses – and passes them off to the data source. The data source then turns that content into table view models and displays them as cells. Before telling the view controller to take an action, the data source will translate the cell models back into content to pass to the view controller. The view controller doesn’t need to know anything about the models or cells: you could change your entire UI and it’s underlying data implementation without changing the logic in the view controller.

As we became better and better at building table views, we started to tackle some more complex cells. Take a look at this cell from the Activity Feed:

This cell has a swipeable photos cell. Swiping back and forth will reveal and load in more photos, while tapping on a photo will make it full-screen. As you can imagine, this is pretty tricky to do!

Our first approach to enabling complex cells like this was to have the cell delegate back to the data source, and the data source would delegate up to the view controller. This prevented the photos cell from being modular – every time you wanted to use it, you had to duplicate the entire chain of delegates. We knew there had to be a better way.

After thinking about it for a while, we decided to try creating child view controllers for our table view cells. Child view controllers allow you to add a new view controller to manage a subsection of your view. They behave like real view controllers and can push new view controllers onto the stack.

Normal table views don’t support child view controllers, but we figured out how to do it with YLTableView. Simply have your cell implement YLTableViewCellChildViewController and set the parent view controller property on your data source. Then, instead of having to deal with the mess of delegates, you have a much simpler view controller:

In addition to having a simpler architecture, this cell is now significantly more reusable. We can use it directly on the business page without having to duplicate any code. Taking it a step further, we can even use the PhotosViewController by itself. A little bit of configuration later, and we can now use the same view controller all over the app.

The cell’s child view controller can also be reused by itself.

The cell’s child view controller can also be reused by itself.

YLTableView has helped us simplify our architecture through the app. Using models has allowed us to build up a system of reusable components, making it easy to build interfaces out of common components. Using child view controllers has made this even more powerful, letting us reuse entire view controllers in table view cells. And, as more and more of our app is written with YLTableView, we’ve found that having a consistent architecture gives developers a base level of familiarity with features the haven’t worked on before.

Give YLTableView a shot, and let us know what you think!