Yelp Engineering Blog

Event Listing Classifier: an Intern Story

What projects do interns work on? What are some of the machine learning algorithms we use to solve real world problems? Do interns work in teams solving critical problems, or just do small research tasks? These are some of the questions I frequently get when talking with candidates, so I invited Shengwei L., former intern and now full-time employee, to talk about one of his projects. As you can see, interns at Yelp get to apply the principles they learned in school to make Yelp better for millions of people. They sit alongside full-time members and take responsibility for projects that can span multiple teams and systems. But you don't have to take my word for it, let’s hear from Shengwei!

---

Yelp events are awesome, and we enjoy throwing great events around the world . After events, which are usually free for attendees, Community Managers create a new event listing specifically for their party so that all the participants can write reviews of the event itself instead of posting their reviews of an event onto the business listing that is only for the venue. It would be misleading to see so many reviews on a business listing if they were for a party and not a standard consumer experience with the business.

In Yelp’s system, those event listings are essentially the same as other real business listings, such as a restaurant or hair dresser. Because event listings are mainly about past events, they are not useful for ordinary users. Hence we want to remove them from search after events are over and people have had time to review them. However, no one’s perfect, and we may occasionally lose track of some event listings and they will show up in search results. In order to improve the information/noise ratio for our users, we have implemented a system to automatically detect such listings so that we may exclude as many event listings as possible from search results.

It can be tedious for a human to pick out a few event listings we have missed from numerous real business listings, but computers are good at screening large datasets in a short amount of time. Before tracking dogs start looking for a suspect, they need to sniff some of the suspect’s belongings to learn about the smell. Similarly, a computer needs to learn what an event listing event looks like, as opposed to a real business listing, before it can differentiate them on its own. This is essentially a classification problem. Naive Bayes classifier is broadly used for classification, and it is quite effective for document classification, such as spam email detection. It is based on the Bayes’ theorem , in the form of , where presents different categories, and denotes distinctive features that could possibly appear. Translated into more readable English, that’s . In our case, classes are “business” and “event,” and the features are explained later in this post.

Given an instance with a fixed set of features, the denominator is a constant for different categories. Therefore, is sufficient to categorize the instance based on certain features. This simplification can be exploited to reduce computation effort. Moreover, as a fundamental assumption, features should be independent of each other, and hence is equivalent to . As a consequence, the probability of the category of one instance based on certain features is assessed by , and we take the category for which the score is higher as the instance’s category.

Our wonderful summer intern Te T. implemented a naive Bayes classifier using log-likelihood based on both multinomial and Bernoulli distributions (known as event model). This classifier has been an instrumental tool in the analysis of our massive data, such as check-in tips. I decided to also use it for listing classification, because event listings exhibit some obvious patterns (i.e., features) that ordinary business listings usually do not have. For example, the listing names typically contain certain keywords (such as “yelp,” “elite” and “event”), and the creators are normally Community Managers. The naive Bayes classifier uses these patterns like the tracking dog uses smells: it can now sniff out those event listings created for elite events. After scrutiny of many samples and thanks to suggestions from team members, especially Artem A. , Marty F. , and Zeke K. , we finalized about 10 features, and they worked well in experiments with Bernoulli distribution as the event model.

After the system was deployed, we spotted a number of listings for events that were created in the early days and not correctly tracked. In the meantime, we regularly check all newly created business listings to see if we have missed classifying any listings for events. We are not blindly marking business listings relying on the automatic classification. Human verification is also involved to make sure we do not mistakenly exclude real business listings.

Going forward, we will continue to monitor and analyze the classification result, particularly false positive and false negative cases, trying to find out more features we may exploit. In addition, since we utilize Bernoulli distribution as the event model for features, there are thresholds to determine true or false value of features based on experiments with samples. We may also tweak the thresholds here and there according to analysis of the classification result.

This was just one of my intern projects. Without support from my team members and help from other groups, I could hardly figure out all the parts I would need to accomplish this. I really enjoyed the progress, interacting with experienced industry veterans and other awesome interns. It is a great approach to acquire hands-on experience and transfer the knowledge learned in class into a working piece in the real world.

mrjob Sprint at Pycon 2013!

Sudarshan G., Yelp's current leading engineer behind mrjob, writes this week about an exciting opportunity to contribute to one of our most popular open source libraries: mrjob.

---

We are happy to announce that we are running a mrjob sprint at Pycon 2013. mrjob is an open source Python framework for running Python scripts to process large amounts of data either on your own Hadoop cluster or in AWS using EMR. The mrjob sprint is targeted at both developers who are familiar with mrjob as well as new users who want to come up to speed on it. The sprints are being held on March 18th and March 19th between 10 am and 6 pm at:

Hyatt Regency Santa Clara
5101 Great America Parkway
Santa Clara, California USA 95054

Most of the maintainers of mrjob will be available on both the days to help get new users and developers up to speed. The sprint is free to attend and there is no requirement in terms of registering for Pycon to be able to attend this sprint. We strongly encourage users with all levels of expertise with mrjob to register on the PyCon website and attend this sprint! An initial lists of tickets to be tackled at the sprint are tracked on Github.

The mrjob project has been developed at Yelp for over 3 years now. mrjob was built to provide a way for Yelp engineers to run log processing jobs on the hadoop cluster at Yelp while retaining all the benefits of working in Python and reusing the Yelp code base. Yelp released mrjob as an open source project in Oct 2010. The open source release came with support for Amazon's EMR framework. Yelp has since retired its local hadoop cluster and now uses mrjob exclusively with EMR to power all the large data processing batches. Over 200 batches at Yelp are powered by mrjob.

mrjob is under active development and is at version 0.3.5 right now. In the last 2 years mrjob has been rapidly improved with features like support for reusing job flows appearing in 0.2.7 and spot instances appearing in 0.3.2. We are about to release mrjob 0.4 which will support jobs written in different languages such as Ruby and Javascript as well as a bunch of other cool features.

Sign up and we'll see you there!

March Events at Yelp

Yelp’s weekly “Learning Group” is one of our favorite engineering events. People come for the free food but stay for the hour-long presentation, usually given by a fellow engineer. Past topics, both technical and non-technical, have included everything from cinematography to typography to steganography, and the presentations are lovingly put together. 

Since we've found these presentations fascinating we've decided to share them! This first one is a Learning Group about learning Chinese, by Mark W. He's spent several years learning to speak Chinese, and in this presentation he introduces the Chinese language along with some tools and techniques for mastering a new skill.

 

(Learning Chinese by Mark Wilson- Slides)

In addition to hosting our own events or meetups, we like to travel to stay connected with the tech community outside of San Francisco. Last weekend, we attended a Hacker Fair in the South Bay hosted by Hacker Dojo. A few of our representatives were able to attend and met several individuals who shared with us their current projects and inventions. The best part? Attendees of the fair rode down to the South Bay and back via party bus, courtesy of one of Hacker Dojo’s sponsors! Work hard, play harder. 

This upcoming weekend, we’re sending Grace, one of our recruiters, to the Tech Career Expo in Austin, Texas. Lucky girl! The Tech Career Expo is hosted during the same weekend as SXSW but is open to all individuals interested in learning more about the tech industry. Come by our booth and grab some food, drinks and great giveaways! Our booth will be located on the corner of 5th Street and Trinity Street, 1 block away from the Austin Convention Center. 

Back at home, we have another great Designers + Geeks event to look forward to. This month, things are getting steamy with Ethan Imboden, the founder of Jimmyjane, a leader in the pleasure product industry. Ethan will be joining us on March 14 to reveal how he went about redesigning and imagining this taboo industry. Ethan has over 10 years of brand and product innovation experience and wants to let you in on his secrets.

Upcoming March events:

• Thursday, March 14, 2013- 7:00PM-  Sex, Design & Disruption (Designers + Geeks)
• Wednesday, March 20, 2013- TBD - (San Francisco & Silicon Valley Web Performance Group)

February Events at Yelp!

Two weekends ago, we participated in and sponsored the 2013 O4U (Out for Undergraduate Technology) Conference in Palo Alto. This conference is hosted in different cities around the nation and strives to inspire lesbian, gay, bisexual and transgender undergraduate students to be more involved in the tech industry. Students were able to spend some quality time with one of our engineers and recruiters and were also able to participate in panels, workshops and demos.

Meetups

SFHTML5 - Introduction to the Backbone.js, Ember.js, and AngularJS Frameworks

 

Sidney, Anthony and Miško brought in their expertise to tackle everyday questions about Backbone.js, Ember.js and AngularJS. While Backbone.js helps to create more reusable code by exposing MVC design patterns in a lightweight library, Ember.js  provides developers  a full fledged framework for building better browser-based apps. Finally, AngularJS simplifies web app development by providing new HTML primitives to bind data to UI elements, reacting to changes automatically.

If you missed our meetup events last month, don’t fret! You’ll have plenty of chances to come hang out with us again this month!

Our monthly partner, Designers + Geeks, always impresses us with their speakers, but this month they’re really stepping it up! This time they’re bringing in Dan Whaley, the creator of the world’s first online travel reservation system which currently brings in over $9 billion a year. Quite impressive.

Upcoming February Events

• Thursday, February 21, 2013 -6:30PM - Annotating Knowledge (Designers + Geeks)
• Monday, February 25, 2013- 6:00PM - Speaker Series: Eric Rodenbeck of Stamen and Santiago Ortiz (Data Visualization Group in Bay Area)
• Tuesday, February 26, 2013- 6:00PM - Hack Night!  (Women Who Code)
• Thursday, February 28, 2013- 6:00PM - Dart with Seth Ladd and John McCutchan  (SFHTML5)

Mission: Beyond The Mobile Browser

This post comes from Arnaud B., an engineer who likes to look after Yelp's mobile site, deal with mobile browsers and curse at them. Here he breaks down two JavaScript tricks used at Yelp to deliver a top-notch mobile site experience. Read on for the full explanation!

---

Last year we published an article explaining the reasoning behind the birth of our mobile site, Mission: Mobile Makeover. Since then, we’ve been hard at work, dedicated to improving this new Yelp property and transform it into an app-like experience. Ideally? Forget about your browser, and keep on Yelpin’!

Today, we’ll focus on some of the work we’ve accomplished during the past year or so. We’ll talk about two technical challenges faced lately: how to boost interaction speed on mobile, and how to make the most out of the screen sitting in your pocket.

Boosting Interaction Speed

Clicks are a core concept on the web because it drives most form interactions and navigation. To support double clicking, mobile browsers have no choice but to wait a short amount of time (typically, around 300ms) after your finger has lifted off the screen to fire a “click” event.

Since Yelp’s mobile site doesn’t require double clicks this behavior is actually detrimental to us: it makes interactions and navigation feel slower. That’s why we decided to boost interaction speed with a global shim that mitigates the delay that a user typically experiences between the moment when her finger lifts off the screen and the moment when a “click” event fires in JavaScript.

The general principle behind the change is:

If a user triggers a “touchstart” event at point A, followed by a “touchend” event at point B and if the two points are not far away from each other (both in distance and time), then it’s safe to assume we have a click intent. Otherwise, the intent was a drag, or something entirely different...but definitely not a click!

To accomplish this, we listen to “touchstart”, “touchmove” and “touchend” events. If we spot a click intent when a “touchend” event is fired, we immediately trigger a “click” event via JavaScript and prevent the default action. As a result, the “click” event fires ~300ms earlier than it would otherwise have.

If we fail to spot a click intent (maybe “touchstart” and “touchend” events were too far away from each other in distance, which means we have a dragging movement), we simply let the browser handle things for us.



Quick notes about this:

1. This idea is not new. Google has a really good article about it, mobile HTML5Boilerplate has fast buttons in its JavaScript helpers, and the Financial times app recently open sourced their solution for faster clicks on touch-enabled devices.
2. Not all devices have this problem. In our experiments, we noticed that the latest Android devices handle touch events and clicks much faster than they used to.
3. If you want to implement fast buttons on your own, we highly suggest that you develop in Chrome Canary. This browser lets you emulate touch events, and also override a bunch of browser properties like the browser’s UA, width, height, etc. For example, here’s a screenshot of the setup used at Yelp to iterate on faster clicks.

Above is a picture of search suggestions on Yelp’s mobile site. This is the quintessential example of a feature that needs fast interactions to exist. With delayed clicks (on form inputs and suggestions) the experience would feel sloppy and awkward. With faster clicks, we’re getting closer to an app-like experience on Yelp’s mobile site.

Making The Most Out Of Small Screens

Let’s take a look at our search map page:



This page has several fixed height parts (3 action bars, search form) and one dynamic height part (the map).
Since our mobile site is going to be displayed on a small screen, we have to take advantage of every pixel we can get. That yields two independent goals:

1. Get as many available pixels as we can 
2. Use 100% of the available space to draw content

The first goal is difficult because the available space to display your Web app is typically dependent on the OS. When working inside a browser, there’s only one thing you can kick out of the way: the browser chrome (containing the URL bar). Getting rid of it is pretty simple: we have to tell the browser to scroll to the top of the page. It’s a well-known trick, and the code fits on one line:

window.scrollTo(0, 0);

There is a catch though: Android won’t work with the code above, and requires a slightly different version:

// For Android
window.scrollTo(0, 1);

Now comes in the hard part: we have to draw our layout (containing both fixed and flexible height parts) on the available screen. The only flexible part is the map so it’s going to fill whatever space we have left after placing the fixed height elements:

var mapHeight = screenHeight - totalFixedHeightsFromOtherElements;

We know the value of  totalFixedHeightFromOtherElements. However we don’t know screenHeight. How tall is the screen exactly? How can we do that in JavaScript? Well, there are a lot of different APIs out there. See for yourself:

• window.innerHeight
• window.outerHeight
• screen.height
• screeen.availHeight
• pageYOffset
• document.body.clientHeight
• document.body.offsetHeight
• document.documentElement.offsetHeight
• document.documentElement.clientHeight

Given the plethora of options available we took several devices, did some testing and iterated to come up with a solution. The code we are using right now looks something like this:

var width = screen.width;
var height = screen.height;
if (screen.width > window.innerWidth
  height = document.documentElement.clientHeight;
  width = document.documentElement.clientWidth;
}

This code is not pretty to look at but it solves the problem at hand — client-side screen size detection — pretty well for us. Let’s see why.

First, we found that screen.width/height was the most consistent API out there considering the devices we support (iOS and Android > 2.2). If you want to scare yourself, please have a look at the massive testing table James Pearce assembled! You will get a sense of how fragmented mobile devices/browsers are these days.

The code snippet above solves another problem: some devices report screen.width/height in physical pixels where others report it in CSS pixels. This matters for devices with double density display. For instance the Nexus 7 reports a screen.height of 1280px.

However, document.documentElement.clientWidth always reports a value in CSS pixels, so we fall back to that API if we spot that screen.width/height is giving us a bigger value than expected.

Why don’t we use document.documentElement.clientWidth all the time then? You guessed it already: it’s not accurate enough, for instance on iOS (it reports 356px instead of the expected 480px because of the browser and system bars).

As you can see this complex problem doesn’t have a definite answer. We’re still iterating on our solution at the time of writing.

By the way you’ll probably find surprising things if you fire up real devices. For instance did you know Nexus7’s reported devicePixelRatio was 1.3250000476837158? Yup, that’s insane.

Going forward

The mobile site is still a young Yelp property. Barely out of its infancy, actually. Since its introduction we’ve been busy working on delivering the best experience for small screens, trying to make you forget you are in a browser.

Fostering faster touch interaction, giving better feedback, making the most out of every available pixel, taking advantage of new HTML5 APIs... this is barely scraping the surface of the ideas we have to improve our mobile site going forward.

We are truly excited about what’s ahead. If you are too, you should think about joining us —we’re hiring ;)

Bringing Health Inspection Scores to Yelp

Our post today is by Will L., an engineering intern on one of Yelp’s backend teams this past fall. Will walks us through the challenges of bringing restaurant health inspection scores to Yelp, a feature we announced today at the United States Conference of Mayors in Washington, DC.

---

As you may have seen on our official blog, we are very excited about our initial release of the Local Inspection Value-Entry Specification (LIVES). LIVES is an open data standard crafted by Yelp in partnership with the cities of San Francisco and New York to allow municipalities to publish restaurant health inspection information in a machine-readable format.

In this post, we want to take you behind the scenes and give you an overview of all the steps and actions that have happened from getting the standard off the ground to having all of the health inspection information showing up on Yelp.

The Local Inspection Value-Entry Specification was first drafted by Yelp engineer John Boiles (also known at Yelp for his Kinect hacking and the legendary KegMate) in mid-June 2012 and was followed by several collaborative revisions with key members of city health departments. Individuals within the cities of San Francisco, New York, and Philadelphia were instrumental to the process of refining the standard with their domain expertise and feedback. On January 9th, 2013, the latest version (1.0) was published.

A LIVES feed contains several comma-separated value (CSV) files to encapsulate the feed data in easy-to-read textual representation: businesses (businesses.csv), inspections (inspections.csv), related violations (violations.csv), score mappings for municipality-specific conventions (legend.csv), and finally data about the feed itself (feed_info.csv). Below is a snapshot of a portion of businesses.csv taken from the LIVES feed provided by the City of San Francisco:

Once the specification began shaping up at the beginning of October, a team of engineers at Yelp started to build a system to process and display LIVES data on our site. The first step was to come up with a scalable and maintainable system based on the requirements and constraints of the standard. While the LIVES standard is currently in use by two cities, Yelp is calling upon municipalities all across the US to share their health inspection data. As such, scalability played a critical role in our design process.

One of the more interesting and challenging aspects of the project centered around matching up a city’s record for a business to its equivalent on Yelp. While this may sound simple at first, it proves technically challenging when you realize cities are more interested in the legal representation of a business whereas Yelp focuses on what you would see if you were standing in front of the business on the street. For example, Starbucks may register itself as “Starbucks Coffee Company” with the City of San Francisco but will show up as just “Starbucks” on Yelp. Similar problems arise with addresses and phone numbers, all of which are attributes we use to help pinpoint the right business on Yelp (e.g., a chain might use a central number for registrations but have its individual numbers on Yelp).

While matching a set of data to a business is something we do routinely here at Yelp (after all, a search on Yelp is a very similar problem), the stakes for this project were much higher, especially in regards to false positives when matching. Just imagine how a 5-star restaurant with a perfect health record would feel if we incorrectly associated a failing inspection with their profile on Yelp.

To fine tune our matching, we ran several sample data sets from San Francisco and New York City through our tools and evaluated our results, paying particular attention to false negatives and false positives. Through a combination of normalization of the raw data from the municipalities and tweaks to how we weigh each piece of data (name, address, and phone number), we were able to dramatically minimize the number of false positives. Matching business records is never a completed project, however, so we’re constantly collecting metrics on how it’s performing with new data sets and tweaking its algorithms and weights appropriately.

Once we had all of the various implementation pieces glued together, the last step was to implement a rollout strategy. At Yelp, we’ve developed several tools to assist in this process to limit the exposure of a new feature. We’re able to release a feature to our internal users only, expose it to only a certain portion of public traffic, or whitelist the feature for certain businesses only. By combining all of these, we’re able to iterate and deploy features quickly all while keeping risks low.

We still have a lot of work to do with LIVES. Besides continuing our gradual rollout of the feature, our priority is to advocate for the adoption of the standard with municipalities so that more health inspection data is available publicly and can be displayed on Yelp. Since LIVES is an open standard, this not only benefits consumers wondering if that food stand on the corner of the street is a good choice; it also allows other organizations, such as research institutions, to use this data to spot trends and perhaps prevent future foodborne illness outbreaks. We’re equally interested in this data and plan on looking at how the average scores evolve across cities as we make this data more readily available to consumers like you. LIVES was one of Yelp’s first forays into developing an open standard. We’re definitely hooked and look forward to working with more local governments in the future to iterate on this standard and help share the wealth of information they have on local businesses.

More January Events at Yelp

We host several external meetup groups every month, but sometimes, they want us to present, too! SF Hadoop Users invited Sudarshan Gaikaiwair, one of our Yelptastic engineers to speak about mrjob, our MapReduce framework.

mrjob is an open source project that runs on top of Hadoop. While mrjob can also be used with your own hadoop cluster, it is great for getting started very quickly with Amazon's EMR offering. mrjob allows programmers with very little Hadoop or EMR experience to quickly write scripts that can process terabytes of logs on hundreds of machines.

This talk covers the basics of MapReduce and presents code examples of how to write mrjob programs based on the problems Yelp faces.

Tech Talk: SF Hadoop Users Meetup with Sudarshan G. from Yelp Studios on Vimeo.

Coming up in the next few days will be several exciting meetups, including a Python developer “speed dating” event! Don’t worry, it’s strictly professional, courtesy of HackerX. We work closely with a variety of meetup groups all with the intention of providing our guests with great learning experiences. Stay tuned to learn about TIZEN, Revitalizing Unloved Devices, Data Visualization and much more!

• Wednesday, January 16, 2013 -6:30PM- HackerX- Python Developers (HackerX)

• Thursday, January 17, 2013 -6:30PM- Revitalizing Unloved Devices (Designers & Geeks)

• Thursday, January 24, 2013- 6:30PM- Graphics in HTML (Girls Develop It SF)

• Tuesday, January 29, 2013- 6:00PM- Data Visualization (Data Visualization)

• Thursday, January 31, 2013- 6:00PM- Introduction to the Backbone.js, Ember.js, and AngularJS Frameworks (SFHTML5)

January Events at Yelp!

Want to know what widows, rivers, and pig bristles have to do with typography?  Listen to Carolina de Bartolo (@carodebartolo) explain why Typography Matters in the video below!  Designers + Geeks is an engaging meetup that covers everything from beautiful architecture to mobile development.  At an event hosted at Yelp in November, Carolina covered topics ranging from the history of typography to practical advice on choosing fonts for your project.

 

Designers + Geeks is just one of the many community events that we host in our spacious 10th floor lounge.  But fair warning: once you attend one event here, with our view of MOMA, two kegs, drinks, and snacks,  you may end up wanting to attend all our events!  Here’s what’s in the pipeline for the first half of January:

• Tuesday, January 8, 2013- 5:30PM  - Learn about Java EE 7 Directly From the Source (Marakana)
• Wednesday, January 9, 2013- 6:00PM - mrjob & Hadoop by Yelp’s own Sudarshan G (SF Hadoop Users)
• Tuesday, January 15, 2013 - Intro on Samsung/Intel/HTC Open Source Tizen Project (co-coordinated by Game and App Devs, GamesJS, and Mobile App Devs)
• Thursday, January 17th -6:30PM- Revitalizing Unloved Devices (Designers + Geeks)

See you at Yelp!

Yelp re:Invents!

Last month, Yelp was asked to talk about our experiences with Amazon’s Web Services at their re:Invent conference in Las Vegas.  I was delighted to be able to attend, speak, and learn from other speakers.  All videos from the conference have been published, and below I’ve highlighted the sections specifically about Yelp.

Big Data with Elastic MapReduce

 

The majority of our AWS usage is Simple Storage Service (S3) and Elastic MapReduce (EMR).  We use these technologies because we want every engineer to be extremely effective, to be able to command a cluster of machines that would normally take another entire team just to manage.  We want our engineers asking and answering questions like:

• “What category did the customer want when she searched for ‘Pool’?”
• “When will our mobile traffic eclipse our web traffic?”
• “Was this review written by someone with firsthand experience?”

At Yelp, whether you’re a senior engineer or an intern, if you want to test the next great data driven product, you don’t have to cajole another team for resources, or sit around waiting for your job to be scheduled.  Deploying is the same way: no need to worry about interrupting someone else’s batch job, or filling out TPS reports estimating the time you need on the production cluster.  Just ship the code; the boxes will be available when you need them.

Yelp AWS Optimization 

Make no mistake: optimizing for developer time can mean trading-off potential cost savings.  While we believe the trade-off is worth it, that doesn’t mean we ignore our costs!  Two of the specific ways we save money on EMR are:

• Re-use of job flows
• Buying reserved instances

We try to find ways of saving money that are invisible to other developers, and base improvements on how developers want to use the resources available.  Contrast this with the philosophy of making every developer justify and micro-optimize their costs.  We build tools to multiply the effectiveness of fellow engineers, instead of having policies that divide their attention between business issues and implementation details.  By sharing tools such as mrjob and EMRio we’re not only letting Yelp developers better focus on business problems, hopefully we’re letting other companies do the same.

I hope you enjoy the videos, and Happy Holidays!

Building and Testing Yelp Mobile

In this post, Steven S. from Yelp's mobile team gives us the lowdown on what tools we use to make a rich, reliable mobile experience.

---

The mobile team at Yelp always strives to bring the best possible experience to your mobile device, but what does it take to accomplish this? For Yelp's iOS app, in addition to tools used by many iOS developers — Xcode for a development environment, Git for version control — we have found other useful tools and libraries to improve our development experience. If you've ever appreciated our app and wondered what goes into its development, we're happy to shed some light on what tools from the iOS development community we make use of, and we even have some of our own to share!

The foundation upon which the Yelp app is built is YelpKit. YelpKit is an Objective-C library that provides reusable controls and helpful extensions for Apple's UIKit framework. YelpKit can assist with tasks from making requests and caching images to using pull-to-refresh. In fact, we found YelpKit to be so useful that we open-sourced it so we could use it in our own personal projects. Check out YelpKit on GitHub for instructions on using YelpKit and to see what features would come in handy for you!

Of course, with all the work you put into building your app, you want to make sure it works! Automated testing is an important tool used by many developers, and the iOS developers at Yelp are no exception. We use GHUnit for unit testing and also make heavy use of its view testing capabilities; tests in GHUnit are run on an iOS device or emulator, and the resulting view of your app can be saved to an image file and compared against previous runs. GHUnit's view tests prove invaluable for catching inadvertent changes, and can even help detect minute changes that occur from updates to the iOS SDK.

As useful as automated testing is, getting your app into the hands of as many people as possible is still one of the surest ways to find bugs. This is where TestFlight comes in: TestFlight is a free service that makes it easy to distribute your app to testers. Testers must join TestFlight and be included in your app's ad hoc provisioning profile. When you want to release a new version of your app for testing, you just upload the compiled IPA file and TestFlight will notify your testers that it is available for download! Testers need only click a link in this email to begin installing the app. You can choose to which testers to distribute the file, and TestFlight provides analytics indicating, for example, which testers have opened the email or if any have had problems installing your app. With our sharp group of fellow iPhone-wielding engineers eager to try out new features, this convenience offered by TestFlight allows Yelp to rapidly distribute and receive feedback on beta builds of apps.

With the help of GHUnit and TestFlight, we manage to keep the Yelp app in good shape! Inevitably, though, some bugs can slip through, so we use Crashlytics to help us get to the bottom of any crashes that crop up once the app is released to the App Store. Crashlytics is a crash reporting solution that supplements the crash logs available through iTunes Connect. After configuring Crashlytics, the debug symbols (dSYM) for your app are uploaded to Crashlytics during the build; if your app crashes, a crash log is sent to the Crashlytics servers and symbolicated. Crashes are ordered by their prevalence in Crashlytics' web interface, so at a glance you can tell what issues are most pressing and then start digging through the stack trace in order to keep your app functional and your users happy.

In the lively iOS development community, you don't have to look very far to find all sorts of useful tools and libraries. In showing you those that helped Yelp reach its success, we hope you're excited about these tools, and we can't wait to see what you can make using them!