CSP is Awesome

Content Security Policy isn’t new, but it is so powerful that it still feels like the new hotness. The ability to add a header to HTTP responses that tightens user-agent security rules and reports on violations is really powerful. Don’t want to load scripts from third party domain? Set a CSP and don’t.  Trouble with mixed content warnings on your HTTPS domain? Set a CSP and let it warn you when users are seeing mixed content. Realistically, adding new security controls to a website and a codebase as large as Yelp needs to be a gradual process. If we apply the new controls all at once, we’ll end up breaking our site in unexpected ways and that’s just not cool. Fortunately, CSP includes a reporting feature - a “lemme know what would happen, but don’t actually do it” mode. By using CSP reporting, Yelp is able to find and fix problems related to new CSP controls before they break our site.

Reading Sample CSP Report

CSP reports are JSON documents POSTed from a user’s browser to Yelp. An example report might look like:

{
   "csp-report": {
     "document_uri": "https://biz.yelp.com/foo",
     "blocked_uri": "http://www.cooladvertisement.bro/hmm?x=asdfnone",
     "referrer": "https://biz.yelp.com",
     "source_file": "https://biz.yelp.com/foo",
     "violated_directive": "script-src https:",
     "original_policy": "report-uri https://biz.yelp.com/csp_report; default-src https:; script-src https:; style-src https:"
   }
}

This report says, “I went to https://biz.yelp.com/foo but it loaded some stuff from cooladvertisement.bro over HTTP and I showed a mix content warning.” Looks like www.cooladvertisement.bro needs to get loaded over HTTPS and then all will be good.

Making Sense of CSP Reports @ Scale

It’s easy to read a single CSP report but what if you’re getting thousands of reports a minute? At that point you need to use some smart tools and work with the data to make sense of everything coming in. We wanted to reduce noise as much as possible so had to take a few steps to do that.

Get rid of malformed or malicious reports

Not all reports are created equally.  Some are missing required fields and some aren’t even JSON. If you have an endpoint on your website where users can POST arbitrary data, there will  be a lot of noise mixed with the signal.  The first thing we do is discard any reports that aren’t well formed JSON and don’t contain the necessary keys.

Massage the reports to make them easier to aggregate

It was helpful to group similar reports and apply the Pareto principle to guide our efforts at addressing CSP reports. We take any URI in the report and chop it down to it’s domain, getting rid of the uniqueness of nonces, query params, and unique IDs, making it easier to group

{
  "csp-report": {
    "document_uri": "https://biz.yelp.com/foo",
    "document_uri_domain": "biz.yelp.com"
    "blocked_uri": "http://www.cooladvertisement.bro/hmm?x=asdfnone",
    "blocked_uri_domain": "www.cooladvertisement.bro",
    ...
  }
}

Discard unhelpful reports

Surprisingly, you’ll see a whole lot of stuff that’s not really about your website when you start collecting reports. We found some good rules to discard the unhelpful data.

blocked_uri and source_file must start with http

We see loads of reports with browser specific URI schemes, stuff related to extensions or the inner workings of a browser like chromeinvoke:// or safari-extension://.  Since we can’t fix these, we ignore them. source_file is an optional field in a CSP report, so we apply this rule to source_file only when it has a value.

document_uri must match the subdomain the report was sent to

If we’re looking at CSP reports that were sent to biz.yelp.com then we’re only interested in reports about documents on biz.yelp.com.  All sorts of strange proxy services or client side ad injectors will land up serving a modified copy of your page and generate reports for things you can’t fix.

Retain some useful data

We don’t want to lose useful data that came in as part of the POST request, so we tack it onto the report. Info like user-agent can be super helpful in tracking down a “Oh… that’s only happening on the iPhone” issue.

{
  "csp-report": {
    "document_uri": "https://biz.yelp.com/foo",
    "document_uri_domain": "biz.yelp.com"
    ...
  },
  "request_metadata": {
    "server\_time": 1408481038,
    "yelp\_site": "biz",
    "user\_agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 7\_1\_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53",
    "remote\_ip": "127.0.0.1"
  }
}

Throw this all in a JSON log

Once we’ve got a nice, well formed JSON report with some helpful extras, we throw it into a log. Logs aggregate from our various datacenters and make themselves available as a stream for analysis tools.

Visualize, monitor, and alert for the win

The Yelp security team is a huge fan of E lasticsearch/ L ogstash/ K ibana. Like we do with pretty much any log, we throw these CSP reports into our ELK cluster and visualize the results.

This image shows a rapid decrease in incoming CSP report volume after fixing a page that caused mixed content warnings

From there it’s easy for our engineers to view trends, drill into specific reports, and make sense of reports at scale. We’re also adding monitoring and alerting to the reports in our Elasticsearch cluster so it can let us know if report volumes rise or new issues crop up.

Give it a try

We’re making sense of CSP reports at scale and that’s super useful in monitoring and increasing web application security. We’d love to hear from you about how you’re using CSP. Let us know at opensource@yelp.com.

Back to blog