JS Fest 2019. Andrew Betts. Headers for hackers

Headers for hackers
Uncovering the Web’s superpowers
Andrew Betts, Fastly

$ telnet bank.example.com 80
GET /statement HTTP/1.1
Host: bank.example.com
HTTP/1.1 200 OK
Date: Tue, 27 Feb 2018 13:28:47 GMT
Content-Type: text/html
Content-Length: 34882
<html>
...

HTTP/1.1 200 OK
Cache-Control: private, max-age=3600
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
Last-Modified: Fri, 02 Feb 2018 07:21:05 GMT
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubdomains;
Content-Security-Policy: default-src 'self'; report-uri https://csp.example.com/
Expect-CT: max-age=86400, enforce, report-uri="https://ect.example.com/"
Alt-Svc: h2="new.example.com:443"; ma=600
Link: /script/bundle.rev-983c15.js>;rel=preload;as=script;charset=UTF-8;
Accept-CH: DPR, Width, Viewport-Width
Feature-Policy: vibrate 'none'; geolocation 'none'; unsized-media ‘none’
<html>

• Alt-Svc
• Feature-Policy
• Origin-Policy
• Clear-Site-Data
New HTTP response headers
• Integrity
• Signature
• Accept-CH
• Server-Timing

ClearClear-Site-Data
Clears the browser cache
https://www.w3.org/TR/clear-site-data/
Domains sending
0.01%
Standardised in
2017

Clear-Site-Data: "cache", "cookies", "storage", "executionContexts"
Delete an entire origin’s storage

https://calendar.perfplanet.com/2017/clearing-cache-in-the-browser/

• Alexa top 1,000,000 websites
• Around 500,000 pages analysed
• Over 50 million requests per run
• Captures full request and
response data, timing metrics etc.
• Runs using WebPageTest
• Makes raw result data available in
BigQuery

https://bigquery.cloud.google.com/savedquery/598614557294:2463981d0f444b6ba6c1a8c376079b90

P3PP3P
Machine readable privacy policy
https://www.w3.org/TR/P3P11/
Domains sending
9.8%
Commonly set to
cp="this is not a p3p policy"
Standardised in
2002

• Intended as a declaration of privacy policy
• Too hard for users to understand/use
• Only ever implemented by Internet Explorer, to gate access to third party
cookies in IFRAMEs.
– ... but not validated
• Commonly set to “this is not a P3P policy” which satisfies the check
Platform for Privacy Preferences Project?
https://bigquery.cloud.google.com/savedquery/598614557294:9c69db8c47f84c4d9a4b57668ac8ba58

ExpiExpires
Sets expiry time for local caching
https://tools.ietf.org/html/rfc7234#section-5.3
Domains sending
78%
Standardised in
1997

Expires: Thu, 01 Dec 1994 16:00:00 GMT

“Note: if a response includes a Cache-Control
field with the max-age directive, a recipient
MUST ignore the Expires field.”

Domains sending Expires
78%
Domains sending Expires and
Cache-Control with max-age:
64%
https://bigquery.cloud.google.com/savedquery/598614557294:98e14323d29740678fe1b3012c9186db

Expires: Thu, 01 Dec 1994 16:00:00 GMT
Cache-Control: private, no-store, no-cache, no-transform, must-revalidate,
max-age=0, post-check=0, pre-check=0
Pragma: no-cache

Cache-Control: private, no-store

X-CacX-Cache
Records whether the page came from cache
upstream (probably)
Domains sending
13%
Standardised in
Never
Commonly set to
HIT

x-cache x-aspnet-version x-varnish x-request-id
x-cache-hits x-cacheable x-aspnetmvc-version
x-runtime x-generator x-drupal-cache host
referer x-served-by x-proxy-cache server
x-type x-cache-group x-cache-status
x-accel-version
Meaningless to the browser
All headers shown above are returned by at least 5,000 domains in the HTTP Archive dataset
https://bigquery.cloud.google.com/savedquery/598614557294:2463981d0f444b6ba6c1a8c376079b90

ExpressJS vs Varnish/edge cache
if (!req.http.Reveal-Debug && !req.http.Cookie:RevealDebug) {
unset resp.http.Server;
unset resp.http.X-Powered-By;
unset resp.http.X-Cache;
// ... etc
}
Works in Fastly and Varnish cache
app.disable('x-powered-by');

X-FraX-Frame-Options
Prohibits third party framing of your site
https://tools.ietf.org/html/rfc7034
Domains sending
22.7%
Standardised in
2013
Almost always set to
sameorigin

Stop anyone from framing your site:
Content-Security-Policy: frame-ancestors 'self'
But... equivalent to...
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors

ViaVia
Lists proxies through which the request passes
https://httpwg.org/specs/rfc7230.html#header.via
Domains sending
8.7%
Standardised in
1997

Proxy behaviour
User Fastly Server
Via: 1.1 varnish
Via: 1.1 varnish

Proxy forwarding loops
User Fastly Another CDN™

ConteContent-Security-
Policy
Restricts network access from a page
https://w3c.github.io/webappsec-csp/
Domains sending
2.9%
Standardised in
2016
Average length
593 bytes

Content-Security-Policy: default-src 'self'; img-src *; media-src media1.com
media2.com; script-src userscripts.example.com
Simple CSP example:
By default, only allow access to the same origin as the page. But allow images to come from
anywhere. Media and scripts from a whitelist of specific origins.

Content-Security-Policy: default-src 'self'; font-src 'self' data: http://*.gstatic.com https://*.gstatic.com https://*.symantec.com https://*.criteo.com http://*.resultspage.com
https://*.resultspage.com; child-src 'self' https://www.google.com/ads/ https://*.listrak.com https://www.google.pl/ads/ https://www.google.ua/ads/ http://*.google.com.ua/ads/
https://www.youtube.com/ https://www.googleadservices.com/ https://googleads.g.doubleclick.net/pagead/ https://*.symantec.com https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com
http://*.resultspage.com https://*.resultspage.com; object-src 'self' http://*.verisign.com http://*.iesnare.com https://*.verisign.com https://*.iesnare.com https://*.symantec.com
https://*.zmags.com; img-src 'self' data: https://ssl.emailcli.com *.sli-spark.com https://*.veinteractive.com http://*.bazaarvoice.com/ http://blog.natchezss.com/ https://blog.natchezss.com/
http://*.avmws.com http://*.websecurity.norton.com http://*.google.com.ua http://*.natchezss.com https://www.pepperjamnetwork.com/ http://*.gstatic.com http://*.ywxi.net
https://www.google.com/ads/ http://s3.amazonaws.com/product.reflektion.com/ http://*.reflektion.com/ https://stats.g.doubleclick.net/ https://d26opx5dl8t69i.cloudfront.net/
http://*.google-analytics.com/ https://seal.networksolutions.com/ http://*.listrakbi.com/ http://*.chartbeat.net/ http://*.scanalert.com/ http://*.avantlink.com https://*.bazaarvoice.com/
https://*.listrakbi.com/ https://*.chartbeat.net/ https://*.google-analytics.com/ https://*.reflektion.com/ https://*.bazaarvoice.com/ https://*.google.com.ua https://*.listrakbi.com/
https://*.chartbeat.net/ https://*.scanalert.com/ https://*.avantlink.com https://*.amazonaws.com/ https://*.scanalert.com/ https://*.norton.com https://*.ywxi.net/ https://*.cloudfront.net/
https://*.zmags.com/ https://*.adnxs.com http://*.adnxs.com https://*.symantec.com https://*.r1cdn.com https://ad.doubleclick.net https://go.flx1.com https://*.g.doubleclick.net
https://*.optimizely.com https://*.yahoo.com https://*.sitescout.com https://*.1rx.io https://*.tubemogul.com https://*.simpli.fi https://*.ipredictive.com https://*.wtp101.com
https://*.pubmatic.com https://*.media.net https://*.demdex.net https://*.smartclip.net https://*.bit.ly https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com http://*.resultspage.com
https://*.resultspage.com; script-src 'self' 'unsafe-inline' 'unsafe-eval' *.sli-r.com *.sli-spark.com *.resultspage.com *.resultsstage.com tagmanager.google.com *.googletagmanager.com
https://*.veinteractive.com http://*.avmws.com http://*.optimizely.com http://*.iesnare.com https://*.listrak.com https://www.googleadservices.com/ http://*.verisign.com http://*.googleapis.com
http://*.nr-data.net http://*.newrelic.com http://*.ywxi.net http://*.bazaarvoice.com/ http://*.cloudfront.net/ http://*.listrakbi.com/ https://seal.networksolutions.com/
http://*.google-analytics.com/ https://ping.chartbeat.net/ http://*.reflektion.com/ http://*.chartbeat.com/ https://*.listrak.com/ https://product.reflektion.com/
https://display.ugc.bazaarvoice.com/ https://www.google-analytics.com/ https://cdn.listrakbi.com/ https://*.verisign.com https://*.googleapis.com https://*.nr-data.net https://*.newrelic.com
https://*.ywxi.net https://*.bazaarvoice.com/ https://*.cloudfront.net/ https://*.listrakbi.com/ https://*.google-analytics.com/ https://*.reflektion.com/ https://*.chartbeat.com/
https://*.optimizely.com https://*.avmws.com/ http://*.zmags.com https://*.zmags.com https://*.dpmsrv.com http://*.dpmsrv.com https://*.adnxs.com/ http://*.adnxs.com/ https://*.doubleclick.net
http://*.doubleclick.net https://*.google.com https://*.gstatic.com/ https://*.symantec.com https://*.statsstory.com https://*.hotjar.com https://*.cloudfront.net https://*.jsdelivr.net
https://c.vepxl1.net https://c.flx1.com https://go.flx1.com https://*.youtube.com https://s3.amazonaws.com https://*.ytimg.com http://*.criteo.com https://*.criteo.com http://*.criteo.net
https://*.criteo.net http://*.resultspage.com https://*.resultspage.com https://*.sli-r.com; style-src 'self' 'unsafe-inline' tagmanager.google.com http://*.amazonaws.com/
http://*.reflektion.com/ http://*.googleapis.com http://*.bazaarvoice.com/ https://*.bazaarvoice.com/ https://*.amazonaws.com/ https://*.listrakbi.com http://*.listrakbi.com
https://*.symantec.com https://*.cloudfront.net https://*.criteo.com http://*.rfksrv.com https://*.rfksrv.com http://*.resultspage.com https://*.resultspage.com; frame-src 'self'
http://*.bazaarvoice.com http://*.listrak.com http://*.youtube.com/ http://*.zmags.com/ woobox.com https://*.bazaarvoice.com https://*.listrak.com https://*.youtube.com/ https://*.zmags.com/
https://*.listrakbi.com https://*.googleadservices.com http://*.googleadservices.com https://*.doubleclick.net http://*.doubleclick.net https://*.google.com https://*.google.ad
https://*.google.ae https://*.google.com.af https://*.google.com.ag https://*.google.com.ai https://*.google.al https://*.google.am https://*.google.co.ao https://*.google.com.ar
https://*.google.as https://*.google.at https://*.google.com.au https://*.google.az https://*.google.ba https://*.google.com.bd https://*.google.be https://*.google.bf
https://*.google.bg https://*.google.com.bh https://*.google.bi https://*.google.bj https://*.google.com.bn https://*.google.com.bo https://*.google.com.br https://*.google.bs
https://*.google.bt https://*.google.co.bw https://*.google.by https://*.google.com.bz https://*.google.ca https://*.google.cd https://*.google.cf https://*.google.cg
https://*.google.ch https://*.google.ci https://*.google.co.ck https://*.google.cl https://*.google.cm https://*.google.cn https://*.google.com.co https://*.google.co.cr
https://*.google.com.cu https://*.google.cv https://*.google.com.cy https://*.google.cz https://*.google.de https://*.google.dj https://*.google.dk https://*.google.dm
https://*.google.com.do https://*.google.dz https://*.google.com.ec https://*.google.ee https://*.google.com.eg https://*.google.es https://*.google.com.et https://*.google.fi
https://*.google.com.fj https://*.google.fm https://*.google.fr https://*.google.ga https://*.google.ge https://*.google.gg https://*.google.com.gh https://*.google.com.gi
https://*.google.gl https://*.google.gm https://*.google.gp https://*.google.gr https://*.google.com.gt https://*.google.gy https://*.google.com.hk https://*.google.hn
https://*.google.hr https://*.google.ht https://*.google.hu https://*.google.co.id https://*.google.ie https://*.google.co.il https://*.google.im https://*.google.co.in
https://*.google.iq https://*.google.is https://*.google.it https://*.google.je https://*.google.com.jm https://*.google.jo https://*.google.co.jp https://*.google.co.ke
https://*.google.com.kh https://*.google.ki https://*.google.kg https://*.google.co.kr https://*.google.com.kw https://*.google.kz https://*.google.la https://*.google.com.lb
https://*.google.li https://*.google.lk https://*.google.co.ls https://*.google.lt https://*.google.lu https://*.google.lv https://*.google.com.ly https://*.google.co.ma
https://*.google.md https://*.google.me https://*.google.mg https://*.google.mk https://*.google.ml https://*.google.com.mm https://*.google.mn https://*.google.ms
https://*.google.com.mt https://*.google.mu https://*.google.mv https://*.google.mw https://*.google.com.mx https://*.google.com.my https://*.google.co.mz https://*.google.com.na
https://*.google.com.nf https://*.google.com.ng https://*.google.com.ni https://*.google.ne https://*.google.nl https://*.google.no https://*.google.com.np https://*.google.nr
https://*.google.nu https://*.google.co.nz https://*.google.com.om https://*.google.com.pa https://*.google.com.pe https://*.google.com.pg https://*.google.com.ph https://*.google.com.pk
https://*.google.pl https://*.google.pn https://*.google.com.pr https://*.google.ps https://*.google.pt https://*.google.com.py https://*.google.com.qa https://*.google.ro
https://*.google.ru https://*.google.rw https://*.google.com.sa https://*.google.com.sb https://*.google.sc https://*.google.se https://*.google.vu https://*.google.ws
https://*.google.rs https://*.google.co.za https://*.google.co.zm https://*.google.co.zw https://*.google.cat https://*.symantec.com https://*.mcafeesecure.com https://*.veinteractive.com
https://*.hotjar.com https://*.youtube.com https://*.hotjar.com https://*.criteo.com *.dotomi.com http://*.rfksrv.com https://*.rfksrv.com; connect-src 'self' https://*.statsstory.com
https://*.hotjar.com wss://*.hotjar.com https://*.veinteractive.com https://*.servicebus.windows.net https://*.optimizely.com https://*.youtube.com https://*.criteo.com;
9643 bytes!
unsafe-inline
unsafe-eval
*.cloudfront.net
tagmanager.google.com
*.amazonaws.com

ExpecX-Content-Type-
Options
aka “Please actually believe my content type”
https://fetch.spec.whatwg.org/#x-content-type-options-header
Domains sending
20.0%
Introduced in
IE8
Almost always set to
nosniff

• You operate a site that accepts user uploads and provides a means of
rendering user content
• Attacker uploads a dangerous file, pretending to be something else
• Attacker somehow gets your site’s users to view that file
• Even though your server applies an image Content-Type, the browser
executes the file as script or renders a web page because it detects the
type.
"MIME confusion attack"

• Content-Type-Options only applies to script-like and style destinations
– Does not apply to navigations
– Browsers already refuse to sniff content on navigation
• Chrome and Firefox will refuse to sniff JavaScript if the advertised content
type starts with ‘image/’
• In practice, attack window is exceptionally small
However...
http://great-sausage.glitch.me/

Maybe prevent some very niche XSS vuln:

StricStrict-Transport-
Security
Always connect over TLS, even for first request
https://tools.ietf.org/html/rfc6797
Domains sending
11.0%
Standardised in
2012

Strict-Transport-Security: max-age=10886400;
Kill off insecure HTTP once and for all

AccepWhat to send in referrals from this page
https://w3c.github.io/webappsec-referrer-policy/
Referrer-Policy
Domains sending
2.4%
Standardised in
2017

Referrer policy options
Policy Referrer (same origin) Referrer (to foreign origin)
no-referrer omitted omitted
no-referrer-when-downgrade https://example.com/page.html?query https://example.com/page.html?query
origin https://example.com https://example.com
origin-when-cross-origin https://example.com/page.html?query https://example.com
same-origin https://example.com/page.html?query omitted

AccesAllow access to cross-origin resources
https://fetch.spec.whatwg.org/#cors-protocol
Access-Control-*
Domains sending
6.7%
Standardised in
2010

Access-Control-Allow-Origin: requesting-origin.example.com
Access-Control-Allow-Credentials: true
Access-Control-Max-Age: 86400
Access-Control-Expose-Headers: X-Debug
Access-Control-Allow-Headers: DNT, User-Agent
Allow authenticated cross-origin requests:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/frame-ancestors

AccepRequests client hints data be sent in future
http://httpwg.org/http-extensions/client-hints.html
Accept-CH
Domains sending
0.001%
Status
Proposed

Accept-CH: DPR, Width, Viewport-Width, Save-Data
Accept-CH-Lifetime: 86400
Request that client send all current CHs:
DPR: 2.0
Width: 320
Viewport-Width: 320
Save-Data: 1
Subsequent requests:

Proposed new client hints!
Sec-CH-UA: "Chrome 74"
Sec-CH-UA-Platform: "macOS 12"
Sec-CH-UA-Arch: "ARM64"
Sec-CH-UA-Engine: "Blink"
Sec-CH-Lang: "en-US", "en", "de"

LinkLink (preload)
Declare a resource that’s important early on
https://w3c.github.io/preload/#x2.link-type-preload
Domains sending
19.1%
Standardised in
2016

Link: <some-font-face.woff2>; rel="preload"; as="font"; crossorigin
Link: <main-styles.css>; rel="preload"; as="style"
Get your fonts and styles down sharpish:

https://calendar.perfplanet.com/2017/a-tinder-progressive-web-app-performance-case-study/

Status code must
be sent before headers

The status code problem
DNS
Lookup
TLS TTFB
Status code +
LINK headers
received
Database
Auth
Templating
API queries

EarlySends headers before status code
https://tools.ietf.org/html/draft-ietf-httpbis-early-hints-05
103 Early Hints
Domains sending
Unknown
Status
Proposed

HTTP/1.1 103 Early Hints
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 10:02:11 GMT
Content-Type: text/html; charset=utf-8
<!doctype html>
Get your fonts and styles down even sharper:

Preloaded resources start preloading earlier
DNS
Lookup
TLS
Early hints +
Link headers
received
Real status
received

ServeAllows exposure of granular server-timing data
https://www.w3.org/TR/server-timing
Server-Timing

Server-Timing: miss, db;dur=53, app;dur=47.2
Server-Timing: customView, dc;desc=atl
Server-Timing: cache;desc="Cache Read";dur=23.2
Tell the browser what happened on the server:

Server-timing metrics appear in developer tools

Wait, what...
A JavaScript API into a header
dictionary?

Server-Timing: customerID;desc=1234567, edition;desc="EMEA"
Server-Timing: paywallArticlesRemaining;desc=8
Tell the browser…. anything?:
performance.getEntriesByType('navigation')[0]
.serverTiming
.reduce((soFar, entry) => ({
...soFar,
[entry.name]: entry.description
}), {})
;
{customerID: "1234567", edition: "EMEA", paywallArticlesRemaining: "8"}

But server-timing might be unknown in headers
DNS
Lookup
TLS TTFB
Headers
sent
Only now we know:
Total bytes sent
Last byte timestamp
Client data rate

HTTP/1.1 200 OK
Date: Tue, 27 Feb 2018 13:28:47 GMT
Content-Length: 12
Server-Timing: miss, db;dur=53, app;dur=47.2
Trailers: Server-Timing
Hello world!
Server-Timing: sentduration;dur=234
Headers, but like, at the end?

FeatuSelectively adjusts browser capabilities
https://wicg.github.io/feature-policy/
Feature-Policy
Domains sending
0.001%
Status
Proposed

Feature-Policy: autoplay 'none'; speaker 'self'; unsized-media some3rdparty.com
Turn off the bad stuff
No-one can
autoplay video on
this page
Only I can use the
speakers. No noisy
ads, thanks.
Images only take size
from their contents when
loaded from
some3rdparty.com

What policies are available?
autoplay
camera
document-domain
encrypted-media
fullscreen
geolocation
microphone
midi
payment
vr

Feature-Policy: accelerometer 'none'; ambient-light-sensor 'none'; animations
'none'; autoplay 'none'; camera 'none'; document-write 'none'; encrypted-media
'none'; fullscreen 'none'; geolocation 'none'; gyroscope 'none';
image-compression 'none'; legacy-image-formats 'none'; magnetometer 'none';
max-downscaling-image 'none'; microphone 'none'; midi 'none'; payment 'none';
picture-in-picture 'none'; speaker 'none'; sync-script 'none'; sync-xhr 'none';
unsized-media 'none'; usb 'none'; vertical-scroll 'none'; vr 'none';
Today's tightest feature-policy (in Chrome):

OrigiMoves origin-wide headers into a file
https://wicg.github.io/origin-policy/
Sec-Origin-
Policy
Domains sending
0
Status
Proposed

HTTP/1.1 200 OK
X-XSS-Protection: 1; mode=block
Strict-Transport-Security: max-age=31536000; includeSubdomains;
Content-Security-Policy: default-src 'self'; report-uri https://csp.example.com/
Expect-CT: max-age=86400, enforce, report-uri="https://ect.example.com/"
Alt-Svc: h2="new.example.com:443"; ma=600
Link: /script/bundle.rev-983c15.js>;rel=preload;as=script;charset=UTF-8;
Accept-CH: DPR, Width, Viewport-Width
Feature-Policy: vibrate 'none'; geolocation 'none'; unsized-media ‘none’
<html>
...

{
"headers": [
{
"name": "Content-Security-Policy",
"value": "script-src 'self' https://cdn.example.com",
"type": "fallback"
}, {
"name": "Referrer-Policy",
"value": "origin-when-cross-origin",
"type": "fallback"
}, {
"name": "Content-Security-Policy",
"value": "object-src 'none'; frame-ancestors 'none'",
"type": "baseline"
}, {
"name": "Strict-Transport-Security",
"value": "max-age=10886400; includeSubDomains; preload",
"type": "baseline"
}, {
"name": "X-Content-Type-Options",
"value": "nosniff",
"type": "baseline"
}
],
"cors-preflight": {
"origins": "*"
}
}
/.well-known/origin-policy/policy-1

HTTP/1.1 200 OK
Sec-Origin-Policy: "policy-1"
Vary: sec-origin-policy
<html>
...

Integrity
Signature
Accept-CH
Early Hints
Sec-Origin-Policy
P3P
Expires
X-Cache
X-Frame-Options
Via
Recap
Feature-Policy
Content-Security-Policy
Strict-Transport-Security
Referrer-Policy
Link: rel=preload
Server-Timing
Clear-Site-Data
Expect-CT
X-Content-Type-Options

fastly.com/blog/
headers-we-dont-want

Thanks for listening
I am Get the slides:
Andrew Betts
@triblondon
abetts@fastly.com
fastly.us/headers

JS Fest 2019. Andrew Betts. Headers for hackers

Related slideshows

More Related Content

JS Fest 2019. Andrew Betts. Headers for hackers