Update
This is now live on Meta Stack Exchange and Meta Stack Overflow. Any bugs and feedback can be posted here as an answer.
I’m Ben and I’m a dev on the Teams team here at Stack Overflow - we're the team focused on building the private Teams experience on SO. I’ve recently been working on our post editing experience and I’d like to show off some preliminary work that’s coming to the network soon.
TL;DR
We’re switching our code block highlighting library from Google Prettify to highlight.js. All your favorite languages are still supported and you won’t need to change how you write posts at all. The only major change is how we render highlighted code blocks. In addition, we’re taking this opportunity to introduce our new highlighting theme as well. We’re rolling this out in stages, starting with MSE/MSO with other sites to follow. (See the FAQ at the bottom of this post for dates)
Some history on Prettify / code block highlighting
I tried to do some digging on when we first adopted Prettify, but it seems that its history goes allll the way back to site’s earliest days. The earliest reference I could find was from back in ‘08. I asked around internally too and the best answers I could get were along the lines of:
¯\_(ツ)_/¯ - Everyone
Ask Atwood - Dean
and
If I had to guess, it was something along the lines of "there's not a lot of options, and this is used by Google so it's probably fine" - Kevin
Eventually the wonderful Tim Post pointed me to Stack Overflow Podcast #11, aired June 2008, where Jeff and Joel talk about how incredible it was for the time and how Google uses it themselves for syntax highlighting in Google Code (RIP). They also put out a call for alternatives, which I’d have to assume came up short.
Why the change?
Google Prettify hasn't been under active development for a while, and was officially discontinued by Google in April, as you all have let us know repeatedly. This means that no new language syntaxes1 are being supported and that existing language syntaxes aren’t getting updated to support all their new features. It’s time to move on to something that supports modern front-end workflows (such as providing an npm package, for starters) and continues to evolve to meet the needs of developers.
What’s changing about how I write posts?
Absolutely nothing :). There is absolutely no change to how posts are written. We still support all the Prettify language aliases you know and love, along with the new aliases from highlight.js. However, we are not adding support for any new languages at this time, instead choosing to keep the initial changeset simple and aiming for current feature parity instead. All the current markdown syntax is still supported, along with determining code highlighting from tags and site defaults.
So what is changing?
The “only” changes are visual. We are updating the client-side code block renderer that styles your code in posts (Questions, Answers, etc) and in the editor preview. Syntax autodetection when a language is not specified should be much better overall, along with syntax highlighting coverage in general. The biggest outward facing change for the typical user is going to be our new theme (see below for details).
Why highlight.js? Why not…
Why did we pick highlight.js over Prettify? Well, first off, you asked for it specifically. More convincingly, it’s open source, actively maintained, and overall just a solid product.
We’re extremely concerned about perf here at SO (both on the client and on the server), so we needed to ensure that this major change on our hottest page on the site didn’t negatively impact our users. There was some prior investigation into highlight.js's perf back in 2016, but I figured we should give it another shot.
In our internal performance benchmarks highlight.js scored better than Prettify consistently across all browsers (except macOS Safari 13.1, where it was actually a bit slower)2. It is a tad heavier than Prettify3, weighing in at an extra ~17kB (over the wire) after including all the languages we support across the network. This extra weight gain was acceptable to us as a tradeoff for what we were getting in return.
Why did we pick highlight.js over other contenders? Simply put, it was the best option that served our needs. We needed a library that we could easily control for use in the browser (deferred loading, theming specific elements), while also being simple to consume via a npm package, not needing specific build steps or a special babel plugin to pull in only the parts we need. Additionally, we could run it on the server (via Node.js) to unify our syntax highlighting in our Stacks documentation, giving us a single syntax highlighter across our products. Also a major plus was the ability to tokenize the highlighting result for use in our new editor (stay tuned!).
What are some potential drawbacks?
The most obvious not-quite-a-drawback is that language autodetection is different from Prettify. In general, it will be much more accurate, but will possibly end up with a different result that what Prettify would give us. This isn’t so much a bad thing, as it is just a thing that might take some getting used to if you’re a Prettify power user.
As mentioned earlier, the overall code bundle size is a bit bigger too. The vast majority of users wouldn’t even notice the change, which would only affect the first fetch since the browser will cache the file locally for subsequent hits anyways.
The last item is a bit of a personal preference. highlight.js tends to not highlight punctuation, which makes it a bit less colorful than other highlighters. This is considered a feature. Not a deal breaker by any means, but something I should mention regardless.
Designing the new theme
To offer some insight into how the new theme was designed, I reached out to the author, principal design systems designer Aaron Shekey.
Since we’re upgrading, we wanted to take this opportunity to design a Stack Overflow-flavored theme that takes advantage of newer tech like CSS variables that are aware of both light and dark modes. While we’ve improved it over the years, it’s highly likely that the current production theme simply used the stock colors provided by Prettify.
We’d need a theme that could work in both light and dark modes, was informed by Stack Overflow’s branded colors, and introduced a bit more contrast throughout.
Thankfully, we weren’t starting from scratch. When we built our Stacks documentation, we’d spent some time making our Jekyll theme display code snippets that got pretty close to accomplishing those goals. However, this was before dark mode was a thing, and we’d only built a single theme that assumed a fixed dark background. We’d have to extend this theme to light mode and revisit contrast along the way.
Using the Stacks documentation as a playground, we’ve now got themes in both light and dark modes that look like Stack Overflow and add or maintain contrast levels. We did our best to accomplish a contrast level of AAA, with a few variables dipping into AA. You can see the exact measurements commented in our colors constants file.
Here are a few screencaps of the new theme taken from my local development environment (click on the images to expand them). You can preview more languages (with an easy dark/light mode toggle) over at the Stacks docs.
Before
After
FAQ
Q: When is the rollout happening?
A: We're planning to roll this out to meta.stackexchange and meta.stackoverflow on Thursday, September 10th. Rollout to the rest of the network is scheduled for September 24th, after the initial testing period. This is a soft rollout date, dependent on any bugs/feedback we get from the community during the testing period.
Q: What if I find a bug?
A: Report bugs in an answer (one per answer) to this question. We'll keep this open for a couple/few weeks (until Friday, October 2nd) to address any immediate issues and then we'll update this post and ask you to post bugs as new questions after that time.
Footnotes
1 I checked, plural of syntax is syntaxes. Take that spell-checker!
2 Client-side benchmarks being what they are, we measured anywhere from ~49%-60% increase in the rate of ops/second depending on the machine and browser. Outliers being Safari 13.1 which had a ~29% decrease (favoring prettify) and Edge “legacy” scoring a ~279% increase over prettify!
3 Size comparisons were done comparing the prettify-full.en.js file taken from production vs the new highlight.pack.js bundle. Both were minified and served via a webpack-dev-server instance with the compress flag set (enabling gzip support). They were then included onto a regular html page with script tags and measured using the built-in browser dev tools. At the time of measurement, prettify landed at 23.3kB over the wire (meaning that the file was minified + gzipped) vs highlight.js at 40.7kB. This is a 17.4kB increase or about a ~74% increase in file size.
